By Starktronix EngineeringNotes from our own systems
Listen to this article · 2 min
We hold a corpus of three-dimensional scans built up over a long period. Recently we went looking for something in it and discovered what we had not recorded.
Not the scans — those are intact. The conditions. What the lighting was. Which device. Which capture mode. What the operator was trying to achieve. Every scan in a large stretch of the corpus carried the same default value in the field that was supposed to distinguish them, because nothing ever forced it to be set and nobody noticed that nothing had.
The asymmetry
Raw capture data is remarkably forgiving. Reconstruction improves, formats change, a better algorithm arrives and you reprocess everything from the originals. The pixels keep their value indefinitely.
Capture context has no such property. It exists for the duration of the capture and then it is gone. There is no algorithm that recovers the ambient lighting from the mesh, no future model that infers operator intent from a point cloud. You either wrote it down at the time or you did not, and if you did not, that information does not exist anywhere in the universe.
Pixels
Recoverable indefinitely
Reconstruction improves and algorithms change; you can always reprocess the originals.
Context
Gone the moment capture ends
No algorithm recovers ambient lighting or infers operator intent after the fact.
Every other kind of data loss is expensive. This one is total.
How it happens
Not through carelessness. It happens because capture metadata has no immediate consumer. Nothing breaks when it is empty. No screen shows it, no test asserts it, no user complains. It is pure deferred value, and deferred value loses every argument against shipping something today.
The specific mechanism in our case: a field with a sensible default, populated automatically, never surfaced in the interface. It looked populated in the schema. Every row had a value. The value was the same value.
No immediate consumer
Nothing breaks when the field is empty; no screen shows it, no test asserts it.
Sensible default, auto-populated
A default value filled the field automatically, never surfaced in the interface.
Looks populated
Every row had a value, so the schema looked complete even though it was the same value throughout.
What we do now
- Capture-time fields are required at capture time, not backfillable afterwards.
- A default that is identical across every row is treated as an empty column, and monitored as one.
- Anything genuinely unrecoverable is written down before the feature that produces it ships, not after.
The corpus is still the most valuable thing we have. It is simply less valuable than it would have been, by an amount we can estimate and cannot recover. That is a fair description of the cost of finding out late.