By Starktronix EngineeringNotes from our own systems
Listen to this article · 3 min
We shipped a connector feature with a full green suite behind it. Every handler was covered. Every assertion passed. It did not work, and it did not work in the most ordinary way possible: you clicked the thing, and nothing happened.
The tests called the handlers directly. `onPointerDown(fakeEvent)` then `onPointerUp(fakeEvent)`, coordinates identical, assertion green. What a real person does with a real trackpad is different. A human click is not a point — it is a short, noisy drag. Two, three, sometimes five pixels of jitter between press and release, because fingers are attached to bodies and bodies move.
The guard that was doing its job
Sitting between the handlers was a drag guard: if the pointer moved more than three pixels between down and up, treat it as a drag rather than a click. Sensible. It existed because an earlier bug had users accidentally creating connectors while panning.
The synthetic events moved zero pixels, so the guard never fired and the tests never saw it. Real clicks moved four, so the guard fired every time and swallowed the interaction. The suite was not testing the feature. It was testing a version of the feature that only exists inside the suite.
Synthetic test event
onPointerDown then onPointerUp fire at identical coordinates, moving zero pixels.
Drag guard checks movement
Movement over three pixels between down and up is treated as a drag, not a click.
Zero pixels: guard stays quiet
The synthetic click never triggers the guard, so the test passes.
Real click: guard fires
A real click moves about four pixels, so the guard treats it as a drag and swallows it.
A test that constructs its own input is testing your assumptions about the input. That is a different thing from testing the code.
What we changed
Verification now dispatches real events into a real document, through the same event path a browser uses. It is slower and more annoying to write. It also caught three more issues in the first week, all of the same family: a focus trap that only trapped synthetic focus, a scroll handler that never fired because programmatic scrolling does not always emit a scroll event, and a keyboard shortcut that had been broken for a month behind a passing test.
Issue 1
Focus trap
Only trapped synthetic focus, not real focus from an actual browser interaction.
Issue 2
Scroll handler
Never fired because programmatic scrolling does not always emit a scroll event.
Issue 3
Keyboard shortcut
Had been broken for a month behind a test that still passed.
The general rule we took from it: the further your test input is from the real input, the more of your test is fiction. Mocking the thing under test is the clearest version of this mistake, but calling a handler directly is the same error wearing better clothes.
The uncomfortable part
The suite had been green for weeks. Nobody had clicked the button in that time, because the suite was green. Coverage told us we had tested it; the coverage was real and the conclusion was wrong. That is the failure mode worth naming — not an untested feature, which everyone knows to worry about, but a confidently tested one that nobody has actually used.
We now hold one rule above the others when calling something done: someone has to have used it, through the real interface, in the real environment. Green is a prerequisite, not evidence.