Ghosts in the Data

Old facts can keep influencing automated systems long after everyone believes they are gone. Signal hygiene is the discipline that makes memory safe to trust.

There’s a story I use when I onboard people to automated systems, and I always flag it the same way: this one is fiction. It’s a composite, assembled from failure patterns every operator eventually meets. I tell it because it lands harder than a diagram.

A small team runs a decision engine. It reads facts from shared context, weighs them, acts. In the fictional setup, the team retires an outside reference source, updates the docs, holds the retro, closes the ticket. Later the engine makes a call no one can explain. The trace runs backward and stops at an intermediate copy, where the retired reference’s final snapshot sits: well-formed, dated, confidently wrong. The engine had been consulting a fact from a world that no longer existed. Nothing crashed. No one lied. The data was still there, and it was still believed.

Systems remember more than you intend

Automated systems accumulate memory in places their operators stop watching. Intermediate copies. Merge points. Replays. Restored records that were never fully unwound. Each one is a spot where a fact can outlive its truth, and when that happens the system starts to feel haunted in the specific, non-supernatural sense: something old is still exerting influence on decisions, and the people responsible believe it’s gone.

The gap between believed gone and actually gone is the whole subject here. In practice it takes three forms, and each one deserves a plain-English name.

• STALE CONTEXT

  An old truth is still treated as current. It has the right shape, clears technical checks, and quietly describes a world that moved on. Demote facts past their useful window from current to historical.

• DUPLICATE ECHOES

  One event is heard twice. Each record may be true, but the aggregate doubles conviction without doubling evidence. Declare join/match rules and reconcile what entered against what was used.

• DELAYED AND PHANTOM ARRIVALS

  Late records arrive outside their useful window; half-finished or placeholder records look real until inspected. Inspect arrivals intentionally and make review trails answer what the system believed.

Staleness is dangerous because it looks healthy. Stale data passes validation, has the right shape and types, and carries an old date that everything downstream ignores. You’ll find no error to fix, because nothing is erroring.

Duplication doubles conviction without doubling evidence, and conviction is what an automated decision-maker runs on.

Late data and phantom records share the classic ghost’s defining trait — visible only under direct inspection, in systems that rarely inspect anything on purpose.

Exorcism is janitorial

No single heroic debugging session banishes any of this. The teams that stay clean treat hygiene as scheduled, owned, recurring work — the operational equivalent of changing the filters whether or not the air smells wrong. Four habits do most of the work.

1. Clean joins

   Every merge point declares how records match and how time is handled, in writing, where a reviewer will see it.

2. Context expiry

   Facts carry a shelf life. Past it, a fact is demoted from current to historical: still available for analysis, no longer eligible to drive a decision.

3. Reconciliation

   On a calendar, count what entered the system against what the system used. Gaps expose delayed arrivals. Surpluses expose duplicates and phantoms.

4. Review trails

   Every automated decision traces to the exact facts it consumed, ages attached. The trail should answer what the system believed and how old that belief was.

Memory earns its keep by staying clean

The previous essay in this series argued that memory is infrastructure — that a system with access to its own history can reason with context instead of twitching at isolated instants. Echo, the historical-context discipline behind this series, rests on that premise. This essay is the maintenance contract attached to it.

History is an asset only while every remembered fact stays dated, deduplicated, and reviewable. A memory layer that kept everything and trusted everything would amplify all three ghosts at once: stale beliefs persisting by design, duplicates accumulating, phantoms acquiring the authority of the archive. Signal hygiene is what lets a system trust its own past.

The satisfying ghost story ends with a confrontation at midnight. The operational one ends with a checklist, a calendar, and a person whose job includes reading reconciliation reports on ordinary weeks when nothing looks wrong. That’s the honest ending and the only durable one.

Old data never announces itself. It waits in whichever corner of the system stopped being watched, and it keeps influencing decisions for exactly as long as the discipline lapses. Keep the joins clean. Expire what has aged out. Count what went in against what came out the other side. Make every decision able to name the facts beneath it. Ghosts persist in systems that assume the past will keep to itself — so build systems that keep checking.