Lessons and evolution

The first version made a scattered archive searchable. Later work made its contents preservable, interpretable and usable again. The lasting lesson is that each step needs its own evidence and failure boundary: finding a file, uploading bytes, generating a preview and confirming a person are different accomplishments.

Discovery was only the beginning

The February 2026 guide described roughly 3.5 million files and a large duplicate-storage estimate. Go workers, streaming hashes and bulk database writes made that inventory practical. But the inspected scanner skips known paths, logs failed insert batches without retrying them, and can retain an observation without a successful hash. A completed progress display therefore does not establish complete ingestion or continuing filesystem accuracy. Original scanner (scanner/main.go).

Derived directories and typed models made the inventory usable without forcing every request to traverse millions of observations. Content identity then gave repeated paths a shared preservation target. The architecture gained value from separating what was observed from what was later decided.

Preserve bytes and make them usable

July 2026 operational records describe completion of the canonical-original upload campaign. Treat that as a dated checkpoint rather than a live total. The current original service supports cloud reads, source verification and bounded materialization, allowing application processing to use preserved content. Upload history (archive-browser/docs/ARCHIVE_UPLOADS.md), original service (archive-browser/app/services/archive_originals.rb).

The distinction between originals and derivatives became essential. Browser images and playable video solve access problems; they do not replace source preservation. Database packages preserve another necessary layer: paths, identities, curation and relationships. Losing that catalog would lose information even if all original objects survived.

Human context needs explicit authority

September curation work added recorded keep/skip decisions and grouped review. A confidence threshold is a rule for applying model suggestions, not measured model accuracy. Human choices survive automatic analysis. Decision implementation (archive-browser/app/services/analysis_decision.rb).

Era membership and photo identity extend this principle. Eras overlap and use explicit inclusion/exclusion. Local face detection produces observations; names and experimental match candidates require owner decisions. These are implemented workflows, while captioning and archive-wide semantic retrieval remain separate research. Life eras, photo analysis.

Synchronization must respect local work

October sync improvements use consistent export snapshots and advance the routine watermark after a confirmed import. Production derivatives remain local to production; authoritative era edits travel back to the development mirror; people annotations use a separate replacement path. A one-way database-copy diagram cannot explain this system accurately. Operations record (archive-browser/docs/DEPLOYMENT.md).

Long-running jobs also need a suitable process boundary. The deployment guide records a dedicated GoodJob worker after a process/fork issue, reinforcing that background execution has an operational contract beyond a controller enqueue call.

Restoration supplies the payoff and its limits. Surviving data can become readable writing, a frozen social wall or a reconstructed montage, while missing services and assets remain missing. Restoration explains what was recovered and what was rebuilt. The guide’s October 13 source review establishes these implementations; it does not claim fresh production coverage, timing or recognition-quality measurements.

Implementation notes refer to repository sources reviewed at e76e74c on October 13, 2026.