File analysis and curation

File analysis helps interpret material whose filename or extension does not explain its role. Avichive combines deterministic path rules, bounded content sampling, structured model output and owner review. It records evidence and decisions on deduplicated content so identical historical copies can share the result.

This page describes source inspected on October 13, 2026, at application revision e76e74c. It establishes an implemented workflow, not whole-archive analysis coverage, measured model accuracy or a newly verified production run.

Recognize reproducible artifacts first

A personal backup includes more than personal work. Installed dependencies, build output, caches, logs and OS metadata can overwhelm useful material. JunkPatternMatcher recognizes configured path patterns and records a reason rather than sending every obvious artifact through a model. Examples include node_modules, compiler output and application caches. Pattern rules (app/services/junk_pattern_matcher.rb:4).

These are heuristics. An invented file beneath Projects/site/node_modules has different evidence from a hand-written file beneath Projects/site/src, but a directory name is not a universal judgment of historical value. The recorded source of a decision lets the owner distinguish a pattern match from a human review.

Send a bounded sample

The batch submitter provides filename, extension, size, nearby folder context and magic bytes. It samples the first 4,096 bytes, adding at most 1,500 characters of decoded text when the header looks textual. Uploaded originals can be sampled with a cloud range read; otherwise an available local canonical file supplies the sample. Sampling and prompt construction (app/services/file_analysis_batch_submitter.rb:124).

The model is asked for a structured result: type, owning application or project, keep/skip/uncertain recommendation, integer confidence and a short description. The inspected implementation versions the prompt and pins a model; those are reproducibility details, not permanent promises about provider offerings. Response schema (app/services/file_analysis_batch_submitter.rb:15).

Sensitive files are excluded from external analysis using scanner and linked-document markings. That protection is meaningful, but it is not proof that all private material has been detected. A content sample differs from the era-analysis workflow, which sends bounded metadata and approved context clues without reading file contents.

flowchart LR
  F[Unique content] --> R[Path rules]
  F --> S[Eligible bounded sample]
  S --> A[Structured AI result]
  R --> D[Recorded decision and source]
  A --> C{Keep or skip with confidence at least 95?}
  C -->|Eligible| D
  C -->|Otherwise| H[Owner review]
  H --> D

Keep category, evidence and decision separate

The scanner category describes the original inventory. AI suggested type and owning app describe an interpretation. archive_decision records keep or skip, accompanied by decision source and timestamp. They answer different questions and should not be treated as interchangeable fields.

An AI keep/skip result with confidence of at least 95 can be automatically applied. Human decisions are protected from automatic overrides. Only a limited mapping fills a missing or unknown category: source becomes code, while config, data and document map directly. Confidence is supplied by the model; 95 does not mean the classifier has demonstrated 95 percent accuracy. Decision rules (app/services/analysis_decision.rb:20).

Review and processing state

Stored batches and items track submission, processing, results and failures. A results job parses each response and applies it to UniqueFile; grouped review helps the owner inspect unresolved recommendations by application and recommendation. Result processing (app/jobs/process_batch_results_job.rb:49), review controller (app/controllers/analysis_reviews_controller.rb:15).

A skip decision does not delete a source file, and curation need not precede preservation. Originals may already be uploaded when analysis runs. The workflow makes interpretation reviewable while retaining the content identity and provenance needed to revisit a decision later.