How to Point a Coding Agent at 400 Years of Archives: What to Verify

If you are a researcher, journalist, or hobbyist wondering whether a coding agent and a single GPU can credibly surface new findings in a digitized archive, the most defensible answer from the available evidence is: yes as a lead generator, no as an oracle. The workflow at the center of the current discussion is Antiquity, a small toolkit open-sourced on October 8, 2026 by software engineer Jesse Waites, who describes it as enabling “anyone with a question and a coding agent” to run archival investigations (Waites’ post). The compute claim, a single GPU over a few days, is project-reported, as is the toolkit’s maturity: no independent third-party usage or review of Antiquity appears in the available sources. What does survive scrutiny is the protocol Waites used, and that protocol is the part worth reproducing. Everything it produces stays a candidate lead until a human traces it to a scanned primary page and, for anything you would publish, until a specialist confirms it.

That distinction, between a verified workflow and validated findings, is the spine of what follows.

What Breen actually found, and why it started this

On October 1, 2026, historian Benjamin Breen published an account of using an AI model to search Dutch East India Company records transcribed by the GLOBALISE project, and surfaced what appears to be a new eyewitness record of the dodo: a 1615 ship’s journal from Mauritius in which sailors “caught many tortoises, dodos” (Waites’ post). The archive context matters for calibrating the “400 years” framing: the VOC operated from 1602 until its dissolution in 1799 (Waites’ post), and GLOBALISE has converted millions of its handwritten pages into searchable text. So the records span roughly two centuries, with individual documents like the dodo journal sitting unread for about four hundred years.

The significance is not that the archive was hidden. It was digitized, transcribed, and searchable. The significance is that search engines answer queries, and nobody had asked the right question. An archive only answers the questions someone thinks to ask, which is precisely the bottleneck an agent can relieve and precisely the place an agent can mislead you, because a fluent answer to a good question feels like a finding whether or not it is one.

The Antiquity pipeline, as described

Waites’ account documents a sequence that is more deliberate than “point a chatbot at a corpus.” He began by asking an AI Deep Research assistant for open historical questions that could be solved with online data, and it returned thirteen candidates. The ranking criterion is the instructive part: candidates were weighted most heavily by whether an answer could be checked against an original page. The shortlist included animals in the VOC archives, a giant 1808 volcanic eruption nobody has located (Waites’ post), felt earthquakes in old Dutch newspapers, unrecorded meteorite falls, and vanished ships.

From there, the pipeline ran over the transcriptions and produced candidate matches, each of which Waites then steered, filtered, and either pursued or dropped. He is explicit that the work was AI-assisted rather than autonomous: he chose the questions, chose the sources, chose which leads to chase, and decided when to stop.

His headline results, all author-reported and, in his own words, “not yet reviewed by the specialists who maintain those catalogues” (Waites’ post): over a few days on a single GPU, the pipeline surfaced a forgotten meteorite, three lost rhinos, and several unrecorded eruptions. Treat those as the output of an interesting instrument, not as history. The reason to take the instrument seriously is the two verification gates built into it.

Gate one: prove the search can find what is already there

Before any novel search, Waites required the pipeline to recover events known to be in the records. In his words:

“Before you trust a search that finds nothing, you have to prove it can find something you already know is there. So before I went looking for anything new, the pipeline had to find Breen’s dodo, the Laki eruption of 1783, Tambora in 1815 and a dozen other known events.” (Waites’ post)

He reports the controls passed. This is the single most transferable idea in the whole project, and it protects against the most dangerous output an archive search can produce: the honest-sounding negative. “The archive contains no record of X” is only evidence if the system demonstrably retrieves the records that are there. A pipeline that misses Tambora will also miss your eruption, and you would never know.

The same logic shows up, in a harder form, in the document-forensics literature. The MIDV-DynAttack dataset for testing identity-document attack detectors is published with no training or validation splits at all: “every attack not seen during training,” explicitly so detectors are evaluated only on cases unseen during setup (arXiv:2607.06466). The principle generalizes cleanly to archives: hold out your known events when you configure the search, then test against them. If you tune the pipeline until it finds Laki and then declare victory, you have tested your tuning, not your search.

Practical implication for a re-run: pick a dozen known events before you write a single novel query, keep them out of any prompt engineering or threshold adjustment, and treat a failed control as a failed pipeline rather than an empty archive.

Gate two: every claim ends at a scan

The second gate is grounding. Waites’ rule:

“AI models can be confidently wrong, so every claim had to end at a real document: a scan of the actual handwritten letter or the actual printed newspaper, with an archive reference that anyone can look up and read for themselves.” (Waites’ post)

This is the rule that separates an archival search from a language model’s prior. The model’s job is to nominate pages; the page is the evidence. A candidate claim that terminates at a transcription excerpt, a summary, or the model’s own paraphrase is not a find. It is a pointer.

Note what this gate does and does not accomplish, because the available evidence is blunt about it.

What grounding buys you, and what it does not

Requiring a real citation eliminates one class of failure while leaving another standing. A useful quantified demonstration comes from clinical ICD-10 coding, where a neuro-symbolic system with hard output constraints drove Type I hallucination (syntactically invalid codes) to 0% (arXiv:2512.23743), while GPT-4 on the same 5,000-case evaluation produced an 18% Type I rate at temperature 0.7 and a 9% rate even at the deterministic temperature 0.0 setting (arXiv:2512.23743). But the constrained system’s Type II hallucination, codes that are valid but wrong, persisted at 12% (arXiv:2512.23743).

The mapping to archival work is direct. Gate two is an output constraint: it forces every claim into the shape of a citable document, which should push the “invented reference” failure toward zero. It does nothing about the claim being valid-but-wrong: a real page, correctly cited, that does not actually say what the pipeline thinks it says, or that says it in a context that reverses the meaning. Expect that residual error class in any pipeline, including a well-built one. The figure comes from a different domain and should not be treated as Antiquity’s error rate, which has not been independently measured. It is evidence that structural verification does not retire semantic checking.

There is also a boundary lesson in how to describe an open-sourced workflow. The DANTE gravitational-wave analysis release states plainly that it “verifies adopted scientific artifacts rather than recomputing them, and therefore makes no new claim of global significance, astrophysical discovery, or public real-time operation” (arXiv:2609.08695). That is the honest template. Antiquity’s release documents that a workflow exists; whether it re-runs cleanly on your hardware is exactly what a first control-pass run establishes. It does not validate any find the workflow produces. If you reproduce the pipeline and it behaves as described, you have confirmed the instrument, not the meteorite.

The weak links live in the documents

Two further risks sit underneath both gates, and they belong to the archive rather than the agent.

First, the transcription layer. Machine reading of aged, degraded pages has been an open research problem for a long time. A 2007 survey of the field notes that “because of the low quality and the complexity of these documents (background noise, artifacts due to aging, interfering lines), automatic text line segmentation remains an open research field” (arXiv:0704.1267). Antiquity searches GLOBALISE’s transcriptions, so its recall is capped by the quality of text it never sees: a misread line is invisible to any query. Gate one partially covers this, since known-event controls exercise the transcription layer too, but only for the regions of the archive your controls touch.

Second, if your target archive is not already transcribed and you plan to do layout analysis and OCR yourself, budget for annotation. Layout models are label-hungry: in semi-supervised experiments on DocLayNet, mean average precision was 74.3 with 100% of annotations available versus 41.3 with only 10% (arXiv:2305.00795), a drop of more than 30% under annotation scarcity. And much of the tooling is validated on digital-born documents rather than scans; the PAL layout database of about 37.9K documents and 441K-plus pages states that “all the documents we collected are PDF native format, that is, they are not scanned images of a document” (arXiv:2306.10046). Performance on clean native PDFs does not transfer automatically to a 1690s ship’s journal with bleed-through.

There is at least a standardized way to measure where you stand: an open, LGPL-licensed pixel-level evaluation tool was used for the ICDAR 2017 competition on layout analysis of challenging medieval manuscripts, giving reproducers an established metric harness for segmentation quality on historical material (arXiv:1712.01656). If you are preprocessing your own scans, measuring your segmentation against a hand-checked sample is the analogue of Waites’ known-event gate, one layer down.

The confirmation gap is the current state of every headline find

Here is where the story stands, and the wording matters. Of the meteorite, rhino, and eruption candidates, Waites writes they are “not yet reviewed by the specialists who maintain those catalogues. The newspaper oddities are separate, unchecked leads. I’ve contacted the meteorite curators, Kees Rookmaaker, and the volcanologists, and I’m waiting for their replies” (Waites’ post).

Pending replies are absence of evidence, not rejection. But they are also not confirmation, and on Waites’ own account these are candidates rather than discoveries. Each candidate was checked against original pages and accessible catalogues, which clears gates one and two. The remaining gate is the one no pipeline can pass on its own: a specialist who knows the catalogue well enough to say whether a record is genuinely absent from it. Novelty is a claim about everything else humanity has noticed, and an agent cannot search that.

A verification checklist for your own run

The table below maps each stage of the workflow to what it establishes and what it cannot. The “required check” column is the discipline the evidence supports; treat it as the minimum before citing anything.

Two budget notes before you start. The “single GPU, a few days” envelope is project-reported for Waites’ specific corpus set, the GLOBALISE transcriptions plus Dutch and American newspapers and ship logbooks; whether it transfers to your archive and hardware is unknown, and the honest move is to treat the first run as a calibration exercise whose deliverable is a control-pass report, not a finding. And if your archive lacks quality transcriptions, the labeled-data numbers above suggest the preprocessing phase, not the agent phase, will consume the budget.

My judgment

I would reproduce this, and I would start with the controls. The protocol Waites documented, rank questions by checkability, gate novelty behind known-event recovery, force every claim to terminate at a scan, is a sound piece of engineering hygiene, and the independent literature supports each element: held-out evaluation from document forensics, the Type I/Type II split from constrained decoding work, and the segmentation and annotation warnings from the layout-analysis field.

What I would not do is publish a find on the strength of the pipeline. The meteorite, the three rhinos, and the unrecorded eruptions are, as of October 10, 2026, one researcher’s checked-but-unconfirmed leads with specialist replies pending. If those confirmations arrive, the story becomes about the finds. Until then, the story is about the method, and the method’s actual contribution is narrower and more durable than the headline: it lowers the cost of generating archival hypotheses to a laptop-scale budget, while leaving the cost of establishing them exactly where it always was, on a scanned page and a specialist’s reply.

Frequently Asked Questions

How do you verify an archival search pipeline before trusting its results?

Before you trust a search that finds nothing, you have to prove it can find something you already know is there. So before I went looking for anything new, the pipeline had to find Breen’s dodo, the Laki eruption of 1783, Tambora in 1815 and a dozen other known events.

What is the final verification step for a candidate archival find?

AI models can be confidently wrong, so every claim had to end at a real document: a scan of the actual handwritten letter or the actual printed newspaper, with an archive reference that anyone can look up and read for themselves.

What is the current status of the meteorite and rhino findings?

not yet reviewed by the specialists who maintain those catalogues. The newspaper oddities are separate, unchecked leads. I’ve contacted the meteorite curators, Kees Rookmaaker, and the volcanologists, and I’m waiting for their replies