Element present ≠ content present

Today, preparing a new benchmark to be indexed for our new RAG evaluation task. I asked the agent to write the corpus transformer and it came back clean: parsed the XML, mapped every field, ran green. On a small dataset I'd have shipped it.

But it's 400k+ documents, and I am always suspicious about human-produced content, so "looks correct" is not an option.

The AI had read a handful of documents and derived a schema from them. That schema is an assumption, so we ran a preflight over the whole corpus before transforming anything.

Every document had the content element we expected: 100% present. A "does this field exist?" check passes all of them. About 1 in 6 had it present and empty, the text sitting in a second element the transformer never read.

Then we redesigned the whole preflight, checked and challenged all the assumptions, discussed and agreed on edge cases and finally after several preflight iterations had the robust transformation plan, and the transformer got written around that. The skill out of this work:

first assumptions → preflight → analysis → strategy → transform