Today, preparing a new benchmark to be indexed for our new RAG evaluation task. I asked the agent to write the corpus transformer and it came back clean: parsed the XML, mapped every field, ran green. On a small dataset I'd have shipped it.
I built Trivi because I kept skipping words.
Just unfamiliar ones. Terms I half-knew, expressions I'd seen but never pinned down, bash commands I couldn't quite remember. Every time I hit one, the same calculation ran in my head: Is it worth breaking my flow to look this up? The answer was usually no.
I keep noticing this new pattern in how I work, think, and make decisions. Recently, it showed up so clearly that I felt it was worth writing down.
I was trained to think economically about software work. Count your operations, mind your memory, justify your complexity. Big-O thinking does not stay in the algorithms course, it shapes how you approach everything: automate and optimize. Now I keep automation and I learn to drop optimizing. I have a vision for how I want to build and evaluate, and for the first time the friction is low enough to actually follow it.
Trust in AI is built on transparency. Every extracted field carries a citation back to the exact source passage, and the full extraction set is scored against a benchmark. They answer different questions, at different moments. The benchmark answers whether the pipeline is good enough to ship; that is my question, once, before release. The citation answers where a specific number came from; that is the user's question, every day after.
Most of the worry around large language models lands on hallucinations, meaning incorrect information. Achieving 100% precision is a prerequisite for any financial data system. According to my evaluations for professional-grade extraction with LLMs, the harder challenge is recall. If an LLM encounters a complex website structure or a 50-page legal document, it misses details buried deep in the text.
High-quality data is often a point of pride for any company collecting them. In a recent engagement, the client had built a reliable database through a rigorous, labor-intensive process where mid-level staff reviewed every entry made by junior analysts. Even the most careful manual process eventually hits a wall: the velocity of information. In investments, data ages rapidly, and a manual team cannot scale their output to keep pace with the market without a linear increase in headcount.
So, Jupyter Notebooks. We had some good times, didn't we?
Writing code and seeing the results instantly felt great. But over time, the quirks I once found charming became... less so. Navigating between cells to find code? Frustrating. Forget to run a cell, and dependencies laugh in your face. And the workflow... Each function, each step in an experiment, means executing cells manually, at a time. It felt like trying to bake a cake but having to preheat the oven separately for every ingredient.