From Ingestion to Agents: How AI Teams Build on Document Intelligence — Adit Abraham, Reducto

The newest frontier model scores about thirty percent on a data lab's benchmark of decisions from PDFs, and Adit Abraham has met people who worked on PDF processing before he was born. The format was built to print, not to be reasoned over, and humans encode meaning visually: merged cells, line charts, unreadable handwriting. Reducto has processed billions of them, and the talk is the lessons, not the product. RAG meant a bad parse cost one answer; with agents, bad inputs compound across every step. VLMs finally read the long tail like a human, but they are not one size fits all: small detectors still find layout on a CPU at scale, and a VLM asked to rewrite OCR will helpfully recompute a total the human got wrong. His agentic OCR applies token level corrections, a zero for an O, instead of regenerating the page. Simple tables go to markdown and complex ones to HTML, but embedding models cannot match how did revenue change to a blob of tags, so a natural language rendering serves retrieval. Parsed structure rather than raw PDFs lifted other frontier models past the newest one on that benchmark and cut reasoning tokens. Classification and splitting keep a hundred page mail packet from distracting it. Agent harnesses crack problems no model solves in one shot, turning a line chart into a data table with a code interpreter and repeated self checks, and they beat a trade off a new benchmark exposed, where frontier models are precise but silently drop rows and document services do the reverse. He closes on evals at every stage and customers who give agents a file system, not a fixed pipeline. Speaker info: - https://reducto.ai Timestamps: 0:00 - Reducto, and the less sexy part of agents that work: data 2:04 - From RAG to agents: bad inputs compound across steps 5:28 - Why PDFs are still hard, and a benchmark frontier models fail 7:19 - Traditional CV versus VLMs: the right place for each 9:07 - Agentic OCR: token level correction, not rewrites 10:42 - Tables: markdown versus HTML, and a form for the embedding model 13:15 - Better inputs lift frontier models and cut reasoning tokens 14:41 - Classification and splitting as orchestration 16:17 - Agent harnesses: line charts to tables, precision and recall together 18:57 - Evals at every stage 20:06 - Where this heads: a file system and CLI for agents 21:00 - Six takeaways