It’s Tokens All The Way Down: How RLMs are Different — Kevin Madura, AlixPartners

Ask a model to sum twelve numbers scattered across thirty thousand tokens and it will often get it wrong. Give it a Python REPL and it writes a few lines of regex instead. That is the shift Kevin Madura, director of advanced technology at AlixPartners, describes in recursive language models. An RLM treats its context as an object living in a REPL rather than as tokens it must attend to, so it can slice, compute over, and iterate on the input as a variable. The second property matters as much: it can delegate to another model, including itself, with its own parameters, so a hard problem decomposes recursively and only the results that matter return to the main context. On a long chain of thought benchmark that moved accuracy from 2.6 percent to 45.4 percent, with the largest gains on tasks that reduce cleanly to code. Madura frames it against what most teams do now. RAG stuffs the window until quality rots. Agents and tool calls shuttle JSON strings back and forth, leaving logic, execution, and results loosely coupled. An RLM keeps all three in one environment. He walks a cohort retention analysis where three data frames go in, the model reasons in its own REPL as if typing in a notebook, and decides itself when to stop and submit a typed answer. Then case studies: consolidating long invoices with no chunking or embedding, surfacing patterns in raw logs, optimizing an agent harness from its own traces, and generating a security report across five hundred thousand lines of code. His closing bet is that models post trained to be RLM aware will make this much stranger. Speaker info: - https://x.com/kmad - https://www.linkedin.com/in/kevinmadura/ - https://kmad.ai Timestamps: 0:00 - What a recursive language model is 1:36 - Recursive decomposition and delegating to sub models 3:25 - Benchmarks and the cost curve 4:31 - A deterministic shell, with the model filling the middle 5:24 - Context rot, and why an RLM avoids it 6:21 - How this differs from RAG, agents, and tool calls 8:00 - When to use one, and when to skip it 9:38 - Small examples, buried numbers and data frames 12:35 - A cohort analysis walkthrough and its traces 15:45 - Case studies, invoices, logs, and a whole codebase 20:04 - What happens when models become RLM aware