The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw

One of Max Drake's colleagues turned a whiteboard app into a window manager, then used it to play Pong with real desktop windows. Drake is a product engineer at tldraw, the London company behind the free infinite canvas app and, more importantly to him, the SDK powering other people's canvases, including Replit's. tldraw exists because people building canvas apps got stuck on the canvas itself, on selection and resizing and matrix math, and never reached their actual product. His talk is about the same trap one layer up. Coding agents work well because code is text in and text out, the medium they were trained on. Ask one to align something in two dimensions and it falls apart. Teaching agents to understand space takes real engineering. The progression starts with getting a model to read a canvas at all, from a screenshot plus the underlying JSON. Told in one shot to make a mouse blow out a candle, it produces wind and then smoke, positioned correctly, out of ordinary shapes. That becomes the agent starter kit, MIT licensed, where an agent sets its own todos and moves its viewport to go looking for things. Then come fairies, agents rendered as visible characters you can throw around and recolor. The whimsy is load bearing: with ten agents running you read their state by glancing at them rather than reading a chat log. Select several and you get a group chat, one of them acting as orchestrator, assigning work and waiting to review it. Last is a dependency graph where every node is a coding agent, and a desktop build that lets any agent script the editor directly. Speaker info: - https://x.com/max__drake - https://www.linkedin.com/in/maxdrake1/ - https://maxdrake.md Timestamps: 0:00 - A demo kicked off before the talk begins 1:08 - tldraw, the app and the SDK behind other canvases 4:14 - Why coding agents work and canvas agents struggle 5:08 - Teaching a model to read and act on the canvas 7:01 - The agent starter kit goes agentic 8:26 - Fairies, multiplayer agents you can see working 9:49 - A group chat of agents with an orchestrator 12:06 - One node per coding agent, in a dependency graph 15:48 - Code mode, a window manager, and Pong 18:05 - The canvas as a place to work with agents