Autoresearch Made Our Models 3x Faster — Tejas Bhakta, Morph

GPU kernels are a near-perfect target for autoresearch, because they're easy to verify: correct, and fast or not. Tejas Bhakta, founder of Morph and formerly on inference optimization at Tesla, shows how his team combined agent-written custom kernels with bare-metal hardware tweaks to make models three times faster on cheaper GPUs. His rule is simple: humans bring the ideas and autoresearch does the tuning. Agents are great at picking block sizes and parameters but bad at the big idea, like spotting that a DeepSeek attention step is loading far more context than it needs. He covers the context an agent needs about the hardware and the model, and the reward hacking to watch for, like disabling CUDA graphs to speed up one kernel while slowing the whole model. He also explains how kernel gains stack, and how bare-metal tweaks add about 25% over a cloud VM. His honest warning: roughly 80% of what autoresearch tries is bad. Speaker info: X/Twitter: @tejasybhakta (https://x.com/tejasybhakta) LinkedIn: https://www.linkedin.com/in/tejas-bhakta/ Website: https://tejasbhakta.com Related links: Morph: https://morphllm.com Timestamps: 0:00 Intro 0:32 What is autoresearch? 1:02 Why GPU kernels fit autoresearch 1:32 Humans bring the ideas 2:17 Compute, memory and overhead bottlenecks 3:02 Cheap GPUs need custom kernels 3:17 Give the agent hardware context 3:52 Give the agent model context 4:12 Reward hacking 5:16 Custom kernels aren't faster everywhere 5:36 Kernel gains compound 6:01 Bare-metal hacks 6:36 3x faster, but 80% of attempts are bad 6:57 TL;DR