When Running Your Own AI Models Beats Paying for APIs - Steven Latré, imec

Steven Latré leads AI at imec, the Belgian semiconductor R&D center, and his team ran thousands of benchmark runs to find out what moving from APIs to your own GPUs really costs for long-running coding agents. Testing open-weight models and three coding harnesses on SWE-Bench Pro, they found that hardware costs jump in steps rather than a smooth curve, that concurrency matters even more than accuracy, that break-even GPU utilization ranges from 89% on small systems to about 15% on 8x B200 machines, and that the right harness can cut GPU time in half for the same result. Chapters: 0:00 Intro 0:59 Why a chip lab studies AI workloads 1:32 The stages of AI maturity 3:28 Owning your stack blows up the design space 4:38 The benchmarking study 5:28 How the tests were set up 7:28 Accuracy by model 8:40 Adding the hardware cost 10:08 Concurrency: how many users fit 12:37 API vs. renting vs. buying GPUs 14:35 Harnesses and token use 16:20 What's next Steven Latré https://www.imec-int.com Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!