Stop Guessing Your vLLM Config: Benchmarking LLMs on Runpod - Chris Vela, Runpod

Most teams spin up a new open model with the default vLLM recipe and hope for the best. Runpod forward-deployed engineer Chris Vela shows how to choose configs from data instead, starting with the use case and concurrency, and demos BenchPod, a benchmarking dashboard built on vLLM that tests configs live without restarts. He runs Kimi K3 on a B300, shows how tuning max batch tokens cuts inter-token latency, and asks whether you need a frontier model at all. Chapters: 0:00 Intro 0:29 What Runpod does 1:32 Everyone's guessing at vLLM configs 2:53 Version and CUDA gotchas 3:48 BenchPod: benchmarking on top of vLLM 6:07 Running Kimi K3 on a B300 7:40 Demo 10:01 Tuning max batch tokens 13:34 From R&D to production 14:21 Do you need the frontier model? Chris Vela https://www.runpod.io Join us in SF for the second annual AIE CODE summit: https://aie.codes our last flagship summit of 2026. Applications live now!