Your Self-Hosted LLM Needs to Say No - Harshul Jain & Parv Khatri, Audible & SAP

Coding agents fan out, retry and burst, and if they hit an LLM you host yourself, your inference layer will happily accept every request and bury the ones you needed first. Harshul Jain and Parv Khatri make the case for admission control: deciding what to refuse before it reaches the GPU. Drawing on a review of 55 systems papers and eight real production incidents, they share five questions to ask any admission policy before you deploy it, and a checklist you can use right away. Harshul Jain, Audible: https://x.com/hj1393 | https://www.audible.com Parv Khatri, SAP: https://www.sap.com