Your Agent Hates Your CLI - Ayush Bhardwaj, Allos AI

An agent can hide a failed test behind a successful shell pipeline. It can also trim away the error message and confidently report that everything worked. Ayush Bhardwaj examined roughly 20,000 tool calls from his own coding sessions and found a product roadmap in the commands agents wrote immediately afterward. Repeated uses of head, tail, grep and Python showed where tools returned too much information, exposed the wrong fields, or made the agent reconstruct the same query again. The examples come from his own traces and his chat history CLI, which lets agents retrieve context from earlier coding sessions. The fixes are small and concrete: return the messages around a search match, offer compact search results, inspect several sessions in one call, and put a useful failure summary at the end of the output. But implementing a flag does little if the skill file still teaches the old interface. In Ayush's case study, agents kept passing a flag that did nothing until the instructions changed. He also explains how to turn recorded calls into replay tests, reserve examples for evaluation, and check whether better output actually improves answers and reduces turns. Smaller responses alone do not establish a faster or cheaper workflow. The useful feedback is already in the transcript: what the agent tried, what it discarded, and what it had to ask again. Speaker info: Ayush Bhardwaj, Allos AI - https://ayushb.me/ - https://github.com/ay-bh/chat-history Timestamps: 0:00 - The workarounds hiding in agent calls 1:37 - Auditing 20,000 calls and misleading exit codes 2:59 - The chat-history CLI case study 4:32 - Smaller slices, compact results and batch requests 6:33 - Skills and help text are the interface 7:36 - Turn transcripts into replay tests 9:19 - Agent mode and measuring useful outcomes 10:14 - Design the interface, then test against real traces