Distill the LLM, Don't Serve It: Search & Personalization at DoorDash — Raghav Saboo, DoorDash

Search for gluten-free pasta and a ranker trained on engagement may show you regular spaghetti, because it sells well. Raghav Saboo, Staff ML Engineer and Tech Lead for Search & Personalization at DoorDash, argues that the real bottleneck in marketplace discovery is semantic understanding, not engagement. As DoorDash grows into grocery, retail, pets and gifting across billions of store-level items, it uses LLMs to understand what items mean and what shoppers actually intend. His core pattern: do the expensive LLM reasoning once, offline, and then distill it into small, fast models for serving. He covers four building blocks. The first is LLM-generated relevance labels, which improved retrieval NDCG by 2.3%. The second is semantic IDs, a learned taxonomy of the catalog that raised ranking MRR by 4–5% and powers query reformulation. The third is consumer memory at three timescales, stored as text, vectors and a graph. The fourth is steerable, LLM-generated personalized collections, which lifted order rate by nearly 1% in the pets category. Speaker info: Substack: https://buildshipai.substack.com/ Timestamps: 0:00 Intro 0:47 Discovery's real bottleneck is semantic understanding 1:42 Beyond restaurants: broad shopping missions 2:52 The four primitives 3:17 Primitive 1: LLM supervision 3:27 Why engagement ranks the wrong pasta 4:11 Human labels vs. behavioral signals 4:46 Building a golden labeled dataset 5:46 A fine-tuned LLM labeler for the whole catalog 6:11 Reason offline, serve cheaply 6:21 Two-stage contrastive retrieval 8:00 Adding a relevance tower to rankers 9:00 Distill, don't replace 9:30 Primitive 2: catalog semantics with semantic IDs 11:00 A learned taxonomy 11:50 Cross-category, cold start and tail coverage 12:55 Results: 4–5% MRR gains in ranking 13:20 Query reformulation 14:05 Primitive 3: consumer memory 15:15 Three timescales of memory 15:55 Memory blocks: text, vectors and graphs 17:04 Context graphs 17:55 Where memory shows up 18:20 Primitive 4: steerable content generation 19:05 Personalized collections on store pages 20:33 Results in the pets category 20:48 Three takeaways