Tell the Robot What You Want — Sandhya Subramani, AWS

Sandhya Subramani asks her rover how many people it can see, a question nobody ever trained it to answer. It thinks for a second, checks its front camera, and reports two, one near the speaker monitor and one further right. Scout is a small four legged robot running a Raspberry Pi, reaching the internet over a SIM card and a 4G connection, and its whole personality comes from an agent layer sitting above the movement policies it already had. The demo goes exactly as live robot demos go. It falls over, gets coaxed upright, announces a signature performance and then just turns its headlights off. The idea underneath is neat and portable. If you give an agent software tools, you can also hand it a hardware tool and let it choose which preset policy to fire, which turns a robot trained for a fixed task list into something you can address in plain language. She wires it up in about five lines using an open source AWS framework that already covers 40 or so robots. Scout actually runs three agents at once, one thinking about the environment, one talking to her over Telegram, and a voice agent she disabled so it would stop interrupting her on stage. Her framing is that the agent decides what to do while the trained policy decides how, and the robot doubles as a rig for collecting the next round of training data. Speaker info: - https://www.linkedin.com/in/sandhyasubramani/ Timestamps: 0:00 - Meet Scout, running on a Raspberry Pi over 4G 1:27 - Answering a question nobody trained it for 3:46 - Giving an agent a hardware tool 5:02 - Five lines to hand a robot to an agent 6:09 - Three agents running at once 7:17 - The four layers, and how observations travel up 9:35 - Splitting work between cloud and edge 10:45 - Where policies go if they scale like language models 11:53 - Asking it to do something complex 14:12 - What is actually under the hood 15:21 - Mirror mirror, over Telegram