Crowd Teaches Robot With Video Games
Cornell researchers teach a robot simple human commands by playing a video game.
While scientists have created robots that look like humans, teaching a robot to think and respond like a human is a more challenging task. Even the most basic spoken command, such as “Heat a kettle of water on the stove”, has a slew of unspoken assumptions that are understood before the action is taken: the location of the kettle, whether it already contains water, how to turn the faucet on, and so on. In robotics, these complexities are usually handled by writing actions directly into a robot’s code, but this leaves anyone without advanced programming knowledge unable to interact with the robot.
Tell Me Dave is a project by Cornell University Assistant Professor of Computer Science Ashutosh Saxena’s Robot Learning Lab. He and his team of graduate students aim to make robots more practical by teaching them to respond to natural speech commands, and they are relying on help from the public to do it.
The project’s robot is equipped with a 3D camera to scan for objects in its environment, as well as software that identifies these objects and associates them with their common usage. This allows it to locate a kettle in the kitchen, identify it as a kettle, and understand that a kettle is used for heating liquids.
What makes the Tell Me Dave robot unique is its ability to take a string of simple actions and associate them with a spoken command. This robot understands that the vague request “Make ramen noodles” entails heating a pot of water, adding the noodles along with spices, draining the noodles after they’ve cooked, and then serving them in a bowl. At the moment, the robot can make both ramen and an Italian desert called affogato.