Teaching Assembly Robots by Demonstration

GIT Researchers to develop robots that use a Kinect sensor to learn by example

Do as I do, not as I say. That is the idea behind a new way to give instructions to robots. Neil Dantam at the Georgia Institute of Technology in Atlanta and colleagues are developing a system in which robots learn by example. Instead of programming a robot to carry out a task, a demonstration could suffice.

However, the robot does not simply mimic the human's actions. The prototype system uses an Xbox Kinect camera to observe the human performing an activity, then breaks that activity into a sequence of key actions necessary to carry out the task. These actions are converted into a general set of instructions – much like those found in an IKEA furniture instruction manual – that can be interpreted even by non-humanoid robots.

"You want to somehow capture the important aspects of the human's motion and transfer that to the robot," says Dantam. "Think about how you'd tell someone how to make a cake," he says. The way people follow those steps may vary. "An adult might bend down over the counter to work while a child may stand on tiptoes."

Dantam's system has been tested by teaching a robot arm to assemble a Lego-like structure consisting of wooden bars and pegs. A robot can also be taught alternative ways to perform a task by demonstrating it many times – if it gets stuck doing it one way, it can try another.

"As robots are already widely used within the manufacturing domain, focusing on assembly tasks makes a lot of sense," says Brenna Argall at Northwestern University in Evanston, Illinois.

"One of the big wins in teaching robots via demonstration is that the act of developing control paradigms or task behaviours is no longer restricted to robotics experts," she says. "Importantly, it now becomes accessible to a broader population of people, which has a huge potential for making robotics more accessible to the general public."

The team presented the work at the International Conference on Intelligent Robots and Systems in Vilamoura, Portugal, this week.