Unlike existing robots on the market, such as Boston Dynamics’ Spot, which moves with internal maps, this robot uses cameras alone to guide its movements in the wild, says Ashish Kumar, a UC Berkeley graduate student who is one of the authors of an article describing the work; is due to be presented at the Conference on Robot Learning next month. Other attempts to use camera signals to guide robot movement have been limited to flat terrain, but they got their robot to climb stairs, climb rocks and jump over gaps.
The four-legged robot is first trained to move through different environments in a simulator, so it has a general idea of what it’s like to walk around a park or go up and down stairs. When deployed in the real world, images from a single camera on the front of the robot guide its movement. The robot learns to adjust its gait to navigate things like stairs and uneven terrain using reinforcement learning, an AI technique that allows systems to improve through trial and error.
Eliminating the need for an internal map makes the robot more robust, because it is no longer limited by possible errors in a map, says Deepak Pathak, an assistant professor at Carnegie Mellon who was part of the team.
It’s extremely difficult for a robot to translate raw pixels from a camera into the kind of precise, balanced movement needed to navigate its environment, says Jie Tan, a research scientist at Google who was not involved in the study. He says the work is the first time he’s seen a small, low-cost robot demonstrate such impressive mobility.
The team has made a “breakthrough in robot learning and autonomy,” says Guanya Shi, a University of Washington researcher who studies machine learning and robotic control, who was also not involved in the research.
Akshara Rai, a research scientist at Facebook AI Research who works on machine learning and robotics, and who was not involved in this work, agrees.
“This work is a promising step toward building robots with such insightful legs and deploying them in the wild,” says Rai.
However, while the team’s work is useful for improving the way the robot walks, it won’t help it know where to go ahead of time, Rai says. “Navigation is important for deploying robots in the real world,” he says.
More work is needed before the robot dog can walk the parks or fetch things at home. While the robot can understand depth through its front-facing camera, it can’t deal with situations like slippery terrain or tall grass, Tan says; could get into puddles or get stuck in the mud.