Training robots from space

[European Space Agency]

Image:

This summer, a team of robots explored a simulated martian landscape in Germany, remotely guided by an astronaut aboard the International Space Station. This marked the fourth and final session of the Surface Avatar experiment, a collaboration between ESA and the German Aerospace Center (DLR) to develop how astronauts can control robotic teams to perform complex tasks on the Moon and Mars.

The session introduced new levels of autonomy and complexity. NASA astronaut Jonny Kim operated two robots – ESA’s four-legged Spot and DLR’s humanoid Rollin’ Justin – to retrieve scattered sample containers and deliver them to a lander. Spot navigated the terrain autonomously, while Justin was guided through a mix of direct control and pre-set commands. This setup allowed Jonny to delegate tasks and focus on higher-level decisions, building on other sessions where robots required full teleoperation.

In a second scenario, ESA’s Interact rover transported DLR’s robot dog Bert to a cave entrance. After removing a boulder, Jonny deployed Bert, which then simulated a malfunction in one of its legs. Jonny had to retrain Bert’s walking algorithm in real time before it continued into the cave and detected signs of martian ice. This tested how operators respond to unexpected challenges and adapt robotic systems on the fly.

The robots are controlled from the International Space Station using a custom-built interface developed by ESA and DLR, combining a joystick and a haptic-feedback device. The interface allows switching between first-person view for immersive teleoperation and a top-down map for broader mission oversight. This flexibility lets the astronaut manage multiple robots efficiently, balancing direct control with strategic delegation.

Over four sessions, the Surface Avatar team has refined its approach to human-robot interaction, improving both teleoperation and task delegation to autonomous systems. The experiment has also helped to identify which tasks astronauts prefer to control directly and which can be safely handed over to robotic systems, offering valuable insight for future mission planning.

Read our blog to find out more.

View the full article