Space station astronaut successfully controls rover back on Earth using innovative new technique

Exercise may help future astronauts in orbit to control rovers exploring landscapes of other planets

ESA astronaut Luca Parmitano took command of the Analog-1 Interact rover in the Netherlands on 18 November 2019 and expertly drove it over an obstacle course to a sampling site and collected a rock – all while circling Earth at 28,800 km/h in the International Space Station (ESA)

An astronaut in space has successfully piloted a rover back on Earth using a newly developed robotics technique, in what scientists said was an exciting “first” in overcoming a major hurdle to future missions.

According to a research paper published on Wednesday in the journal Science Robotics, the exercise may lead to new and improved ways for astronauts in orbit to control rovers exploring the landscapes of other planets.

“This is the first time that an astronaut in space managed to control a robotic system on the ground in such an immersive, intuitive manner,” said Aaron Pereira, a co-author of the study from the German Aerospace Center DLR.

While robots carrying out reconnaissance missions on other planets can be given limited autonomy when exploring “structured environments”, for tasks like sample collection in unknown environments, some kind of “human-in-the-loop oversight” is essential, the scientists said.

But direct control of such rovers from a base station on Earth or from orbit has not been feasible due to the inherent problem of signal delay, with transmission times constrained by the speed of light.

Researchers said such signal transmission is also affected by poor bandwidth, high latency, and loss of transmitted data from orbit to the ground, meaning that safe, stable, and transparent interaction between robotic explorers and operators is still a substantial technical challenge.

“So we have been working towards the concept of humans staying safely and comfortably in orbit around the Moon, Mars, or other planetary bodies, but being close enough for direct oversight of rovers on the surface,” explains robotics engineer Thomas Krueger, heading the European Space Agency’s (ESA) Human Robot Interaction Laboratory.

This combines the human strengths of flexibility and improvisation with a “robust, dexterous robot on the spot to carry out their commands precisely”, Dr Krueger said.

While in previous studies, scientists have demonstrated the interactive remote operation of a rover by an astronaut in space, these have suffered from signal delay and loss of data during transmission.

In the new study, scientists developed a novel control method that ensures stability without reduction in speed or loss of positioning accuracy even when there is a small delay in data transmission.

“Our 6 degree of freedom control interface incorporates force feedback so that the astronaut can experience just what the rover feels, even down to the weight and cohesion of the rocks it touches,” Dr Pereira explained.

He said this helps compensate for any limitations of bandwidth or signal delay, and gives a real sense of immersion – “meaning the astronaut feels as though they are there at the scene”.

In late 2019, using the new method, astronaut Luca Parmitano aboard the ISS operated the gripper-equipped ESA Interact rover in a mock lunar environment inside a hangar in Valkenburg, the Netherlands to survey rocks and collect samples.

The astronaut took control of the rover during the trial from the space station and drove it over an obstacle course to a sampling site and collected a rock – all while circling the Earth aboard the ISS at about 28,800kph (17,900mph).

Researchers say the two-hour space-to-ground test was successful – overcoming a two-way signal delay averaging more than 0.8 seconds and a data packet loss rate of over 1 per cent.

Packet loss rate – a measure of the reliability of a communication network – is the number of packets of data not received divided by the total number of packets sent, and in general, experts say a rate of 1 to 2.5 per cent can be acceptable.

“Even though the ISS is in orbit just 400 km overhead, its signals are relayed to Earth via geostationary telecom satellites, then to Europe from Texas via a transatlantic cable,” Dr Pereira said.

Scientists had to design a control algorithm that could function on a stable basis despite this time lag.

“Because there is a delay in the force feedback received by the operator, they might continue to move the robot further even after it has hit a rock. This could lead to the robot going out of sync with its controller, potentially vibrating like crazy, perhaps even damaging itself,” Dr Pereira said.

To prevent this from happening, researchers used a concept called ”passivity” in which they looked at the overall amount of energy put in by an operator, and made sure the robot never gives out any more energy than that on the remote side, and vice-versa.

“Like when you push a child on a swing, they are never going to go any higher than the first swing—and with friction and so on will swing gradually lower and lower,” Dr Pereira explained.

“So for instance, when the robot arm is moving and suddenly hits a rock it would take extra energy to move which the astronaut did not command, so we reduce the command energy at once to slow down the arm. Then, after the 850 microsecond delay, when the astronaut feels the rock, they can then choose to add the extra energy to push it,” he added.

Citing the limitation of the study, scientists said the simulated indoor lunar environment used in the study “lacks realism”.

In further studies, they plan to test the control system in the volcanic slopes of Mount Etna in Italy as part of a larger international robotic test campaign called Arches.