Nasa ‘holoported’ doctors to space station to visit astronauts

Combined with augmented reality, Nasa says technology can enable ‘tele-mentoring’

Holoportation team members are projected virtually on to the International Space Station on 8 October 2021 (ESA (European Space Agency) astronaut Thomas Pesquet)

In a first, Nasa beamed holograms of doctors to the International Space Station to visit astronauts in 2021, the American space agency has revealed.

Nasa flight surgeon Josef Schmid, industry partner AEXA Aerospace chief Fernando De La Pena Llaca, and their teams were the first humans “holoported” from Earth into space, the space agency noted in a post published earlier this month.

Holoportation is a type of capture technology that allows high-quality 3D models of people to be reconstructed, compressed, and transmitted live anywhere in real-time, Dr Schmid explained in a statement.

In this 2021 attempt, European Space Agency (ESA) astronaut Thomas Pesquet had a two-way conversation with live images of Dr Schmid and Mr De La Pena placed in the middle of the International Space Station, marking the first holoportation handshake from Earth in space.

Astronauts used the Microsoft Hololens Kinect camera and a personal computer with custom software from Aexa to achieve the feat.

Nasa noted that using mixed reality displays such as HoloLens allows users to see, hear, and interact with remote participants in 3D as if they are actually present in the same physical space.

While the technology has been in use since at least 2016 by Microsoft, experts say this is the first use in such an extreme and remote environment like space.

Nasa flight surgeon Dr Josef Schmid gives a space greeting on 8 October 2021 as he is holoported to the International Space Station (ESA (European Space Agency) astronaut Thomas Pesquet)

“This is completely new manner of human communication across vast distances. Furthermore, it is a brand-new way of human exploration, where our human entity is able to travel off the planet. Our physical body is not there, but our human entity absolutely is there,” Dr Schmid said.

“It doesn’t matter that the space station is traveling 17,500 mph and in constant motion in orbit 250 miles above Earth, the astronaut can come back three minutes or three weeks later and with the system running, we will be there in that spot, live on the space station,” he added.

Nasa views the recent demonstration as a precursor for more extensive use on future missions, such as for innovative 3D telemedicine applications to keep astronauts healthy.

Researchers also plan to use the technology for private medical conferences, private psychiatric conferences, private family conferences and to “bring VIPs onto the space station to visit with astronauts.”

The space agency plans to apply the technology next for more two-way communication, where people on Earth are holoported to space and astronauts are placed back on Earth.

In the future, Nasa says combining holoportation with augmented reality could “truly enable” tele-mentoring.

“Imagine you can bring the best instructor or the actual designer of a particularly complex technology right beside you wherever you might be working on it. Furthermore, we will combine augmented reality with haptics,” Dr Schmid said.

“You can work on the device together, much like two of the best surgeons working during an operation. This would put everyone at rest knowing the best team is working together on a critical piece of hardware,” he added.