Space research conducted at Rényi Institute contributes to the development of numerous terrestrial robotic devices
The experiment carried out by Hungarian astronaut Tibor Kapu and designed at Rényi Institute has brought significant progress both in inertial navigation and in my own research and professional career,” summarized Miklós Kornyik, research fellow at Rényi Institute, following the series of experiments conducted on board the International Space Station (ISS). Dr. Kornyik, an applied mathematician who prepared the experiments, focuses on the use of inertial measurement units (IMU sensors) and GPS-free, so-called dead-reckoning navigation. Since the necessary measurement instruments are manufactured and tested under terrestrial conditions, and accelerometers used on Earth inevitably also measure gravitational acceleration, filtering out Earth’s gravitational acceleration and studying the data-providing sensors in microgravity space environments are of great importance. As the researcher put it, the space experiment and the HUNOR program catalyzed both his opportunities and his motivation.
The goal of the IMU-DRS project of the HUN-REN Alfréd Rényi Institute of Mathematics is to study navigation in microgravity based exclusively on data provided by accelerometer and gyroscope sensors. Tibor Kapu took a mobile device to the International Space Station, along with a special application created at Rényi Institute under the leadership of Miklós Kornyik. First, calibration measurements were performed with the phone at rest, consisting of 30 seconds of data with 100 measurement points per second. These measurements provided information on how measurement errors – so-called sensor noise – behave in a microgravity environment, as inadequate filtering and error handling can significantly degrade the quality of tracking and dead-reckoning navigation. In the second phase, Tibor Kapu performed sequences of movements that were recorded by the phone’s sensors, followed by the collection of an additional three hours of calibration data. Evaluating these data and drawing the most important conclusions remain an ongoing scientific tasks for Miklós Kornyik at Rényi Institute following Tibor Kapu’s return to Earth.
Dead reckoning is a navigation method that does not rely on external reference points such as GPS. Instead, starting from a known initial position, the current location of an object equipped with sensors can be calculated based on its direction, velocity, and the elapsed time.
An IMU (Inertial Measurement Unit) typically consists of an accelerometer and a gyroscope (measuring tilt and rotation). It measures acceleration, rotation, and changes in orientation, and from these data it is possible to calculate how an object moves through a particular space. The IMU provides the raw sensor data, while dead reckoning is the method that makes use of this data. In other words, the IMU can determine what the object is doing, while dead reckoning determines where the object is.
Following Tibor Kapu’s return, the reconstruction of the trajectories he traced with the device has now been completed. The current phase involves analyzing the video recordings made by the external camera, then comparing them witht the reconstructed trajectories,” explains Miklós Kornyik in response to an inquiry from renyi.hu. This means that the trajectories calculated from the data recorded by the phone are matched with those visible in the video footage captured by a wall-mounted camera. “So if Tibor Kapu traced a triangle with the phone, or any other simple geometric pattern, we can verify it in two independent ways. Preliminary analyses indicate that sensor measurement errors are smaller under microgravity conditions, although the difference is not significant,” he adds. (The external camera is owned by NASA, and the recordings were provided to Miklós Kornyik by NASA.) “This is how I validate,” he continues, “whether the (flight) path was reconstructed correctly, since I am familiar with the technical parameters of both the camera and the phone. Based on the initial evaluations, it can be stated that every part of the experiment was successful.”
“Dead reckoning has been a long-standing problem, and we have not solved it yet – we are only getting slightly better each time,” Kornyik Miklós continues. “There is gravity both on Earth and on the ISS; on the ISS astronauts simply do not feel it because it is in continuous free fall. On Earth, objects do not fall continuously for hours or even days. The key question was whether, in the absence of this constant, gravity-induced latent acceleration sensed by the instruments, the noise characteristics would remain the same or not.
In the case of properly functioning instruments, we expect nearly the same noise on the ISS as on Earth, but it is important to confirm this experimentally as well. This turned out to be the case, with only a slight difference. Understanding noise characteristics is crucial, because it allows measurements to be filtered more effectively, resulting in values that are closer to reality.”
There were also additional scientific observations, such as correlations between measurements along different axes; the investigation of these relationships is still ongoing. Based on the studies and results described above, a scientific publication is being prepared, which will represent another significant scientific milestone.
“My plan is to steer my research career toward the creation of miniature drones,” adds Rényi Institute’s researcher. “These would be small maintenance robots capable of assisting around satellites and space stations. Their development is a 5 –10-year project, involving the establishment of the necessary mathematical and technical foundations as well as the creation of a simple prototype, the latter to be carried out with the involvement of external experts.”
Miklós Kornyik’s ultimate goal is therefore the development of so-called maintenance drones. The next step along this path is the creation of a drone capable of operating inside the ISS. “Similar devices are already in use on the Station, but their application is quite limited and also depends on ownership – i. e. which country they belong to. For example, the Japanese space agency (JAXA) operates the so-called Int-Ball drone, while the European Space Agency (ESA) tested the CIMON drones in previous years. These devices can perform various tasks such as filming, reading out instructions, or even interacting verbally, and they are capable of autonomous movement. Based on detailed consultations with Tibor Kapu, it is clear that there is a demand for mini-drones on the ISS for additional tasks as well.”
“Among my plans is the prototyping of an active, workstation-like assistant that could support work inside the space station in several ways. It is no secret that tools are very easy to lose on the ISS: due to the air circulation system, there is constant airflow, and objects simply float away. Such an assistant could follow the astronaut and allow tools to be attached to it. In the more distant future, it could even be asked to bring, for example, a wrench. This part may still sound like science fiction, but what is not science fiction at all is a drone that, in addition to filming, can provide basic assistance.” “For many experiments conducted by Tibor and his colleagues external camera recordings were also requested. All this had to be set up so that the field of view was appropriate, often requiring another astronaut to help. The device I envision would be a multifunctional workstation that could greatly facilitate the mandatory auxiliary tasks associated with experiments,” says Miklós, who ultimately aims to develop a drone capable of operating outside the ISS as well. “According to current plans, the ISS will remain in service until around 2030–31, but this smart device could also assist around satellites or other automated platforms orbiting Earth,” he explains, outlining the long-term vision.
“We have stayed in touch with Tibor ever since. He liked the app and found it creative as well as intuitive, and we have also held brainstorming sessions about the space drone to better understand what might be needed and which development paths are realistic,” Miklós reports on recent developments.
The results may prove useful in space navigation, the coordinated operation of space assets, docking procedures, and maintenance tasks. Kornyik Miklós’s research – and the ISS experiment conducted with Tibor Kapu’s involvement – also has a number of terrestrial applications. “GPS-free tracking plays a particularly significant role in robotics. This is not about everyday navigation between two geographic locations, where a few meters of error may be acceptable. Rather, it concerns situations where a robot is operating in a cave or inside a building, where even a few meters of inaccuracy cannot be tolerated – even if GPS were technically available.”
Trajectory reconstruction is an essential element of maneuvering autonomous, self-driving, task-performing devices, planning their routes, or correcting these routes if unexpected events occur. In the case of the increasingly common ‘robot nurses,’ for example, it is crucial that movements are always well structured and precisely controlled – such as reaching for a patient’s leg accurately and lifting it at the correct angle without jerking. Trajectory reconstruction is an integral part of such execution.
“What the IMU-DRS experiment set out to investigate is tracking, which is closely linked to the analyses of data reflecting the forces acting on a robot – such as collisions with environmental objects or tipping over, all of which are visible in the data. To ensure that robots can perform tasks safely, trajectory analysis is indispensable. Long-term navigation based solely on IMU, that is, inertial data, is virtually impossible – and that is not even our goal. In practice, most robots are equipped with many additional sensors, including cameras and ultrasonic distance sensors, which together support navigation. One of these components is inertial navigation, which assists maneuvering. My research contributes to another step forward in technological development in an everyday world that increasingly relies on automated devices.”
