Advancements in Human Movement Prediction for Robots

In recent years, the use of autonomous robots has grown quickly, especially in fields like logistics, transportation, and healthcare. One major requirement for these robots is the ability to move safely around humans. This means they need to detect and predict human movement accurately. This capability is vital not only for avoiding collisions but also for understanding what people are likely to do next.

Different systems exist for predicting human movement. Some approaches ignore the surrounding environment, while others consider the context in which the robot operates. For example, knowing if a robot is in a busy store or a quiet warehouse can change how it predicts human actions. Context-aware systems tend to perform better because they can factor in the many ways people interact with each other and with objects around them.

To improve how robots understand human movement, we have developed a new system called NeuROSym. This system uses advanced technology that combines traditional rules with modern neural networks. The goal of neuROSym is to help robots predict human movements more accurately and in real-time when they are deployed in real-world settings.

#Background

Current methods for predicting human motion can be divided into two main categories: those that do not consider context and those that do. There are various approaches within these categories. Some systems rely on numerical data and physical models, while others use simpler, more understandable representations of movement.

One effective method is the Qualitative Trajectory Calculus (QTC), which represents how people or objects move in relation to each other using easy-to-understand symbols. This approach is beneficial because it simplifies the complexities of motion while retaining vital information about how agents interact.

In our previous work, we developed the NeuroSyM model that combines both human movement data from neural networks and symbolic knowledge to enhance performance. We found that this combination outperformed systems that relied solely on neural networks in experimental setups.

However, much of the existing research focuses on testing models using pre-recorded data rather than in real-time situations. To fill this gap, we created the neuROSym package, which allows for more practical applications of these theories.

#neuROSym Package

The neuROSym package is designed for robots to use in real-time scenarios. It consists of three main components:

1. Inference Model Node: This part is responsible for predicting human movement based on input data. It analyzes the movements of people around the robot and uses these insights to forecast what they will do next. It also provides data for performance evaluation.

2. Data Visualization and Analytics Node: This node runs simultaneously with the inference model to visualize movements and calculate performance metrics. It helps users see how well the robot is predicting movement in real-time.

3. Data Post-Processing Node: If the tracking system misses any detections, this node corrects those errors. This ensures that the robot can track individuals consistently, even if some movements are lost due to occlusions or other tracking challenges.

Each of these components works together to allow the robot to not only predict where a person will go but also adjust its predictions based on ongoing movements in its environment.

#Application and Testing

To test the functionality of neuROSym, we used a TIAGo robot equipped with advanced sensors. This robot was placed in a controlled environment where it could monitor the movements of two people for a duration of two minutes. The aim was to evaluate the performance of neuROSym under different human motion patterns.

We conducted two main types of experiments:

1. Scenario A: Parallel Movement: In this scenario, both individuals walked parallel to each other. The robot’s prediction system had to recognize and adapt to this simple moving pattern, which is often seen in public spaces.

2. Scenario B: Crossing Paths: In this more complex scenario, the two individuals crossed each other's paths. Here, the robot faced a greater challenge as it had to predict more unpredictable movements and avoid potential collisions.

During both experiments, we collected data on how accurately the robot predicted the paths of the individuals and how quickly it could process this information. The results allowed us to compare the neuROSym model's performance against a baseline model, SGAN, which does not use the same symbolic approach.

#Results

The results from both experiments showed that the neuROSym model was more accurate in predicting human movement compared to the SGAN baseline. Specifically, the average displacement errors (the distance off from the actual paths) were lower when using the neuROSym system. This indicates that the robot could foresee human actions more effectively in both testing scenarios.

In terms of speed, the neuROSym model took a bit longer to make predictions than the SGAN baseline. However, the trade-off was worth it since neuROSym produced significantly more accurate results. With some improvements to the code and execution speed, it can become even more viable for real-time use within robot systems.

#Discussion

These findings highlight the importance of context in predicting human motion. By incorporating qualitative knowledge about movements, neuROSym enhances the robot's ability to navigate and interact safely with people. Even though it may take slightly longer to process predictions, the improvements in accuracy justify this delay, especially in complex environments.

As robots become more integrated into daily life, the need for reliable and effective motion prediction systems will only grow. The ability to understand and anticipate human actions will increase user trust and safety, making such technologies more appealing in various industries.

#Future Work

There is still much work to be done to improve the neuROSym model. Future research will focus on testing it in a wider range of scenarios. This includes more complicated movement patterns, larger groups of people, and different environmental conditions. We also plan to explore different types of tracking systems to see how they can increase the reliability of the predictions.

By expanding the evaluation of neuROSym, we aim to ensure that it can handle the diverse situations that a robot may encounter in real-world applications. The ultimate goal is to enhance the technology further, making robots more capable of safely and efficiently operating alongside humans in everyday settings.