Integrating Cognitive Maps with Active Inference

Living beings need to understand their surroundings and make plans to navigate them, especially in uncertain situations. This paper discusses a way to combine two approaches: Cognitive Maps, which help in learning about the environment, and Active Inference, which aids in planning actions when things are unclear. Despite both concepts being explored separately, how to effectively combine them remains an area of research. This study introduces the idea of integrating a cognitive map model into an active inference agent for better Decision-making in tricky situations.

#Cognitive Maps and Active Inference

Cognitive maps are mental tools that help individuals recognize spatial relationships in their environment. They play a vital role in reasoning and planning, particularly during navigation. Various studies in recent years have focused on developing computational models to mimic these cognitive maps, showing impressive capabilities in recognizing patterns similar to those found in biological systems.

However, many of these models lack complexity in planning tasks. They often rely on straightforward strategies, which may not be effective in more complicated environments. Therefore, this paper investigates how active inference can enhance planning when using cognitive maps. Active inference suggests that intelligent agents take actions that minimize expected surprises, balancing exploration of new Information with goal-oriented decisions.

#Clone-Structured Cognitive Graphs (CSCG)

The research develops a model called the clone-structured cognitive graph (CSCG), which captures essential features of cognitive maps. Firstly, the CSCG allows flexible planning. If the current observations do not match what was expected, the agent can adjust the plan accordingly. Secondly, it helps differentiate similar observations based on the context they are in. This is significant because recognizing the correct context can lead to better decision-making.

Given these features, it is hypothesized that connecting the CSCG with active inference will help agents better identify their location and make smarter decisions.

#Comparing Two Types of Agents

To evaluate the benefits of active inference, two types of agents are compared: one using a naive clone graph and another driven by active inference. The performance of these agents is analyzed in three different navigation scenarios. Each scenario varies in complexity.

1. Open Room Scenario: This simple environment evaluates the effectiveness of both models. Here, the agent has to navigate to a specific corner in a room.
2. Ambiguous Maze Scenario: In this more challenging environment, the agent must first locate itself before reaching a central goal. This scenario tests how well the agents can gather information.
3. T-Maze Scenario: This environment demands careful decision-making. Agents must infer the correct path based on a cue shown earlier. Making the wrong choice leads to a loss.

#Navigating in an Open Room

In the first experiment, both agents were tested in an uncomplicated open room where they had to reach a defined goal. The study anticipated that the active inference agent wouldn’t show marked advantages in this straightforward setting, as the clone graph was already capable of gathering enough information to reach the goal.

Using a grid layout for the room, each corner was associated with a unique visual clue. After training, the agents were tasked with navigating to one of these corners across numerous trials. Results showed both agents successfully reached their goals with similar performance in terms of time taken and success rates. This suggested that in unambiguous settings, both methods could perform equally well.

#Self-localization in an Ambiguous Maze

The second experiment introduced an ambiguous maze to examine how each agent handled a more complex environment. In this maze, the agents faced challenges due to the limited information available. They could only see the tile they were currently on, making localization difficult.

The goal was set at a specific tile representing the only clear observation in the maze. Both agents were tested to see how quickly they could navigate to this target after starting from a random position on a less informative tile. The results indicated that while both agents could achieve the goal, the active inference agent was notably faster than the clone graph agent. This finding underscored the effectiveness of using active inference in situations with high uncertainty.

#Decision-Making in the T-Maze

The final experiment focused on decision-making in the T-maze environment. Here, agents had to make a critical choice between two paths without direct knowledge of where the reward was hidden. They were influenced by colored cues located behind them, advising them of the right direction.

In this environment, the active inference agent demonstrated clear advantages. It consistently made informed decisions based on cues, leading to a perfect success rate. Conversely, the clone graph agent relied on chance, achieving a success rate only slightly above random guessing. The active inference agent’s careful approach ensured that it didn’t just act randomly but thought through its choices, leading to a higher success rate.

#Conclusion

In summary, this research highlighted the benefits of combining cognitive maps and active inference for planning and decision-making in uncertain environments. The active inference agent showed significant advantages in situations requiring informed choices, while performance in straightforward environments indicated that both approaches could perform similarly. Future work will extend this research to allow for continuous learning and improvement, which could further enhance training efficiency and performance in real-world applications.

This research emphasizes the importance of thoughtful decision-making in navigation and planning processes, and the potential of integrated models to achieve this goal effectively.