Introducing PLAYER*: A New AI Framework for Murder Mystery Games

Recent improvements in Large Language Models (LLMs) have made it easier for AI agents to communicate and interact in different settings. However, creating agents that can think and act well in dynamic human environments, especially in competitive and cooperative scenarios, is still difficult. This article presents PLAYER*, a new AI framework designed to improve the communication and problem-solving abilities of agents in murder mystery games.

#Challenges in Current Approaches

Despite advancements in LLMs, many agents struggle with understanding the complexities of human social interactions. Most existing agents rely on structured methods that may not work well in unpredictable environments. These methods often fall short when dealing with situations that require nuanced social reasoning, such as cooperative games where players interact through conversation.

#What is PLAYER*?

PLAYER* is a framework that uses a flexible method for planning and reasoning in complex games. It combines LLMs with an anytime sampling-based planner, which allows it to adapt to changing situations by asking relevant questions. This new approach offers a way to optimize path planning in environments that require intricate social interactions.

#Designing the WellPlay Dataset

To evaluate the effectiveness of PLAYER*, a dataset called WellPlay was created. This dataset includes 1,482 question and answer pairs based on the interactions in murder mystery games. The questions focus on character relationships, objectives, and the reasoning behind specific actions. By using WellPlay, researchers can measure the performance of PLAYER* and other agents in a structured way.

#Game Structure

In a typical murder mystery game, players take on different roles, including that of the murderer or a civilian. The game consists of several stages, starting with players getting to know their characters and their objectives. They then engage in discussions to gather information and deduce who among them is the murderer. The final stage involves voting to determine the suspect.

#How PLAYER* Works

#Components of PLAYER*

To make agents behave like human players, PLAYER* includes several key features:

1. Game Rules: Every player has essential information about the game.
2. Chain-of-Thought: This helps in reasoning step by step.
3. Sensors: These evaluate emotional and motivational factors during interactions.
4. Memory Retrieval: This keeps track of dialogues and information shared during gameplay.
5. Self-Reflection: This helps agents analyze their decisions based on past experiences.

#Questioning and Action Refinement

In PLAYER*, agents use a dynamic questioning strategy to interact with other players. They ask questions based on the values detected by their sensors, focusing on elements like emotion, motivation, and suspicion. This way, the agents can refine their targets and make more informed decisions about whom to question next.

#Experimental Setup and Methodology

The experiments conducted to evaluate PLAYER* involved comparing its performance against other multi-agent algorithms in similar scenarios. Various models, including GPT-3.5, were utilized for conversation and memory retrieval. Results indicated that PLAYER* performed better than other models in achieving game objectives through effective questioning.

#Performance Evaluation

PLAYER* was tested using the WellPlay dataset to assess its understanding of character objectives and relationships. The scoring system assigned points based on the agents' ability to achieve game objectives, reason correctly, and provide additional relevant information. PLAYER* outperformed other methods, especially in objective-based questions, demonstrating its strength in handling complex situations.

#Addressing Subjectivity in Evaluation

Current evaluation methods for assessing agent performance often rely on subjective interpretations, which can lead to biased results. PLAYER* addresses this by using a quantifiable evaluation method with multiple-choice questions. This approach provides clearer metrics for comparing different agents' performances.

#Conclusion

PLAYER* represents a significant advancement in the development of AI agents for complex, interactive environments like murder mystery games. By optimizing communication and reasoning through a unique planning strategy, it opens new avenues for research and application in multi-agent systems. The creation of the WellPlay dataset further enhances the ability to evaluate and improve the performance of AI agents in these dynamic settings.

#Future Directions

As PLAYER* continues to evolve, future research may explore its application in various other gaming scenarios beyond murder mysteries. By adjusting rules and objectives, the framework can be tailored to fit different types of games. The goal is to create AI that can seamlessly engage with human players, providing a richer gaming experience while navigating complex social dynamics.

#Final Thoughts

The development of PLAYER* is a step towards creating more intelligent and capable AI agents that can engage in meaningful interactions. As technology advances, the possibilities for AI in gaming and other interactive environments will continue to expand, paving the way for more immersive and enjoyable experiences for players.

This was an overview of PLAYER*, a new AI framework designed to improve communication and problem-solving in murder mystery games. By focusing on effective questioning strategies and creating a quantifiable evaluation method, PLAYER* enhances the way AI agents interact in dynamic environments. The future looks promising for intelligent agents as they continue to develop and learn from their experiences in games.