Improving Decision-Making Models with Neural Networks

In recent years, there has been a growing interest in how humans make decisions in changing environments, especially in areas like Cybersecurity and email phishing. Understanding these decisions can help improve systems that aid human judgment and create better protection against online threats. Traditional methods often use broad models that treat all humans as if they make decisions in the same way. However, every individual's decision-making is unique, shaped by their past experiences. This article discusses new approaches that utilize neural networks to better capture the distinct ways different people make decisions in dynamic situations.

#Background

Decision-making is a complex process that varies from person to person. When faced with a new problem, individuals often rely on previous experiences. For example, when deciding whether an email is a phishing attempt or not, a person will consider similar emails they have encountered in the past.

A well-known model in psychology called Instance Based Learning (IBL) helps explain this process. IBL suggests that humans make decisions based on past instances and experiences that are similar to the current situation. Each instance consists of specific information about the situation, the decision made, and the outcome of that decision.

To improve decision-making models, we seek to combine ideas from IBL with advanced neural networks. This combination aims to create personalized models that reflect individual decision-making patterns better than traditional methods.

#New Neural Network Models

We introduce two attention-based neural network models to better understand human decision-making. These models are tested using two different datasets: one focuses on how people identify Phishing Emails, and the other examines decisions made in a cyber-attack scenario.

#Model Development

In our work, we create two distinct models. The first is called the Token-Level Personalized Memory Integrated Model (TL-PMIM). It uses a large language model (LLM) that has been trained on a broad dataset. This model can generate human-like responses but is less interpretable. The second model, the Instance-Level Personalized Memory Integrated Model (IL-PMIM), balances expressiveness and interpretability by using a simpler attention-based framework. In this model, the attention weights can be understood better, offering insight into how decisions are made.

The models are trained using data from human experiments, allowing us to simulate how individuals make decisions in real-world scenarios.

#Applications in Phishing Detection

Phishing emails are deceptive messages designed to trick people into revealing personal information. Understanding how people identify these emails is essential for developing effective training methods. In our experiments, participants were given emails to evaluate, and we collected data on their responses.

We explored how different models performed based on the quantity of training data provided to the participants. The results showed that the TL-PMIM model outperformed others in mimicking human behavior, especially when the participant was skilled at identifying phishing attempts.

In our analysis, we found that both the TL-PMIM and IL-PMIM models struggled more when individuals did not perform well on the tasks. This indicates that successful participants showed more consistent decision-making patterns, while those who made errors had less predictable behaviors.

#Insights from Cybersecurity Studies

Our second dataset involved a game where participants acted as attackers in a cybersecurity scenario. Here, participants had to decide whether to continue an attack on a target computer or withdraw. The models we developed were assessed based on how accurately they replicated human decisions in this context.

The results indicated a similar trend to the phishing study, where models performed better when aligned with participants who made more accurate choices. It was also noted that previous knowledge embedded in some models did not always improve performance when simulating individual behaviors.

#Fine-Grained Analysis of Decision-Making

To deepen our understanding, we also examined how well the models could interpret the reasons behind human decision-making. By extracting which past emails or actions were relied upon when making new decisions, we were able to shed light on the inner workings of the models. This can inform future improvements in how we model human behavior.

The findings suggested that models could identify relevant past instances that influenced current decisions. For instance, if a participant had previously classified similar phishing emails as threats, the model would learn to weigh those instances heavily when faced with a new email.

#Limitations and Future Directions

While our work shows promise, it is important to acknowledge certain limitations. The experiments were conducted in controlled environments, and real-world decision-making can often involve additional complexities, such as distractions or fluctuating environments. Future work should aim to include these contextual factors to improve the models further.

Additionally, while the IBL model offers strong interpretability, the TL-PMIM sacrificed some of this in favor of performance. Finding ways to explain the decisions of these more advanced models will be critical to ensuring they can be trusted in real-world applications.

#Conclusion

Our research highlights the potential of integrating neural networks with cognitive models to better understand and predict human decision-making in dynamic environments. As technology continues to advance, finding ways to enhance decision-making, particularly regarding security issues like phishing, will be crucial. The work presented here lays the groundwork for future efforts to create more personalized and interpretable models that truly reflect the intricacies of human cognition.