Violent Language Trends in Incel Forums

This study looks into the use of violent language in a specific online group called Incels. The term "Incels" stands for "involuntary celibates," referring mainly to men who struggle to form romantic relationships, often expressing frustration and anger online.

We gathered a large number of posts from an Incel forum called incels.is to study how violent speech appears in these messages. We sorted the posts into different categories: non-violent, explicitly violent, and implicitly violent. Two human reviewers carefully labeled a subset of these posts to help us understand how to identify violent language better. We then used advanced language models to evaluate the rest of the posts.

Our results show that, over time, there has been an increase in violent speech on incels.is. This increase is notably higher among users who are more involved in the community. While the amount of directly violent messages has decreased, the number of non-directed violent messages has risen, and discussions about Self-harm have become less common, particularly after users have been active for a significant time.

We found that both human reviewers agreed on their assessments of violent language, and the language models we tested also showed good alignment with their labels. Our best-performing model was able to analyze and categorize more posts for further review.

#Background on Incels

The Incel community has gained attention because of its links to acts of violence. Since 2014, more than 50 deaths have been connected to individuals associated with this group. The dangerous ideas shared within this online community, particularly towards women, have alarmed researchers and officials alike. Some authorities have classified Incels as a potential terrorist threat due to their ideologies.

The discussions in these forums often revolve around feelings of jealousy towards men who appear to attract women effortlessly, referred to as “Chads.” These dialogues can lead to harmful thoughts and expressions, reinforcing toxic beliefs about relationships and interactions with women.

Although many people view Incels primarily as violent, research has shown that they discuss a wide array of topics, including everyday life and hobbies. However, abusive and discriminatory language remains a significant issue, affecting both the community's dynamics and its members' mental health.

#Research Goals

This study aims to classify the types of violent language seen in incels.is and analyze how this language has evolved over time. We started by labeling some posts manually to set a standard for classification, then applied advanced language processing tools to analyze a larger set of data. We also examined how violent language changes over time among active users.

#Methodology

We collected posts from incels.is, obtaining over a million entries. From this pool, we categorized a random sample using a labeling system that distinguishes between non-violent, explicitly violent, and implicitly violent speech.

Two human coders reviewed a portion of these posts and provided insights into how best to identify various forms of violent language. Their assessments were used to fine-tune parsing methods for language models that would analyze the remaining posts.

The language models we used were tested with various prompts and in different batches to determine the most effective ways to classify the violent language. The best-performing model analyzed additional posts for our study.

#Findings

Our findings reveal that about 10% of the posts from our analysis contained some form of violent language. This rate showed a slight but significant upward trend over time. Interestingly, we noticed that after being inactive for a long period, users tended to employ less violent language than when they were actively engaged in the forum.

The experiments we conducted to enhance language labeling provided a practical and efficient way to manage data. We also observed that the size of input batches significantly impacted the model's ability to classify language.

#Comparative Work

In recent years, researchers have used various techniques to explore the language used in Incel forums. Natural language processing has been helpful in identifying patterns of aggression and negativity in comments. Earlier studies have focused on Hate Speech and toxic language, establishing a framework for understanding the ideology present within the Incel community.

Work continues to highlight how the community operates and how its language reflects broader patterns of discrimination and hostility. The research is crucial for developing strategies to address the harmful implications of these online discussions.

#Categories of Violent Language

When it comes to categorizing violent language, we recognized two primary types: explicit and implicit violence. Explicit violence is straightforward and easy to identify, while implicit violence tends to be more subtle, often lacking clear aggressive words.

In our findings, explicit acts of violence, such as threats or abusive statements, were more recognizable. Implicit language often involved sarcasm or coded terms that could imply aggressive sentiment without directly stating it.

We also classified language based on whether it was directed at specific individuals or groups. Directed violence targets someone specifically, while undirected violence addresses broader groups. In this context, the language used often reflects negative views towards women or particular groups in society.

#Classification Process

To classify the posts accurately, we employed a systematic approach that included labeling a sample of posts by human annotators. Their classifications helped establish a reliable standard against which we could measure the performance of language models in identifying violent content.

We used metrics like Cohen's Kappa to measure agreement between different annotators and language models. This was important for assessing the reliability of classifications across various posts.

Using feedback from human reviewers, we continuously adjusted our prompts and batch sizes for the language models to improve their categorization accuracy. The goal was to achieve consistency and reliability in identifying the nuances of violent language online.

#Discussion of Findings

Our analysis indicates that violence in posts has increased overall, particularly among users who have been active in the forum for extended periods. This suggests that individuals may become more entrenched in their views over time, leading to more extreme expressions of violence.

We also noted a shift in the types of violence expressed. Specifically, directed violence towards specific individuals has decreased, while non-directed violence has increased. This change suggests a potential shift in community norms over time, where users may feel more comfortable expressing general aggression rather than targeting individuals.

Interestingly, the amount of self-directed violence has dropped significantly, particularly after extended user engagement. This decline may reflect the influence of support or changes in community dynamics that encourage more positive expressions of sentiment.

#Implications for Moderation and Research

Understanding the prevalence and nature of violent language within the Incel community can aid content moderators and researchers alike in tackling the issues surrounding online hate speech. Identifying patterns in language usage can help develop targeted interventions, whether through moderation strategies or educational efforts aimed at mitigating harmful beliefs.

The utilization of language models provides a scalable approach to monitoring online forums, but it is essential to maintain ethical standards in data handling and user privacy. Additionally, addressing the emotional impact on human annotators should remain a priority during the review process.

#Conclusion and Future Directions

This study reveals important trends in the evolution of violent speech on incels.is, highlighting significant increases in violence across user levels and time frames. Understanding these patterns can inform future research and practical efforts to address the harmful language present in online communities.

Further investigations may delve into how specific subgroups within the Incel community express violence differently and the role that external events, such as societal changes or crises, may play in influencing online behavior.

Adapting our frameworks for classification to more accurately reflect the terminology and culture of the Incel community could lead to more effective strategies for language moderation. Ongoing research and collaboration between scholars and practitioners will be crucial in tackling the challenges posed by online hate speech and promoting healthier online environments.