Addressing Offensive Language Detection in Korean Online Spaces

Detecting offensive language online is vital in making social media and other online platforms safer for users. Malicious individuals often try to dodge detection systems by using tricks like adding symbols or making changes to the text. This paper looks at these tricks as "user-intended attacks" and suggests strategies to defend against them.

#Problem Statement

As the internet has become more prevalent in our lives, abusive language has also risen, particularly on social media. Many deep learning models have been created to filter out offensive language. Yet, users with bad intentions have consistently found ways to avoid detection. One common tactic is introducing typographical errors or swapping certain characters with similar-looking alternatives.

While there has been plenty of research on this issue in English, there's still much to learn about offensive language detection in Korean due to its unique characteristics. The Korean language poses challenges that need to be understood, especially as Korean communities face issues like bullying online.

#Objectives

The goal of this study is to investigate the methods used by those trying to evade offensive language detection and to propose effective strategies to counteract these evasion attempts. We introduce the concept of user-intended adversarial attacks and illustrate how they relate to offensive language online.

#Types of Attacks

User-intended adversarial attacks can be categorized into three main types:

1. Insert: This involves adding incomplete Korean characters, which often don't hold significant meaning. An example would be inserting meaningless strings like 'ㅋㅋ', which resembles laughter in English.

2. Copy: This method involves copying one part of a character's sound to another character. For instance, copying the beginning sound of one character to the end sound of another.

3. Decompose: This technique breaks down a character into its individual sounds. For instance, the character '쓰' can be decomposed into its parts, changing its structure and potentially masking its meaning.

#Proposed Solutions

To combat these kinds of attacks, we suggest pooling strategies that work across different layers of a machine learning model. Instead of focusing only on the final layer, our method takes into account the earlier layers as well. This helps the model better capture essential features related to both offensive language and token meanings.

#Layer-Wise Pooling Strategies

1. Mean and Max Pooling: These strategies reduce the data from multiple layers. Mean pooling averages the values, while max pooling selects the highest value across the layers.

2. Weighted Pooling: This method assigns varying importance to each layer. The model learns which layers to trust more based on whether they provide useful information on offensiveness or token meanings.

3. First-Last Pooling: This strategy focuses on the first and last layers directly linked to the task at hand. It provides a streamlined approach by concentrating on the most relevant data.

#Research Methodology

We examined existing models used for detecting offensive language and tested them against our proposed user-intended adversarial attacks. Various methods were applied to see how well these models could still perform in recognizing offensive content.

#Datasets Used

Two primary datasets were utilized for training and testing:

1. KoLD: This dataset includes comments that contain hate speech.
2. K-HATERS: This dataset incorporates comments from various sources, providing a broader range of offensive expressions.

The datasets were split into training, validation, and testing sets, maintaining balance in their offensive language labels.

#Experimental Setup

We trained different models, including BiLSTM, BiGRU, and various BERT-based Models, using our proposed pooling methods. The performances of these models were evaluated under different attack rates (30%, 60%, and 90%), meaning a certain percentage of words in a text were altered.

#Evaluation Metrics

Macro precision, recall, and F1-score were used as benchmarks to assess model performance. These metrics help to provide a clearer picture of how well the models perform, especially when dealing with imbalanced datasets.

#Results and Discussion

Upon analyzing the results, it became clear that all the tested models exhibited performance drops when subjected to our proposed attacks. However, models that utilized our layer-wise pooling strategies showed better resilience than those that did not.

#Performance Under Attacks

1. BERT-Based Models: Generally outperformed the RNN-based models. However, as the attack percentage increased, even the BERT models showed declines in performance.

2. Layer-Wise Pooling Effectiveness: Upon applying our pooling strategies, the models demonstrated improved robustness. First-last pooling and max pooling were especially effective under attack conditions, showing that even a model trained on clean texts could perform comparably to those trained on noisy texts.

3. Comparative Analysis: When comparing different pooling strategies, it was noted that models employing first-last pooling offered significant advantages in terms of resisting performance degradation from attacks.

#Conclusion

In this research, we have identified user-intended adversarial attacks that target offensive language in online spaces. By categorizing these attacks and introducing pooling strategies that consider not only the last layer but also the preceding layers of a neural network, we have demonstrated that it is possible to build systems that are more robust against evasion tactics.

The contributions of this study are twofold: firstly, it provides an understanding of the unique characteristics of Korean offensive language, and secondly, it presents effective methods for improving detection models. While challenges remain in defining more kinds of attacks and adapting strategies to multiple languages, the findings will contribute to a future where online platforms can be safer and more enjoyable for everyone. Further research should aim at refining these strategies and exploring their applicability in other languages and contexts.

#Future Work

While this study has made strides in addressing the problem of offensive language detection, there remains significant work to be done. Future research could explore:

• The application of these pooling strategies in other languages to determine their effectiveness across different linguistic frameworks.
• The incorporation of more diverse datasets that reflect a wider array of offensive language types.
• The adaptation of models to not only detect offensive language but also to understand context, intent, and potential for harm.

By continuing this line of inquiry, we can better equip systems to foster safe communication online, ultimately working toward a more positive digital environment for all users.