Addressing Bias in AI: A New Zealand Perspective

Recent advancements in artificial intelligence (AI), especially with large language models (LLMs), have shown benefits in various areas. However, these models can carry biases that might affect fairness and equity, particularly for groups that are not well represented. This raises issues about how AI interacts with different societal groups. There's growing interest in researching these biases, but most of the studies focus on well-known demographics like white and black populations in the United States, as well as on male and female classifications.

There’s a clear need for more Research on Bias in less represented communities. Such communities may not have enough data or technological resources to be included in many studies. To address this gap, we aimed to create Datasets specifically for New Zealanders, a country with a small but diverse population. This article describes our experiences and challenges in gathering and analyzing data to measure bias in AI regarding underrepresented groups.

#Understanding Bias in AI

Bias occurs when a model's performance is inconsistent across different groups of people identified by traits like gender, income, or ethnicity. In this article, we focus primarily on Ethnic differences. AI models, including LLMs, often reflect historical data, which may include social inequalities. These models can perpetuate unfair treatment in their predictions, potentially leading to negative impacts on certain groups.

#The Need for Datasets

To study and measure bias effectively, we need annotated datasets-essentially labeled information that helps us understand how biased AI might be. However, the existing datasets primarily represent American demographics. Therefore, creating annotated datasets for New Zealand's population becomes essential. New Zealand has around 5 million residents, including about 17% indigenous Māori and roughly 70% New Zealand Europeans. Both groups speak English, and Māori is also spoken in various contexts. There are significant social disparities between these groups.

#Challenges in Data Annotation

Creating datasets involves multiple steps: generating text, defining ways to measure bias, and labeling this text through independent reviewers. Our project used specific templates to generate text involving various New Zealand demographics. We then used these templates to prompt a language model to create relevant text.

After generating the text, we manually labeled it with the help of three annotators. However, we encountered several challenges in this process. Only 35% of the generated texts received agreement from all annotators regarding their labels, indicating that even individuals with similar backgrounds have differing opinions on bias. Finding qualified annotators in a smaller country like New Zealand is difficult, as is ensuring they interpret bias consistently.

#Evaluating Bias

To evaluate bias, we need techniques to measure differences in how the model performs with various demographic groups. We looked at sentiment scores, language inference measures, and toxicity detection as ways to evaluate bias in the text generated.

For our analysis, we labeled text based on its regard towards different ethnic groups. This shows not just whether the language is positive or negative but also how societal perceptions are reflected in the generated text.

#Results of the Annotation Process

From the generated texts, we found significant variation among annotators. This inconsistency demonstrates that bias labels can differ greatly, even among a similar group of annotators. In smaller communities, the influences of individual annotators can amplify bias in the analysis.

Specifically, we looked at the jobs associated with various demographics in our generated texts. While there was some overlap, we noted a lack of professions like law enforcement in the Māori or Pacific community depictions.

#Ethical Considerations

Handling sensitive data requires careful ethical considerations. The potential for misuse of findings and the inherent risks in discussing certain topics must be acknowledged. Despite our efforts to ensure that our research is responsible, there are always chances for misinterpretations. Future researchers should be mindful of these challenges and the potential for harm in their work.

#Recommendations for Future Research

Based on our experiences, we recommend several approaches to improve the process of annotating datasets for bias. Firstly, we suggest a thorough review of how annotators are chosen and trained. Conducting an initial evaluation of the annotators’ perspectives might help in understanding variations in their judgments.

Encouraging diversity within annotator teams is also essential. If possible, having members from minority groups can provide valuable insights and lead to better understanding of the biases in the texts. Additionally, clear guidelines and discussions among annotators are crucial to ensure they are aligned in their understanding of bias.

Moreover, establishing robust, standardized metrics for measuring bias could help the research community. Currently, many tools used to define and measure bias can be subjective. The pursuit of consistent definitions and evaluations should remain a priority.

#Conclusion

Our research on creating annotated bias datasets for New Zealand’s demographics highlights the need for a deeper understanding of bias in AI, particularly for underrepresented societies. While there is a growing body of work focusing on bias in AI, little attention is given to communities like ours. The lessons learned from our challenges can aid future efforts to tackle bias through more comprehensive datasets reflecting the diversity of different populations.

The insights gained from this process will benefit researchers working on similar issues in underrepresented communities, and raise awareness of the importance of considering the broader implications of AI technology. Addressing biases in AI is essential for creating a fairer and more equitable future.