Evaluating Language Models for Factuality

Large Language Models (LLMs) have changed how we use artificial intelligence. They are good at understanding and generating text. However, one big problem with these models is that they sometimes give wrong information, referred to as the Factuality issue.

In this article, we will discuss a new way to evaluate how well these models provide correct information. We will use a large test dataset collected from a knowledge graph, which is a structured database containing a vast amount of facts. This approach helps us assess LLMs without relying too much on human input.

#The Importance of Factual Accuracy

LLMs are powerful tools in AI, but their ability to produce accurate responses can be questionable. Sometimes, they create convincing sentences that are not true. This problem is known as hallucination. Hallucination can happen for various reasons, including using outdated data or the model making incorrect associations from the information it was trained on.

To address this, we need effective evaluation methods to test how accurately LLMs can generate factual content. Traditional methods often involve looking at the model's answers directly, which can be time-consuming and costly. Instead, we propose a more efficient way to evaluate these models using a Judge Model that can quickly determine if the LLM’s responses are right or wrong.

#Challenges in Evaluating LLMs

Many current methods for evaluating LLMs have limitations. Firstly, the data used for testing is often too narrow or incomplete. This means that they do not cover all the different topics these models may encounter, leading to inaccurate assessments of their overall capabilities.

Secondly, evaluating factuality can take a lot of time and resources. It usually requires generating large amounts of text and meticulously checking each response for accuracy. This method is not very practical for frequent evaluations.

Lastly, limited testing datasets can introduce biases, which might distort how we understand the models’ performances. To overcome these challenges, we need a scalable and efficient way to test LLMs using comprehensive datasets.

#Introducing a New Evaluation Framework

We propose a framework that utilizes Knowledge Graphs to evaluate LLMs’ factuality. Knowledge graphs are useful because they systematically represent facts about the world, allowing us to create rich tests without extensive manual work.

#Step-by-Step Framework

1. Data Collection: First, we collect statements from a knowledge graph that holds millions of facts. We can produce a large variety of questions based on these facts.

2. Judge Model Creation: Next, we create a judge model trained to classify the LLM's responses as True, False, or I don’t know. Instead of generating long pieces of text, the judge model only gives these three options, making the evaluation faster and cheaper.

3. Performance Evaluation: Finally, we use the judge model to evaluate the LLM based on all the statements from the knowledge graph. This process allows for a thorough assessment of the models’ factuality from different perspectives.

#Why Use Knowledge Graphs?

Knowledge graphs are a powerful resource because they encapsulate factual information about real-world entities. They can be drawn from sources like Wikipedia, making them reliable for AI tasks. Using these graphs allows us to form a broader and more complete understanding of an LLM’s factual abilities.

By relying on knowledge graphs, our evaluation framework can automatically generate millions of prompts, drastically cutting down the need for human involvement in labeling data. This leads to a more diverse and far-reaching evaluation process.

#Benefits of the Proposed Method

Our proposed method has several advantages over traditional evaluation strategies. It allows us to assess LLMs on the entire knowledge graph rather than just selected subsets. This ensures a more complete evaluation, capturing a wide range of topics and questions.

The use of a judge model also improves efficiency. Instead of generating detailed outputs for every question, this model simplifies the evaluation process, making it faster and requiring fewer resources.

#Analyzing LLM Performance

In our evaluation, we compare how different LLMs perform on various metrics, such as Correctness, truthfulness, and informativeness. These metrics help gauge how well the models answer questions accurately and provide substantial information.

We observe that the LLaMA-2 series of models generally performs well as their size increases. However, larger models can sometimes struggle with providing useful information. On the other hand, the Gemma models tend to produce detailed responses but may not always be truthful or accurate.

#The Role of Relation Types

Different types of relationships in knowledge graphs can affect how well an LLM performs. For example, certain relation types might be easier for LLMs to understand and generate responses for than others. An analysis of these relations shows varied performance, indicating that the type of information being modeled can change how accurately a model can respond.

#Understanding Factuality Issues

The factuality issue in LLMs stems from several factors. It can arise when models lack expertise in specific areas or are unaware of the latest developments. Models can also forget information or fail to reason properly with the knowledge they have.

Various approaches have sought to improve LLMs by integrating additional sources of knowledge, fine-tuning the models, or using methods to enhance their understanding. However, our focus here is on evaluating LLMs instead of enhancing their knowledge base.

#Evaluating with Knowledge Graphs

Using knowledge graphs allows for a structured way to assess LLMs. Instead of randomly sampling smaller subsets, our method lets us evaluate performance comprehensively across different topics.

The knowledge graphs help us create wide-ranging questions based on factual statements, ensuring that the model’s evaluation reflects its true capabilities. This method stands in contrast to traditional evaluations that may focus on a narrow set of information.

#Collecting Statements and Labels

To assess LLMs, the first step is to collect statements related to facts from the knowledge graph. We can convert these factual triples into simple declarative sentences, which can be posed as questions to the LLMs.

In addition to generating true statements, we also need to create false statements. This can be done by swapping elements in the original triples to ensure that the LLM can accurately identify incorrect information.

#Judge Model Training

After collecting the statements, we train the judge model to evaluate them. The judge model learns to categorize responses into True, False, or I don’t know, based on the hidden states of the LLMs. This allows us to quickly assess a large number of responses without generating lengthy texts.

#Performance Metrics

When evaluating LLMs, we use specific metrics to analyze their performance. Correctness measures how often the model's answers match true facts, while truthfulness looks at how likely the model is to provide honest responses. Informativeness assesses whether the LLM offers substantial information beyond just admitting uncertainty.

By using these metrics, we can get a clearer picture of how well an LLM performs in providing accurate and informative responses.

#Experiment Setup and Results

To put our framework to the test, we used the DBpedia knowledge graph, which contains millions of facts drawn from Wikipedia. By sampling and generating a set of true and false statements, we evaluated how various LLMs responded to these queries.

We found that larger models, like LLaMA-2, showed improvements in overall performance as their size grew. However, some models, despite their size, did not perform well in providing accurate or reliable information.

#Conclusion

Our proposed framework provides a new angle on evaluating the factuality of LLMs. By leveraging large-scale knowledge graphs, we can thoroughly assess LLM performance without heavy human input. This method presents an opportunity for more efficient and wide-ranging evaluations of these powerful AI systems.

As we look to the future, this approach can help enhance the reliability of information generated by LLMs, ensuring that they not only produce text but do so with a focus on accuracy and factual integrity.