Addressing Relation Hallucinations in Multimodal AI

Large language models (LLMs) have changed the way we interact with artificial intelligence. They can generate text, answer questions, and even understand images. However, they face problems known as "hallucinations," where they produce wrong or misleading information not supported by real knowledge.

These issues become even more complicated when we look at multimodal large language models (MLLMs) that combine text and images. Here, hallucinations can appear when the model misrepresents objects or relationships in an image. For example, if a model sees a boy next to a table but claims that the boy is on the table, that would be a hallucination. It’s essential to address these issues to ensure that MLLMs can be trusted in real-world scenarios.

#What Are Relation Hallucinations?

Hallucinations in these models can be broken down into three main types: object hallucinations, attribute hallucinations, and relation hallucinations.

• Object hallucinations focus on whether the model can correctly identify basic objects in an image.
• Attribute hallucinations look at whether the model can accurately describe properties like color or shape of those objects.
• Relation hallucinations are more complex. They revolve around how well the model understands the relationships between multiple objects in an image.

For instance, if a model sees a cat and a chair and claims that the cat is sitting on the chair when it is actually under it, that would be a relation hallucination.

#Challenges with Existing Research

Most research on hallucinations focuses on the first two types (object and attribute) and does not delve deeply into relation hallucinations. Current ways to evaluate these hallucinations often miss details. They may rely on simple methods that don’t give a full picture. This can lead to biases based on how the data is collected and labeled.

For example, existing datasets might not represent real-life situations well or might overemphasize certain relationships. Therefore, there’s a need to create a benchmark that better assesses relation hallucinations in MLLMs.

#Introducing Reefknot

To address these challenges, we created a new benchmark called Reefknot. This benchmark focuses on relation hallucinations in MLLMs, consisting of over 20,000 real-world examples.

First, we define relation hallucinations clearly, combining ideas from how we perceive things and how we think about them. We then build a dataset using a trusted source called Visual Genome, which helps us gather meaningful relationships between objects.

In our evaluation, we looked at current MLLMs and found they struggle significantly with relation hallucinations. To help with this problem, we propose a new strategy that involves measuring the model’s Confidence in its answers to reduce the occurrence of these hallucinations.

#Evaluating Relation Hallucinations

Our evaluation uses three tasks:

1. Yes/No Questions (Y/N): These questions ask the model if a certain relationship exists based on the image.
2. Multiple Choice Questions (MCQ): This task presents a correct answer and three incorrect options to test the model's understanding.
3. Visual Question Answering (VQA): In this task, the model answers open-ended questions about the image.

Across these tasks, we discovered that current models often fail to effectively manage relation hallucinations.

#The Importance of Confidence in Responses

One key finding is that many hallucinations arise when models lack confidence in their responses. When a model is unsure, its chance of generating a hallucination increases. To combat this, we developed a technique called "Detect-then-Calibrate."

The idea is simple: if a model’s confidence drops below a certain level, it suggests that the answer it has provided might be incorrect. In these cases, we adjust the model’s output using information from earlier processing layers to improve the final answer. This method has shown promising results, reducing hallucinations by nearly 10% across our tests.

#Building the Reefknot Dataset

Creating the Reefknot dataset was a careful process. We started by identifying relation triplets from the Visual Genome dataset. Each triplet consists of a subject, a relation, and an object. After filtering out less useful examples, we categorized the relationships into two types: perceptive and cognitive.

• Perceptive Relationships: These involve clear, locational terms like “on” or “behind.”
• Cognitive Relationships: These are more abstract and relate to actions like “watching” or “holding.”

Next, we constructed a series of questions based on these relationships, ensuring that each question was directly tied to the content of the image while avoiding ambiguity.

#Evaluating MLLMs with Reefknot

We tested several popular MLLMs using the Reefknot benchmark. Results showed significant differences in performance. Some models did better in specific tasks and struggled in others, revealing a need for tailored adjustments to improve their overall performance.

Interestingly, cognitive hallucinations appeared less frequently than perceptive ones. This might seem counterintuitive. The models are often trained on datasets rich in visual descriptions, giving them an edge in understanding cognitive relationships while missing perceptive ones.

#Analyzing Probability Distributions

Our study also looked at how confidence levels change when hallucinations occur. It seems that when models generate incorrect information, their confidence significantly drops. For accurate predictions, models usually exhibit high confidence, nearing 95%. However, when hallucinations arise, this confidence can plummet to around 70%.

By examining these probability patterns, we were able to identify instances of hallucination more effectively. This analysis helps us understand the deep layers in MLLMs where hallucinations are more likely to occur.

#Detect-Then-Calibrate Method

Our "Detect-then-Calibrate" method is key in tackling relation hallucinations. By monitoring when models lack confidence, we can better adjust their responses. If a model is found to be unsure, we utilize hidden states from earlier layers, which are generally more reliable, to enhance the final outputs.

Through rigorous testing, this method demonstrated improvements across multiple datasets, confirming its effectiveness.

#Conclusion and Future Directions

In closing, our work highlights the significant gaps in addressing relation hallucinations in MLLMs. The Reefknot benchmark serves as a valuable tool for evaluating these models and guiding future improvements.

While our current approach successfully mitigates basic hallucinations, further exploration is needed for understanding and addressing relation hallucinations in broader contexts. Moving forward, we aim to investigate the root causes of these issues and refine our techniques for better reliability.

By focusing on these areas, we hope to contribute to the advancement of trustworthy multimodal AI systems, ensuring they provide accurate and meaningful interactions in real-world applications.