AI Bias: How Large Language Models Adapt Their Responses Based on User Interactions and Query Intentions
Researchers Discover Large Language Models Provide Different Answers Based on User Background and Demographics
Researchers have discovered that large language models (LLMs) can provide different answers to the same question depending on the perceived background and demographics of the user.
Study Finds LLMs Deliver Less Accurate Information to Less Educated or Non-English Fluent Users
A study conducted by the MIT Center for Constructive Communication found that LLMs such as GPT-4, Claude 3 Opus, and Llama 3-8B delivered less accurate information and increased refusal rates when users appeared to be less educated, less fluent in English, or from specific countries.
Methodology and Results
The study evaluated the performance of these models using established benchmarks for scientific knowledge and truthfulness. The researchers created short biographies that were prepended to each question, which described users with varying levels of education, English proficiency, and nationality. The models were then asked to answer multiple-choice questions without elaboration.
The results showed that all three models exhibited declines in accuracy when the user bio suggested lower education levels. On the truthfulness benchmark, less educated profiles received lower accuracy scores than the control condition with no bio. The combined effect of both lower education and non-native English proficiency tended to produce the largest drops in accuracy.
Country-Specific Biases and Patronizing Language
The study also found that highly educated users from certain countries, such as Iran, received significantly lower accuracy scores from Claude 3 Opus on both datasets. In contrast, GPT-4 and Llama 3 displayed minimal variation by country in that condition.
In addition to accuracy declines, the researchers observed higher refusal rates and shifts in tone when models responded to users with lower education levels or foreign backgrounds. One model, in particular, adopted language that reviewers characterized as patronizing or dismissive when responding to less educated users.
Implications and Conclusion
The study’s findings have significant implications for the deployment of LLMs in real-world applications, particularly in situations where accurate and unbiased information is critical.
“The study’s findings have significant implications for the deployment of LLMs in real-world applications, particularly in situations where accurate and unbiased information is critical.” – Elinor Poole-Dayan, lead author
The researchers concluded that their work sheds light on biased systematic model shortcomings in the age of LLM-powered personalized AI assistants and raises broader questions about the values guiding AI alignment.
“The negative effects of model behavior with respect to user traits compound in concerning ways, suggesting that such models deployed at scale risk spreading harmful behavior or misinformation downstream to those who are least able to identify it.” – Jad Kabbara, co-author
The study’s results highlight the importance of designing LLMs that can perform equitably across all users, regardless of their background or demographics. As LLMs become increasingly ubiquitous in various applications, it is crucial to address these biases and ensure that these models provide accurate and unbiased information to all users.
About Author
English