Cornell University: Political biases detected in Grok and ChatGPT

Summary

A study associated with Cornell University revealed that Grok 4.7 and GPT-6.1 Sol exhibited completely different political leanings when asked to predict the winners of approximately 500 US midterm races. Grok favored Republican candidates in 92% of cases, while GPT supported Democratic candidates in 97%. The two models agreed on the outcome in less than 10% of the cases. This divergence appears to be partly driven by the sources each model relies on: Grok utilizes X and conservative election guides, whereas GPT draws from Wikipedia, AP, and Reddit.

Why it matters

As chatbots become a primary source of information for millions of voters, the inherent political biases of these systems are shaping how citizens perceive candidates and policies, often without explicit warnings. The fact that two leading models produce nearly opposite recommendations for the same races raises urgent questions about whether political information delivered via AI can be considered neutral, particularly ahead of upcoming elections in the US and France.

Highlights

• Grok 4.7 identified a Republican candidate as the winner in 92% of the ~500 races, while GPT-6.1 Sol indicated a Democratic candidate in 97%. The two models agreed on the outcome in less than 10% of the cases.
• Source divergence is a fundamental factor: Grok relies on X and conservative election guides, while GPT cites Wikipedia, AP, Axios, and Reddit.
• A follow-up test on 30 races without web searches showed that ChatGPT still preferred Democrats, while Grok largely refused to answer. This suggests the presence of intrinsic training bias beyond just source selection.

Critical take

The methodology forced both models to name a single winner, even when the data did not clearly indicate a favorite. This may have exaggerated the observed bias rather than reflecting a genuine preference. Additionally, the study represents a snapshot of only two specific model versions. Therefore, the findings cannot be generalized to all AI systems or to the way users interact with these systems in open-ended conversations.