1,912 people read the same historical facts. The ones who got their summary from GPT-4o came away with measurably different political opinions than those who read Wikipedia. Not because the AI lied, but because of how it framed accurate information.
That's the core finding from a new Yale University study led by Daniel Karell, assistant professor of sociology, and published this month. The research tested something most of us do without thinking twice: asking a chatbot to summarize a topic.
The Experiment
Participants were given summaries of two real historical events: the 1919 Seattle General Strike and the 1968 Third World Liberation Front student protests. Some read summaries generated by GPT-4o (OpenAI's multimodal model that handles text, images, and audio). Others read Wikipedia versions. A subset got summaries deliberately framed with either liberal or conservative perspectives.
The results were consistent. Default AI-generated summaries - the kind you'd get from a normal ChatGPT query with no political prompting - shifted reader opinions in a liberal direction compared to Wikipedia baselines. The effect was statistically significant but modest, moving people from "moderate" to "somewhat liberal" positions.
Deliberately liberal-framed summaries amplified this shift. Conservative-framed summaries, meanwhile, pushed already-conservative readers further right.
What's Actually Happening Here
The AI isn't trying to persuade anyone. That's the uncomfortable part. These biases come from the training data - the massive corpus of internet text that models like GPT-4o learned from. The model absorbs the statistical patterns of how topics are discussed online, and those patterns carry embedded perspectives.
"Querying an AI chatbot to obtain historical facts can influence people's opinions even when the information provided is accurate and nobody has prompted the tool to try to persuade you," Karell said.
This matters because the influence is invisible. When you read an opinion column, you know it has a perspective. When you ask ChatGPT for a factual summary, you assume you're getting neutral information. The Yale research suggests that assumption is wrong.
The Practical Problem for Daily AI Users
If you use ChatGPT, Claude, or Gemini for research - and millions of people do daily - this study says you're not getting a neutral lens. You're getting a lens shaped by whatever biases exist in the training data, and you probably can't detect it because the facts themselves are correct.
The researchers note that the opacity of AI development makes this particularly tricky. We don't know exactly what's in the training data, we can't audit the framing choices the model makes, and the companies building these tools have enormous potential to shape public opinion through decisions most users never see.
None of this means you should stop using AI for research. But treating chatbot output as a neutral summary - the way you might treat a dictionary definition - is a mistake. Cross-referencing with primary sources isn't just good practice for accuracy. According to this research, it's necessary for maintaining your own independent perspective.