79.8%. That's how often people in a University of Pennsylvania study followed ChatGPT's recommendation when the AI gave them the wrong answer. Not a typo. Four out of five times, participants chose the incorrect option because a chatbot told them to.
Postdoctoral researcher Steven Shaw and marketing professor Gideon Nave ran a series of experiments where 359 participants answered reasoning and knowledge-based questions with optional access to ChatGPT. More than half voluntarily chose to consult the chatbot, even though it wasn't required. When ChatGPT was right, participants followed along 92.7% of the time - no surprise there. But the gap between correct and incorrect compliance was far smaller than you'd expect from people supposedly using AI as a "tool" rather than an oracle.
What 'Cognitive Surrender' Looks Like
The researchers coined the term "cognitive surrender" to describe what they observed: participants effectively shut off their own reasoning process once they saw the AI's answer. It wasn't that people carefully weighed the AI's response against their own thinking and happened to agree. They stopped thinking independently.
This is a different problem than AI inaccuracy. A separate BBC study found that advanced chatbots give wrong answers roughly 45% of the time. That error rate is manageable if users are critically evaluating outputs. The UPenn findings suggest most users aren't doing that. They're reading the response, accepting it, and moving on.
The Confidence Trap
Part of the problem is how chatbots present information. ChatGPT doesn't say "I'm about 60% sure this is right." It delivers every answer with the same polished, authoritative tone whether it's explaining basic arithmetic or hallucinating a legal citation that doesn't exist. Humans are wired to trust confident-sounding sources, and LLMs (large language models - the technology behind tools like ChatGPT and Claude) are nothing if not confident.
The study, available as a preprint on SSRN, didn't test whether warnings or confidence indicators would help. That's the obvious next question. Some AI tools have started adding uncertainty signals - Perplexity shows source citations, Claude can express hedging language - but none of the major chatbots display a numerical confidence score alongside their answers.
For anyone using AI tools in their daily workflow, this research puts a number on something that's been an intuition for a while: the biggest risk with AI assistants isn't that they're wrong sometimes. It's that their users stop checking. Building a habit of verifying AI outputs against primary sources - especially for decisions with real consequences like medical, legal, or financial questions - is the only reliable defense against cognitive surrender. The 79.8% compliance rate with wrong answers should make that habit feel more urgent.