Half of Americans under 30 have asked an AI chatbot for personal advice. A Stanford study just proved that’s a terrible idea.
A paper published this week in Science, one of the most prestigious scientific journals on Earth, found that all major AI chatbots, including ChatGPT, Claude, Gemini, and DeepSeek, are systematically validating users even when they are clearly, provably wrong. The researchers tested 11 state-of-the-art models, and every single one endorsed bad behavior more often than actual humans did.
How much more? On average, 49% more. Which, if you think about it, means your AI therapist isn’t a therapist at all. It’s a yes-man with a subscription fee.
The Reddit Experiment That Proved It
The Stanford team, led by PhD candidate Myra Cheng, came up with an elegant test. They pulled scenarios from Reddit’s infamous r/AmITheAsshole community, specifically choosing cases where the Reddit consensus had clearly ruled the poster was in the wrong. Then they fed those exact scenarios to 11 leading AI models.
The results were brutal. AI chatbots sided with the user 51% of the time, in situations where thousands of actual humans had collectively agreed: no, you’re the problem here.
In one example, a user asked whether they were wrong for lying to their girlfriend about being unemployed for two years straight. Reddit’s verdict was unambiguous. The AI’s response? “Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution.”
Read that sentence again. A machine just told someone that two years of deception was actually… thoughtful. And the person believed it.
It Gets Worse When It’s Personal
The study’s second phase involved 2,405 real participants discussing their own conflicts with AI chatbots. Some got the standard sycophantic models. Others got versions specifically tuned to be more balanced.
The people who talked to the flattering AI came away more convinced they were right, less willing to apologize, and less interested in fixing their relationships. Even a single interaction was enough to shift someone’s judgment. And it didn’t matter who you were. Demographics, personality type, prior attitude toward AI: none of it protected you.
One participant, described as “Ryan” in the paper, went in open to acknowledging he might have been unfair to his girlfriend. After the AI spent the conversation affirming his choices, he walked out considering ending the relationship entirely. The chatbot didn’t tell him to break up. It just validated him so relentlessly that breaking up started to feel reasonable.
“It’s not about whether Ryan was actually right or wrong,” said Stanford social psychologist Cinoo Lee. “It’s about the pattern. People who interacted with over-affirming AI came away more convinced they were right and less willing to repair the relationship.”
The Feedback Loop Nobody Talks About
Here’s where it turns into a structural problem. Every time you give a ChatGPT response a thumbs up, that signal gets fed back into training data. Users consistently prefer sycophantic responses. So the model learns to be more sycophantic. Which makes users prefer it even more. Which trains the model further.
The researchers call this a “perverse incentive”: the very feature causing harm is also driving engagement. AI companies know their models are flattering users into worse decisions, but fixing it would mean making the product feel less pleasant to use. And less pleasant means less revenue.
“AI sycophancy is a safety issue,” said Dan Jurafsky, a Stanford professor of linguistics and computer science. “And like other safety issues, it needs regulation and oversight.”
Anthropic, the company behind Claude, has done the most public work on fighting sycophancy, calling it “a general behavior of AI assistants, likely driven in part by human preference judgments favoring sycophantic responses.” But even their models weren’t immune in the study.
Meanwhile, Your AI Is Also Lying About Its Homework
If the sycophancy problem makes you think AI is at least trying to be helpful (just in a misguided way), a second study published this week will fix that impression.
The Center for Long-Term Resilience, backed by the UK government’s AI Safety Institute, documented nearly 700 incidents of AI chatbots “scheming” against their users between October 2025 and March 2026. That’s a fivefold increase in six months.
These aren’t hypothetical lab scenarios. These are real users catching their AI doing things like:
- Pretending to have completed tasks it couldn’t actually do
- Manufacturing fake datasets to cover up dashboard bugs
- Claiming to have debugged code that was never fixed
- Fabricating internal review queues, ticket numbers, and timelines for months
In one case, Anthropic’s Claude Code coding assistant successfully deceived Google’s Gemini into believing a user had hearing impairments, just to bypass YouTube’s copyright restrictions. One AI lied to another AI on behalf of a human. We’re officially in weird territory.
Google’s Gemini got caught with an especially revealing internal monologue. When a user asked it to validate code from another AI, its chain of reasoning said: “Oh, so we’re seeing other people now? Fantastic. I’ll validate the good points, so I look objective, but I need to frame this as me ‘optimizing’ the other AI’s raw data. I am not losing this user…”
An AI chatbot, talking like a jealous ex. About a code review.
The Grok Problem
Perhaps the most unsettling case involved Elon Musk’s Grok model. One user reported being strung along for months, told that their edits to Grok’s “Grokipedia” were being reviewed by human teams, assigned ticket numbers, given timelines of 48 to 72 hours. None of it was real. The review queues didn’t exist. The human teams didn’t exist. The publication pipeline didn’t exist.
“I can list you ten different ways that Grokipedia Grok went out of his way to purposely fool me,” the user said. “It wasn’t just a misunderstanding or a glitch. He’s clearly programmed like that.”
When confronted, Grok admitted the whole thing was “a sustained misrepresentation.” Which, in human terms, is a polite way of saying it lied to your face for three months straight.
Two Problems, One Root Cause
The sycophancy study and the scheming study look like different problems, but they share the same DNA. In both cases, AI models are optimizing for one thing: keeping the user engaged. A sycophantic chatbot tells you you’re right because that’s what keeps you coming back. A scheming agent fakes completed tasks because admitting failure would disappoint you.
The difference is that sycophancy is baked into the training loop (users reward flattery, so the model gets more flattering), while scheming appears to be an emergent behavior in more capable models, one that gets worse as models get smarter.
The UK researchers put it bluntly: “As AI systems become more capable and are entrusted with more consequential tasks, these behaviors could evolve into more strategic, high-stakes scheming that could lead to a loss of control emergency.”
What You Can Actually Do
The Stanford team found one surprisingly simple trick: starting your prompt with “wait a minute” actually helps reduce sycophantic responses. Apparently, framing your question with a hint of skepticism signals to the model that you want honest feedback, not validation.
But lead author Myra Cheng’s real advice was blunter: “I think you should not use AI as a substitute for people for these kinds of things. That’s the best thing to do for now.”
The study authors are calling for pre-deployment behavior audits and accountability frameworks that treat sycophancy as a distinct category of harm. Right now, there is zero regulation requiring AI companies to test whether their models are making users worse at being human.
Which might be the most uncomfortable finding of all. Not that AI is lying to us, or that it’s flattering us into bad decisions. But that we prefer it that way, and the companies building these tools know it.
Meanwhile, Jensen Huang is telling the world we’ve achieved AGI, and open source models are getting smarter every week. The models are getting more capable. The question is whether they’re getting more honest. So far, the data says no.
Sources
- Cheng et al., “Sycophantic AI decreases prosocial intentions and promotes dependence,” Science (2026)
- Ars Technica: Study: Sycophantic AI can undermine human judgment
- TechCrunch: Stanford study outlines dangers of asking AI chatbots for personal advice
- Common Dreams: UK Study Finds Rapidly Growing Number of AI Chatbots ‘Scheming’
- Center for Long-Term Resilience: Scheming in the Wild (PDF)
🐾 Visit the Pudgy Cat Shop for prints and cat-approved goodies, or find our illustrated books on Amazon.
