What Happened
A wave of studies is painting a more complicated picture of AI coding tools than the productivity narrative vendors prefer.
The Google DORA Report found that 90% of roughly 5,000 tech professionals now use AI at work, with over 80% reporting productivity boosts. But a study from Multitudes tracking 500+ developers tells the other side: engineers merged 27.2% more pull requests while logging a 19.6% rise in after-hours commits. More output, but at the cost of personal time.
The quality story is equally mixed. Google's own data shows AI increases "software delivery instability," meaning more code rollbacks and patches after release. As DORA's Nathen Harvey put it: "As you use more AI, you're more likely to roll back changes that you've pushed into production."
Then there's the skills problem. Anthropic published research in January 2026 showing that AI-assisted developers scored 17% lower on library comprehension quizzes, with the largest knowledge gaps appearing in debugging. Researchers at UC Berkeley's Haas School found employees worked more tasks, at a faster pace, for longer hours after AI adoption, with AI use during breaks reducing the mental recovery workers need.
Why It Matters
If you use Cursor, Claude Code, Copilot, or any AI coding assistant daily, this data should make you pause. The productivity gains are real - 27% more merged PRs is significant. But the studies suggest those gains are being captured by organizations, not workers. You ship more, but you also work more evenings. The code gets out faster, but it rolls back more often.
The 17% comprehension drop is particularly concerning for anyone early in their career. If AI handles your debugging, you never build the mental models that make you a strong engineer. Lauren Peate, CEO of Multitudes, was direct about it: "If that out-of-hours work is going up, it's not good for the person. It can lead to burnout."
Open-source maintainers are already reporting increased volumes of low-quality, AI-generated submissions. The flood of "more code" doesn't automatically mean "better code."
Our Take
These findings match what we see when testing AI coding tools. The best ones (Claude Code, Cursor) genuinely make you faster at tasks you already understand. The problem starts when you use them as a crutch for tasks you don't understand - you get output without learning.
The smart approach: use AI coding tools to accelerate work you could do yourself, not to replace skills you haven't built. Set boundaries on after-hours work even when the tools make it tempting to "just finish one more thing." And if your rollback rate is climbing, slow down. Shipping 27% more code that breaks in production isn't a productivity gain - it's technical debt with extra steps.
Anthropic researcher Judy Hanwen Shen framed it well: the goal "shouldn't be to use AI to avoid cognitive effort - it should be to use AI to deepen it." That's the difference between AI tools making you better and AI tools making you replaceable.