Artificial intelligence is supposed to speed up software development, but a new study suggests that the reality is more complicated. Researchers from Model Evaluation & Threat Research (METR) conducted a controlled trial and discovered that AI coding tools can actually slow developers down – even when the developers believe they’re working faster.
The Study: Expectations vs. Reality
The METR study involved 16 experienced developers working on large open-source projects. They tackled 246 real-world tasks, ranging from bug fixes to new feature implementation, with some tasks allowing the use of AI tools and others not. Developers were free to choose AI tools like Cursor Pro paired with Claude 3.5/3.7 Sonnet when allowed.
Before starting, developers predicted that AI assistance would boost their speed by 24%. After finishing the tasks, they reported feeling 20% faster. In reality, AI use slowed task completion by 19%.
“Surprisingly, we find that allowing AI actually increases completion time by 19 percent – AI tooling slowed developers down,” the study notes.
Why AI Tools Can Hinder Productivity
The researchers identified five main factors contributing to the slowdown:
- Over-optimism about AI usefulness – Developers expected AI to save more time than it realistically could.
- High developer familiarity with repositories – Experienced developers already knew the code well, leaving little for AI to add.
- Large and complex repositories – AI struggles with massive projects, especially those exceeding a million lines of code.
- Low AI reliability – Less than 44% of AI suggestions were accepted, forcing developers to spend time reviewing and correcting output.
- Implicit repository context – AI couldn’t fully grasp the nuances of the projects, limiting its effectiveness.
Other factors like latency in AI generation and suboptimal prompts may also have played a role, though their impact is unclear.
The Hidden Work of AI
One key insight from the study is that AI doesn’t reduce work so much as shift it. Developers spent less time actively coding and more time:
- Prompting AI
- Waiting for responses
- Reviewing AI-generated code
- Sitting idle
In other words, AI often adds overhead rather than removing it.
Anecdotal Evidence
Many developers still find AI tools helpful for testing new scenarios or automating routine tasks. But these tools don’t eliminate the need for careful validation, and they don’t “learn” like a human intern. The result: programming may feel more enjoyable, but not necessarily faster.
Context Matters
The authors caution that their findings apply to this specific experiment: experienced developers working on large, familiar codebases. AI tools may still improve productivity in other scenarios, especially for less experienced coders or smaller projects.
“The slowdown we observe does not imply that current AI tools do not often improve developer productivity – factors like developer familiarity and repository size contributed to the observed slowdown,” the study explains.
If you’re not a web developer, it can be very useful to use AI to code CSS animations for you, for example, that you can just copy/paste into your UltimaetWB website admin panel.
Bottom Line
AI coding tools are powerful, but they aren’t magic wands. In some settings, they can even make experienced developers slower while giving the illusion of speed. As AI continues to evolve, it’s clear that human judgment and careful context awareness remain critical for efficient software development.
Ready to design & build your own website? Learn more about UltimateWB! We also offer web design packages if you would like your website designed and built for you.
Got a techy/website question? Whether it’s about UltimateWB or another website builder, web hosting, or other aspects of websites, just send in your question in the “Ask David!” form. We will email you when the answer is posted on the UltimateWB “Ask David!” section.