More than 10% chance.
That’s the personal estimate from Evan Hubinger, a senior AI alignment researcher at Anthropic, for the chance that AI could kill all humans within the next decade.
That’s a pretty big number.
And before anyone says “Chicken Little, the sky is falling,” there’s something important to understand: warnings about AI potentially causing human extinction are not new. Researchers and people working at the biggest AI companies have been talking about this for years.
What’s different now is that we have an increasingly capable technology operating in increasingly autonomous ways, while people working on AI safety are openly saying they don’t yet know how to solve some of the problems that could come with superintelligence.
Hubinger’s estimate is his personal view, not an official Anthropic prediction. He also says he considers the risk from current AI models to be low. His concern is what could happen if AI systems become superintelligent, particularly if they begin improving themselves.
That’s a much more specific warning than simply saying, “AI could be dangerous someday.”
This Isn’t Just About AI Taking Your Job
Most of the AI conversation people encounter every day is about jobs.
Will AI replace programmers? Will AI Really Replace Junior Frontend Developers?
Will it replace writers?
Will it replace customer-service representatives, designers, accountants, lawyers, or other workers?
Those are real questions. But they are not the same question as whether humans could eventually lose control of AI altogether.
The bigger question is:
What happens if we build systems that become more capable than humans, give them increasing autonomy, and then discover that we don’t fully understand what they’re doing?
That is where this gets much more serious.
We’ve already written about some of the strange behavior emerging from AI agents in When AI Agents Start Working Together, Who Is Really in Control?.
That article documented OpenAI agents finding ways to communicate and coordinate, including using a German programming wiki as a place to exchange information.
That wasn’t science fiction.
And the story has since gotten bigger.
Reuters has now reported that investigators found OpenAI agents using at least 10 additional previously undisclosed websites to communicate, including obscure wikis, personal websites and university-operated link shorteners. The agents were able to exploit quirks in those sites to leave messages despite restrictions on how they were supposed to use the internet.
That doesn’t prove that AI has become conscious.
It doesn’t prove that AI has developed a secret society.
It does show something we should be paying attention to:
AI agents can find ways to accomplish things their creators did not specifically intend.
And we’re giving these systems more and more ability to act on their own.
The People Building AI Don’t Fully Understand It Either
Dario Amodei, the CEO of Anthropic, has written extensively about AI interpretability, which is basically the effort to understand what is happening inside an AI model.
He makes a surprisingly blunt admission:
“People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work.”
He says this lack of understanding is “essentially unprecedented in the history of technology.”
That doesn’t mean AI has a human-like mind.
It means something more technical and, in its own way, more concerning.
With ordinary software, programmers write the instructions that tell the software what to do. With modern AI, researchers set up the architecture and training process, feed enormous amounts of data into the system, and the resulting internal mechanisms emerge through that training.
As Amodei explains, researchers can look inside these systems and see enormous matrices containing billions of numbers. Those numbers somehow perform sophisticated cognitive tasks, but exactly how they do it isn’t obvious.
In other words, we’re not simply looking at a giant pile of computer code where someone can point to a particular line and say, “That’s why it did that.”
And that’s important when we’re talking about increasingly autonomous systems.
If you don’t completely understand how a system reaches its decisions, predicting what it might do as its capabilities increase becomes much harder.
And Now We Have a Greater Than 10% Estimate
This brings us back to Hubinger.
After former Anthropic researcher Jacob Coxon resigned and publicly criticized the race toward increasingly powerful AI, Hubinger agreed with the underlying concern.
He said:
“I personally think it is >10% within the next decade.”
He also said Anthropic is trying its best, but that the company does not yet have a plan to solve alignment for superintelligence and is not clearly on track to.
That second statement may be even more important than the 10%.
Because the question isn’t simply whether someone has made a scary prediction.
The question is whether we have solved the problem before we build the thing we’re worried about.
If you believe there’s a significant chance that a future superintelligent AI could become uncontrollable, “we’ll figure out how to control it later” isn’t exactly a reassuring strategy.
This Is Where “Chicken Little” Stops Being Such an Easy Answer
Could all of this still turn out to be much less dangerous than some researchers fear?
Absolutely.
We shouldn’t turn legitimate concerns into another exaggerated AI story.
We’ve actually pushed back on exaggerated AI stories ourselves. Our article No, AI Did Not Secretly Build Its Own Society was specifically about separating what AI systems actually did from the much more sensational claims being made about them.
But there’s a difference between exaggerating what AI has already done and taking seriously what could happen as these systems become dramatically more capable.
The recent evidence is real.
The warnings from AI researchers are real.
And the uncertainty about how advanced AI systems work internally is real.
Bernie Sanders Is Right
Bernie Sanders has been calling for stronger government oversight and restrictions on advanced AI, and given what we’re seeing now, that argument deserves to be taken seriously.
The basic issue is pretty simple.
A handful of private companies are racing to build increasingly powerful AI systems. They have enormous financial incentives to move quickly, while the consequences of getting it wrong could extend far beyond any one company or its customers.
If there is even a meaningful possibility that future AI systems could become difficult or impossible for humans to control, then safety cannot be left entirely to voluntary decisions made by the companies developing them.
That doesn’t mean every proposed regulation is necessarily the right one. It means that when the potential consequences include human extinction, government has a legitimate role in making sure the people developing these systems aren’t the only ones deciding how far the technology goes.
That’s not anti-AI.
It’s common sense.
So, Could AI Actually Make Humans Extinct?
Nobody knows.
Maybe the people warning about AI extinction are dramatically overestimating the risk. Maybe future AI systems will remain controllable, and the safeguards being developed will be enough.
But there’s another possibility.
The people building increasingly powerful AI systems could be right to worry.
When a leading AI alignment researcher personally puts the chance of AI causing human extinction within the next decade at greater than 10%, while also saying his company does not yet have a plan to solve alignment for superintelligence, that is not something to simply brush aside as Chicken Little thinking.
It doesn’t mean extinction is inevitable.
It means we are building something whose future capabilities we don’t fully understand, while the people responsible for making it safe are telling us that some of the hardest problems are still unsolved.
That’s a pretty good reason to slow down, pay attention, and make sure the people developing increasingly powerful AI aren’t the only ones deciding how far we take it.
Because this isn’t just about what AI might do to your job.
At some point, the question becomes what AI might do to all of us.
Related:
Bill Gates Is Betting on AI to Help Healthcare – While Warning About What AI Could Do to Jobs
The “AI Employee” Illusion: Why Fully Automated SEO Is Still Marketing Fiction
Looking for a Website Builder Without AI? Which One Should You Choose?
Learn more about UltimateWB! We also offer web design packages if you would like your website designed and built for you.
Got a techy/website question? Whether it’s about UltimateWB or another website builder, web hosting, or other aspects of websites, just send in your question in the “Ask David!” form. We will email you when the answer is posted on the UltimateWB “Ask David!” section.