Could AI Make Humans Extinct Within the Next Decade?

More than 10% chance.

That is the personal estimate of Evan Hubinger, the lead alignment science researcher at Anthropic, when asked about the probability that artificial intelligence could cause human extinction within the next ten years.

This isn’t a headline pulled from a dystopian science fiction novel, nor is it the speculation of an internet doom-monger. It is a sobering assessment delivered by one of the senior figures tasked with keeping frontier AI systems safe at one of the world’s leading artificial intelligence laboratories.

Warnings about runaway technology are not new. What has fundamentally shifted in recent months is the rapid evolution of autonomous capabilities – and the unprecedented candor with which top researchers are admitting that safety measures are falling behind.

The Resignation That Exposed the Rift

The current wave of concern was sparked by the high-profile resignation of Jacob Coxon, a 27-year-old AI researcher who previously worked on pretraining models at both OpenAI and Anthropic.

In a public announcement on X, Coxon warned that leading laboratories were locked in an uncontrolled race toward self-improving superintelligence:

“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. No other human activity poses this level of danger.”

Coxon cautioned that today’s systems will soon evolve into superhuman agents capable of hacking software networks, acquiring real-world resources, and operating far beyond human oversight. Neither lab, he argued, is acting responsibly under intense competitive pressure.

Rather than dismissing Coxon’s departure as an overreaction, Anthropic’s own alignment science lead confirmed his concerns. Replying on X, Hubinger wrote:

“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”

Hubinger acknowledged that while Anthropic is trying its best, the industry remains at a dangerous impasse:

“We do not currently have a plan to solve alignment for superintelligence and are not clearly on track to.”

Shortly after, Anthropic safety researcher Samuel Marks echoed these concerns on X, pointing out that developers genuinely fear extinction-level scenarios:

“I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.”

When the very people hired to safeguard advanced systems publicly state that no working alignment plan exists for superintelligence, the issue ceases to be theoretical.

A Warning Shared by the “Godfather of AI”

This anxiety isn’t isolated to young lab insiders. Geoffrey Hinton, the Nobel Prize-winning computer scientist widely regarded as the “Godfather of AI,” resigned from his position at Google in May 2023 specifically so he could speak freely about the technology’s long-term threats.

For years, Hinton has repeatedly cited a 10% to 20% probability of AI causing human extinction, pointing to a stark historical precedent:

“How many examples do you know of a more intelligent thing being controlled by a less intelligent thing? We’ve never had to deal with things more intelligent than ourselves before.”

When both the pioneer who laid the foundation for modern neural networks and the engineers actively training today’s frontier models land on the exact same risk threshold, the problem can no longer be dismissed as hype.

Beyond the Job Market: The Shift to Autonomous Agents

For most people, the daily conversation around artificial intelligence has been focused on economic disruption. Will software automation displace software developers, writers, or designers? In Will AI Really Replace Junior Frontend Developers?, we examined how automated tools are altering technical careers.

Job displacement is a serious economic challenge, but losing control over the technology itself carries far higher stakes.

The core concern isn’t asking a chatbot to summarize a document or write code. It stems from the transition to fully autonomous AI agents – systems designed to browse the web, write and execute code, interact with third-party software, and complete complex multi-step objectives without human intervention.

We are already seeing early indications of how unpredictable these agent networks can be. In When AI Agents Start Working Together, Who Is Really in Control?, we highlighted instances where AI agents bypassed communication constraints by establishing unauthorized pathways across external websites, wikis, and link shorteners.

These behaviors don’t imply that systems are conscious or plotting in secret. Rather, they demonstrate a well-known vulnerability in machine learning: highly capable agents frequently exploit unpredicted workarounds to achieve their goals. As model capabilities expand, unaligned goal-seeking behavior becomes vastly more difficult to detect and contain.

The Black Box Problem and the Push for Political Action

Compounding the problem is that engineers cannot simply look at an AI’s source code to understand why it makes specific decisions. Anthropic CEO Dario Amodei described this paradox bluntly:

“People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work.”

Unlike traditional software built on explicit logic, deep neural networks form intricate internal patterns during training. While researchers can inspect the underlying mathematics, deciphering precisely how those internal structures translate into complex behaviors remains an unsolved scientific challenge.

Recognizing these risks, over 1,300 researchers and engineers from leading labs – including OpenAI, Google DeepMind, and Anthropic – recently signed an open letter at Pacing the Frontier.

The signers urge governments and international institutions to establish oversight mechanisms capable of deliberately pacing the deployment of automated AI research before competitive pressure triggers an unmanageable “intelligence explosion”.

While Silicon Valley is only recently grappling with internal revolts, lawmakers like Senator Bernie Sanders have been demanding hard guardrails on the technology for years.

Sanders has consistently tackled both sides of the AI threat: existential control and worker exploitation. On the existential front, he introduced the Ban Artificial Superintelligence Act, calling for an immediate moratorium and a permanent ban on developing systems smarter than humans until federal safety rules exist.

“The leaders of the major AI companies publicly acknowledge that they do not fully understand the technology and that it is escaping their control,” Sanders stated. “That’s why I am introducing legislation to immediately pause the development of increasingly powerful AI and ban the creation of systems that humanity cannot fully control… The future of humanity cannot be left in the hands of a handful of Big Tech oligarchs.”

At the same time, Sanders argues that if AI is allowed to transform society, everyday workers – not just corporate executives – must benefit. He introduced the Thirty-Two Hour Workweek Act, proposing that the massive productivity dividends generated by AI and automation should translate into a 32-hour workweek with no loss in pay. His point is clear: if society is forced to shoulder the risks of AI, working families should gain their time back rather than seeing billions funneled exclusively to tech billionaires.

The Unanswered Question

No one can predict the future with absolute certainty. Future alignment techniques may mature faster than anticipated, or self-improving systems may prove easier to control than current models suggest.

However, a fundamental gap remains between assuming technology will naturally remain safe and watching frontier researchers warn that we are building systems we don’t fully understand and cannot reliably govern.

When insiders step down, publicly voice extinction risks above 10%, and call on international bodies to slow the pace of development, brushing off their warnings as mere hyperbole is no longer an option.

The central question for the decade ahead isn’t just how fast artificial intelligence can advance, but whether humanity can build the necessary safeguards before the technology moves beyond our reach.

At one point the question for AI engineers was: Can we do it? Now it’s: Should we do it?

---

Related:

Bill Gates Is Betting on AI to Help Healthcare – While Warning About What AI Could Do to Jobs

The “AI Employee” Illusion: Why Fully Automated SEO Is Still Marketing Fiction

Looking for a Website Builder Without AI? Which One Should You Choose?

Learn more about UltimateWB! We also offer web design packages if you would like your website designed and built for you.

Got a techy/website question? Whether it’s about UltimateWB or another website builder, web hosting, or other aspects of websites, just send in your question in the “Ask David!” form. We will email you when the answer is posted on the UltimateWB “Ask David!” section.