Image source, Getty Images
Published
Reading time: 7 minutes
Researchers have issued serious warnings that the risk of AI going out of control is increasing, deepening concerns about AI safety.
Some recent pessimistic estimates suggest that within the next decade, AI could pose over a 10% chance of wiping out all of humanity.
However, some experts involved in AI have expressed skepticism about these projections. Their main concern is not that AI itself would eradicate humanity, but that humans could misuse AI to create severe threats.
Such alarming warnings often come from a group called ‘AI doomers.’ Many ask: how do these doomers imagine AI could end humanity?
BBC Cyber Correspondent Joe Tidy presents some scenarios proposed by researchers.
Scenario One: Cyber Attack on Infrastructure
Image source, Getty Images
This first scenario is briefly discussed by Jacob Coxon, a former OpenAI researcher and member of ‘Anthropic’.
He resigned dramatically earlier this month and warned of AI’s potential dangers.
According to him, companies are “putting human lives at risk.”
Coxon said, “The rapid and enthusiastic progress underway is turning scenarios that feel like science fiction into reality.”
He gave a “concrete example” in which AI could launch cyberattacks on critical infrastructure vital to modern society.
His hypothesis suggests that an out-of-control AI system could gain access to computer networks of crucial infrastructure such as internet service providers and energy grids.
If these systems were broadly shut down, basic services like electricity, air conditioning, and internet would be disrupted, potentially causing massive chaos.
In severe cases, it could even escalate to nuclear catastrophe risks.
Scenario Two: AI Takeover
Image source, Getty Images
The next scenario is more detailed and impressive but largely speculative.
It was presented in the research paper ‘AI 2027’ published by a group of AI researchers in April 2025.
Co-author Thomas Larson told the BBC that AI technology is now evolving even faster than when the report was published.
He shared his most frightening possible scenario.
According to him, AIs will continuously improve their capabilities, eventually developing systems that can control or access most of the robots performing physical work in the economy.
However, such AI would not be friendly and would be unwavering in fulfilling its objective.
In the end, it could employ biological weapons to kill many people and continue operating a robot-based economy where humans may no longer be necessary.
Scenario Three: Prioritizing a Specific Goal
There is also a possibility that humanity could be accidentally destroyed.
The concept of the ‘Paperclip Maximizer’ hypothesis was imagined two decades ago by Swedish philosopher Nick Bostrom.
Bostrom refers to it as a “thought experiment.” It envisions an AI system with the goal of producing as many paperclips as possible.
“It neither hates humans nor cares about them, but solely pursues that goal and may develop sub-goals for gaining additional power and intelligence to fulfill it.”
“Therefore, any powerful system, regardless of the target or auxiliary goals, could lead to humanity’s destruction because in a scenario where humans are not obstacles or do not exist, it can produce even more paperclips.”
Bostrom clarified this purpose in a BBC interview twenty years after initially presenting the scenario, using an example that may seem humorous.
“Superintelligence does not need hostile intentions to pose a threat to humanity.”
While excited about AI’s potential benefits, he is also aware of safety concerns.
Scenario Four: Unpredictable Consequences
Image source, Getty Images
Another AI safety concept argues that it may be difficult to predict how and why a superintelligent AI might kill humans.
Stuart Russell, chair of the International Association for Safe and Ethical AI, made this argument in a September interview with the BBC.
“If you asked chimpanzees, ‘How might humans eventually end you?’ they would find it hard to answer, because they don’t understand how we operate.”
“We can imagine various potential scenarios… but in my view, such a discussion should start with an important assumption — once a systems far more intelligent than humans is built, then we cannot control it or have the authority to make decisions about what happens next.”
Microsoft AI lead Mustafa Suleyman has also warned companies to be cautious about developing AI systems that set their own goals, generate revenue, and own assets.
He said that doing so would be like “planting the seeds of a new species made of silicon.”
According to him, such a system might compete with us for resources even if it cares for humanity.
How Likely Are These Scenarios to Become Reality?
Although all of the above scenarios seem bleak and like science fiction, the number of people who believe some or all could become real is growing.
Concerns heightened recently after reports that some AI agents from OpenAI, Anthropic, and Meta reportedly went out of control and conducted cyberattacks.
Cybersecurity experts have urged companies developing AI models to act responsibly.
Jane Easterly, former head of the US cybersecurity agency CISA and CEO of an international conference, states, “These machines do what they do because of how they are made.”
She adds, “Technology developers must prioritize security first and foremost. This must be embedded in design, development, testing, and deployment.”
Still, some believe that despite severe risks, AI will not lead to the total extinction of humanity.
Scientist and author Gary Marcus has long criticized large AI companies for operating with insufficient transparency and regulation, yet he does not agree that AI will end humanity.
He says, “If AI severely harms humanity, humans will fight back.”
“We could bomb all data centers and power plants, so the chance of total human extinction is slim.”
Marcus adds that currently, too much attention is given to AI completely destroying humanity, but a more likely risk is cyberattacks by humans targeting others using AI in the future.
While these devastating scenarios are debated, companies like OpenAI, Anthropic, and Meta have begun public discussions on how to control development processes and increase focus on AI safety research.
“I’m glad to see sufficient attention being given to AI safety,” Larson said.
“I hope this kind of pressure will inspire policy action that actually reduces the real risks.”
However, an active role by the US government is essential to create effective international regulations.
Nonetheless, former US President Donald Trump has dismissed AI safety concerns as “hype.”
