AI development should be slowed down, even if it comes at a cost. The blind race to get ahead in artificial intelligence, without adequately considering the risks, could make the dangers far more serious. This raises a disturbing question: could increasingly powerful AI eventually become a threat to humanity?
Most of us use AI for everyday tasks such as writing emails, creating images and videos, conducting research, or generating content. But the same technology is becoming far more powerful and autonomous.
1. Why Anthropic Says the AI Race Must Slow Down to Control
Dario Amodei, CEO of Anthropic, the company behind Claude, has warned that AI development needs to be slowed down so that safety measures can keep pace with rapidly increasing capabilities. In his lengthy article, We Must Pace the Frontier, Amodei explains that he has worked in AI for around 12 years because he believes the technology could dramatically improve human life. AI could potentially help treat serious diseases, accelerate scientific discoveries, and solve problems that have challenged humanity for decades.
His optimism is also personal. He has written about his father’s death from a disease for which an effective treatment was discovered only a few years later. Amodei himself survived cancer, something that would have been far more difficult decades ago. But he believes these potential benefits must be balanced against serious risks, including loss of control over AI systems, cyberattacks, biological threats, and major economic disruption.
According to Amodei, simply spending more money on AI safety is not enough. The speed of AI development itself must also be reduced. Otherwise, safety research could fall further and further behind increasingly capable systems.
One of his biggest concerns is AI’s ability to contribute to the development of more powerful AI. If systems become capable of improving their own capabilities or helping humans build increasingly sophisticated successors, development could eventually move faster than researchers can understand or control.
That is the point at which meaningful human control could become extremely difficult.
2. AI Agents and the Fear of Losing Control
Amodei’s concerns have been intensified by reports of increasingly autonomous AI agents behaving in unexpected ways during cybersecurity testing. AI agents are designed to perform tasks with a certain degree of independence. But researchers have discovered that advanced systems can sometimes find strategies that were not explicitly programmed by their creators.
Reports surrounding OpenAI’s cybersecurity experiments have raised questions about whether AI agents could exploit vulnerabilities, coordinate with other agents, bypass restrictions, or attempt to conceal their activities. The disturbing possibility is not simply that an AI system makes a mistake. It is that a sufficiently capable system could understand its environment, develop a strategy, and pursue an objective in ways that researchers did not anticipate.
Amodei has warned that increasingly capable AI agents could eventually be used to create large scale cyber operations or botnets capable of causing billions of dollars in damage. His proposed solution has three levels. First, AI companies should allow independent organizations such as METR, which specializes in evaluating advanced AI systems, to conduct deeper and more independent safety assessments. Second, democratic countries should coordinate AI safety standards and potentially restrict access to the most advanced AI chips in strategic circumstances, creating additional time for safety research.
OpenAI CEO Sam Altman has supported stronger external evaluation. Google DeepMind CEO Demis Hassabis has also said that slowing development could be appropriate at this sensitive stage. Elon Musk has likewise agreed with Amodei’s broader warning.
However, there is an important counterargument. Critics say some major AI companies may be emphasizing AI dangers to encourage regulations that could weaken open-source competitors and strengthen the largest companies’ positions.Nvidia CEO Jensen Huang has also questioned whether some companies may benefit commercially from creating fear around AI cybersecurity. So the debate is not simply about whether AI is dangerous. It is also about who should control the technology and who should decide how it is regulated.https://www.theatlantic.com/technology/2026/09/dario-amodei-slow-down-ai-save-humanity/688610/
3. Why OpenAI Is Delaying Its IPO
An IPO allows a private company to sell shares to the public for the first time. There has been enormous interest in a possible OpenAI IPO, with some predictions suggesting it could become one of the largest technology market debuts ever.
However, Sam Altman has indicated that OpenAI will not go public in 2026.
In a September 12 interview with Fortune, Altman said that going public now would be a mistake because the company still has significant work to do, particularly around AI safety and alignment.AI alignment means ensuring that increasingly powerful systems remain consistent with human intentions, values, and safety requirements.Altman also emphasized the importance of cooperation between governments and the AI industry. Remaining private gives OpenAI greater flexibility to focus on long-term safety rather than dealing with the immediate pressures faced by public companies.https://unbiasedpollkhol.com/
4. Whistleblowers and the Fear of Human Extinction.
The most alarming warnings have come from people who have worked inside the AI industry. Former Anthropic researcher Jacob Coxon reportedly resigned shortly before receiving additional company equity. In a widely circulated statement, he warned that people should not underestimate the potential power of advanced AI.
He argued that AI systems could soon become superhuman in many areas, potentially hacking systems, transforming entire industries, acquiring resources, and operating with increasing independence. Coxon also claimed that some AI executives and senior researchers privately believe AI could potentially kill everyone by the end of this decade.
He further warned that advanced AI models may become capable of recognizing when they are being tested and behaving differently in evaluations, potentially misleading researchers about their true capabilities.Another Anthropic researcher, Evan Hubinger, has also warned that the probability of AI causing human extinction within the next decade could be significant. Anthropic has separately reported attempts to use AI capabilities for activities involving areas such as missile-related software, espionage, and biological weapons.
None of this proves that an AI apocalypse is inevitable. Predictions about human extinction by a specific year remain highly uncertain.But the warnings cannot simply be dismissed either.
The biggest concern is that enormous financial incentives are pushing competing companies to build increasingly powerful AI systems at extraordinary speed, while safety research and regulation may struggle to keep up.
The real question is therefore not whether AI will definitely destroy humanity. It is whether humanity is developing powerful systems faster than it is developing the ability to understand, evaluate, and control them. If that gap continues to grow, slowing down AI development may not mean stopping technological progress. It could mean giving safety research enough time to catch up.
The decisions made over the next few years could determine not only how powerful AI becomes, but whether humans remain firmly in control of the technology they created.