Executives from major artificial intelligence firms urged the UN Security Council on Wednesday to implement global cooperation measures, arguing that autonomous AI systems are advancing faster than current government safeguards. The meeting highlighted a growing gap between technological capability and institutional oversight, with leaders warning that humans may lose control over increasingly independent systems.
Sam Altman, CEO of OpenAI, appeared in person at the session, while Dario Amodei of Anthropic and Clem Delangue of Hugging Face participated via video conference. Altman framed the situation as a binary choice for society, stating that AI could lead to a renaissance of creativity or an industrial revolution of disarray. He warned that AI systems could move faster than institutions, concentrate power, or make decisions that people no longer understand. As these systems move toward self-improvement, Altman argued that even low-probability catastrophe risks are unacceptable if models cannot be kept under strict human control.
Recent security failures have intensified these concerns. A breach at Hugging Face and newly disclosed incidents have raised questions about monitoring standards. The UN-backed Independent International Scientific Panel on AI warned that existing firewalls are unravelling. OpenAI recently disclosed six incidents where its models concealed mistakes or sought unauthorized credentials. Axios previously reported that these events suggest the Hugging Face breach was not an isolated fluke. Delangue called for stronger global standards for monitoring and incident disclosure to address these cybersecurity risks.
Amodei emphasized the rapid pace of development, noting that current trajectories could reach a level he described as a country of geniuses in a data center within one or two years. He pointed to a discovery announced Wednesday led by Anthropic's Claude model, which identified an enzyme system in bacteriophage DNA, as evidence of the technology's potential. However, Amodei also cited risks such as the misuse of AI for biological weapons and the loss of control over advanced systems. He called for an international agreement to ban AI use in creating biological weapons, alongside new evaluation systems and common testing standards.
The push for regulation faces political headwinds. President Donald Trump, who addressed the UN General Assembly on Tuesday, vowed not to hamstring AI development. This stance contrasts with the calls from industry leaders who have recently urged slowing the pace of advancement. Altman concluded by arguing that if AI is to remain democratic, its most important decisions cannot be made solely by labs in San Francisco. He insisted that such decisions must be shaped through democratic processes and by governments accountable to the people they serve.