WASHINGTON, D.C.: The heads of leading artificial intelligence companies have urged the United Nations to establish safeguards for the rapidly advancing technology, warning that poorly managed AI could pose serious risks to humanity.
Speaking before the UN Security Council, Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman called for international measures to prevent AI from becoming too powerful to control. Amodei warned that poorly managed AI could become “a risk to humanity as a whole,” while Altman said there was a possibility that humanity could “lose control of the future to AI.”
AI regulation takes centre stage at UN Security Council
The technology leaders said governments must work together to establish safeguards while ensuring that AI’s growing power does not become concentrated in a single company or country. At the same time, they highlighted AI’s potential to improve people’s lives and expand individual capabilities.
British Foreign Secretary Ed Miliband said the warnings from technology executives demonstrated that the industry could not be expected to regulate itself entirely.
“They are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world,” Miliband said, adding that their concerns should not be ignored.
Miliband also said that when the UK assumes the G20 presidency next year, AI governance will be a major focus. He said Britain would seek common international standards aimed at ensuring the technology develops safely.
US opposes new global AI governance framework
The United States, however, pushed back against calls for a new global governance structure for AI. White House Science Adviser Michael Kratsios said concerns about AI should not be used as a reason to halt further development or impose new international governance mechanisms. He instead urged allies and partners to share in the benefits of the technology.
The debate also reflects growing competition between the United States and China over AI leadership. Chinese UN Ambassador Fu Cong argued that countries should retain the freedom to choose the AI technologies they use.
Fu warned against creating an “us-vs-them” divide in the technology sector and accused unnamed major powers of pursuing what he described as “digital hegemony.” He promoted open-source AI models, which China has increasingly supported.
Who should control life-and-death decisions?
A central concern raised at the meeting was whether humans or machines should ultimately control decisions involving life and death. Amodei called for international agreements preventing AI systems from being used to create biological weapons. He also urged countries to develop ways to verify the capabilities of AI models and establish common standards for safety testing, misuse prevention and situations involving loss of human control.
OpenAI CEO Sam Altman and Hugging Face CEO Clément Delangue also addressed the Security Council, with Delangue participating remotely.
French Foreign Minister Jean-Noël Barrot, whose country convened the meeting, compared the challenge posed by AI with the emergence of nuclear technology. He also referenced HAL 9000, the fictional computer from 2001: A Space Odyssey, to illustrate concerns about systems becoming difficult to control.
“Just as with the atom back then, the international community must now regulate AI in order to make the most of its potential and avoid the worst,” Barrot said.
Concerns over autonomous weapons
The discussion comes amid growing international concern over the use of AI in warfare. UN Secretary-General António Guterres warned a day earlier about the danger posed by “killer robots.” Ukrainian President Volodymyr Zelenskyy also called for human control over autonomous systems, saying such safeguards were urgently needed as warfare increasingly incorporates advanced technologies.
Zelenskyy said there was already a possibility that AI systems could begin influencing battlefield decisions traditionally made by humans. He argued that peace was needed before such systems became more deeply integrated into warfare.
Information technology professor Kentaro Toyama of the University of Michigan said in an email that autonomous weapons were no longer merely a science-fiction concept. Data scientist Rumman Chowdhury, co-founder of Humane Intelligence, similarly argued that the key issue was determining who authorises an AI system to select and attack a target and who remains accountable for the consequences.
US-China AI competition intensifies
The UN discussions took place as Chinese President Xi Jinping arrived in Washington for a closely watched state visit with US President Donald Trump, with AI expected to remain an important area of discussion between the two countries.
US Treasury Secretary Scott Bessent had earlier said Washington and Beijing had discussed a possible “notification mechanism” for AI incidents that could have national-security implications.
Trump, meanwhile, has described AI as “superintelligence” and opposed efforts to impose controls that he believes could hinder the United States in its competition with China. Kratsios echoed that position at the Security Council, arguing that international cooperation should focus on benefiting from AI rather than constraining its development through a new global governance structure.
