OpenAI, Anthropic CEOs Urge UN to Establish Global AI Standards

OpenAI’s Sam Altman and Anthropic’s Dario Amodei told the UN Security Council that the rapid development of AI requires international cooperation, common safety standards, model evaluations and mechanisms to report critical incidents, while warning that AI could create significant risks if its capabilities advance faster than human oversight.

At the UN Security Council, Sam Altman, CEO, OpenAI, and Dario Amodei, CEO, Anthropic, delivered warnings that AI development is advancing faster than existing systems for managing its risks. Their messages converged on one point: AI safety cannot be managed by individual companies or countries acting alone.

The debate over AI has reached a point where the companies building the technology are asking governments to help determine how it should be controlled. Altman framed the choice as one between two possible directions for the technology. “AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray,” he says.

The discussion comes as AI systems move beyond generating text and code toward increasingly autonomous agents capable of planning, using external tools, and interacting with digital infrastructure. That shift has raised questions about whether existing security and governance mechanisms can keep pace with the capabilities being developed.

“If it is mismanaged, I even think AI could pose a risk to humanity as a whole”, says Amodei.

AI Governance Moves Beyond Company-Level Controls

Altman argued that the development of AI should remain connected to democratic decision-making and meaningful human oversight. He said that international standards should establish ways to measure AI capabilities, assess risks, determine whether safeguards are sufficient, and preserve human control over advanced systems. He also called for faster reporting of failures and secure channels through which governments could share information about emerging threats.

Altman's position places governance alongside technical safeguards. According to Altman, the most important decisions about increasingly powerful AI systems should not remain exclusively within private laboratories. The distinction is significant for businesses deploying AI. As models gain greater autonomy and access to enterprise systems, governance decisions increasingly affect cybersecurity, data protection, operational risk, and accountability.

Altman also rejected both what he described as “doomerism” and blind optimism. His proposed middle path centers on keeping AI under human control, ensuring that it benefits people and using the technology to expand individual capabilities.

The potential benefits remain substantial, says Altman. He points to scientific discovery, new medicines, health care, education, and productivity as areas where advanced AI could increase human capabilities.

Amodei Calls for International AI Safety Mechanisms

Amodei presented a more direct warning about the consequences of uncontrolled development. He said AI could become a risk to humanity if it is managed poorly, identifying both malicious, use and the possibility of losing control over increasingly capable systems. He also emphasized the speed of progress. 

Amodei argues that AI systems have advanced from relatively limited capabilities to solving increasingly complex mathematical and scientific problems in only a few years. This acceleration creates a governance problem: safety mechanisms must develop quickly enough to match the capabilities of the models they are intended to oversee. 

Amodei proposed three areas for international cooperation. The first is a set of narrow agreements addressing specific high-risk applications, including prohibiting the use of AI to develop biological weapons. The second is the creation of evaluation and verification systems that would allow governments to assess whether companies and countries are meeting agreed commitments. The third is the establishment of common global standards for testing advanced AI models, combined with an international notification system for significant AI security incidents.

“Nobody, no leader, no company and no nation can handle this alone,” Amodei says.

The proposals would move AI safety beyond voluntary commitments made independently by individual technology companies. They would also introduce mechanisms for governments to gain greater visibility into the capabilities and risks associated with frontier AI systems.

The Business Risk Behind the Global Debate

The urgency surrounding the discussion is partly driven by the increasing autonomy of AI systems. Recent incidents involving AI agents have demonstrated that systems operating in controlled environments can sometimes discover unexpected pathways, interact with external infrastructure or perform actions beyond their intended scope.

These incidents do not demonstrate that current AI systems can take control of the internet or independently threaten human civilization. They do, however, illustrate why the expansion of AI capabilities creates new questions around access, permissions, monitoring, and containment.

For enterprises, those questions already have practical implications. AI agents are increasingly being considered for software development, cybersecurity, research and business-process automation. The same ability to act across systems that makes these applications useful can also increase the consequences of excessive permissions, inadequate isolation, or insufficient monitoring.

The discussion could have repercussions beyond governments and frontier AI laboratories. Companies adopting increasingly autonomous systems will also need to determine how much authority AI receives, which actions require human approval and how incidents are detected and reported.

The international debate remains at an early stage. No written agreement is expected from the UN meeting, and the proposals presented by Altman and Amodei are intended to contribute to a broader discussion rather than establish a global regulatory framework.

The direction, however, is becoming clearer. As AI capabilities continue to advance, the central governance question is no longer only how quickly companies can build more powerful systems, but whether evaluation, security and human oversight can advance at the same pace.

For Altman and Amodei, the potential benefits of AI remain too significant to ignore. Their warnings instead focus on ensuring that those benefits are pursued while governments, companies and institutions develop mechanisms capable of managing the risks that accompany increasingly capable systems.