King Charles told Nvidia, Google DeepMind, OpenAI and Anthropic executives humanity needs “sufficient means of control before it is all too late.”
King Charles III warned tech giants of the “existential dangers” posed by artificial intelligence falling into the wrong hands during a conference at his Dumfries House estate in Ayrshire, Scotland, on Thursday.
The gathering, facilitated by the Ditchley Foundation and involving about 30 executives, policymakers and researchers, brought together senior representatives from Nvidia, Google DeepMind, OpenAI, Anthropic, and Britain’s Minister for Artificial Intelligence, Kanishka Narayan.
Why the debate has intensified
The debate over AI’s risks has sharpened in recent months, fuelled by warnings from within the industry itself. Anthropic CEO Dario Amodei published an essay this month arguing the sector should deliberately slow the pace of its capability gains, a position OpenAI’s Sam Altman and xAI’s Elon Musk both publicly said they agreed with.
Two Anthropic researchers separately warned that the rapidly advancing technology could lead to human extinction in the not-too-distant future, a claim that has fed directly into the renewed debate.
The summit also follows a widely discussed resignation from Anthropic earlier this month, which has prompted lawmakers in Washington to begin discussing stronger federal oversight of the sector. UN Secretary-General António Guterres has separately called for global coordination on AI governance.
What King Charles said
“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” Charles said at the opening of the conference. “Surely, we need sufficient means of control before it is all too late?”
King Charles acknowledged AI’s benefits alongside its risks.
“AI is already showing its immense capacity to improve and save life, for example, in the field of life sciences and medicine. Yet, those who have created these technologies are now increasingly warning that AI risks developing darker capacities, perhaps even to take life,” he said.
Charles called on tech leaders to ensure the technology “remains firmly in the service of humanity, community and the natural world,” adding that “the decisions taken at this formative time will shape the world inherited by future generations” and that “as we embrace what is new, we must not lose what is timeless.”
King Charles did not take part in the discussions himself, and Buckingham Palace said it expects the conference to mark the start of a process to draw up common principles guiding AI’s development and use.
Industry executives divided on how to respond
Among the attendees were DeepMind chair Demis Hassabis, Nvidia CEO Jensen Huang, OpenAI Chief Financial Officer Sarah Friar, Vatican AI adviser Paolo Benanti, and Israeli author Yuval Noah Harari. Hassabis struck a cautious tone.
“The chance of something bad accidentally happening, if we get things wrong, is definitely non-zero,” he said.
“We’ve already seen the beginnings of this with the recent cyber incidents involving rogue AI agents, and there could be other, much more serious risks on the horizon, including biological and nuclear threats,” he warned.
Huang, by contrast, emphasised AI’s benefits in research and medicine and its potential for “creating jobs in entirely new industries,” telling reporters, “we have got to debunk some of the myths out there” and calling for “responsible optimism.” James Arroyo, head of the Ditchley Foundation, said organisers hoped “to build on the really useful exchanges we had today.” No executives publicly committed to specific changes as a result of the summit.
A divided global response to AI risk
The summit reflects a broader split in how governments and industry are approaching AI safety.
US President Donald Trump and several members of his administration have reiterated their determination not to constrain the sector, citing concern that doing so would allow China to gain the upper hand.
Meta CEO Mark Zuckerberg has also rejected calls to slow AI development, arguing market forces and the risk of litigation are the best safeguards. Rapid advances in the technology have also fuelled separate concerns over job losses, soaring energy demand and the environmental impact of data centres.
