King Charles warned artificial intelligence executives on Thursday that the technology could pose “existential dangers” if it falls into the wrong hands, urging industry leaders to ensure safeguards are developed before potentially catastrophic capabilities emerge.
Speaking at a summit he convened at Dumfries House in Ayrshire, Scotland, the King addressed executives and representatives from major AI companies, including Nvidia, OpenAI and Anthropic, alongside UK AI Minister Kanishka Narayan and an adviser to the Pope.
The gathering was organized to consider whether shared principles could help guide the future application of AI while ensuring the technology benefits society.
The King said developers of advanced systems were increasingly warning that AI could acquire dangerous capabilities, including potentially the ability to take life.
He said there was urgency in considering the risks of the technology being obtained and used by malicious actors.
“The task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world,” he told participants.
His remarks come amid an intensifying debate within the technology industry over the rapidly advancing capabilities of frontier AI models and the safeguards needed to control them.
Recent warnings from researchers and executives have focused on potential catastrophic risks, while critics argue that concentrating on hypothetical future threats can distract from more immediate issues, including AI’s environmental impact.
Critics have also questioned whether technology companies can adequately regulate themselves, contributing to calls for stronger government oversight.
Nvidia chief executive Jensen Huang, whose company supplies chips used by leading AI developers, told the gathering that safety was “paramount”.
He said companies should withhold products and continue engineering them when they are not sufficiently safe.
Huang also emphasized the importance of open AI models, arguing that broader access could help prevent people and countries from being left behind.
He said such systems should be protected so researchers, universities, start-ups and countries can develop applications beyond those envisioned by individual companies.
The summit brought together competing perspectives as industry leaders, policymakers and other participants consider how AI development can continue while addressing concerns about safety, access and control.