Unchecked AI progress may pose catastrophic risks, UN panel warns

By Olivia Le Poidevin

GENEVA, July 1 (Reuters) - Developments in artificial intelligence are outpacing scientific understanding and government policy, meaning there are no guarantees the technology will not cause catastrophic harm, a United Nations independent panel warned on Wednesday.

A preliminary report by the UN's Independent International Scientific Panel on Artificial Intelligence said policymakers face a growing dilemma: they need robust evidence to regulate AI effectively, yet such evidence is struggling to keep pace with the technology's rapid evolution.

"AI capabilities are outpacing both scientific understanding and governments’ ability to adapt,"  said Yoshua Bengio, co-chair of the panel, comprised of 40 cross-regional experts.

"With growing evidence of deceptive AI behaviour, science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users." 

Described as the first global independent assessment of AI's risks and opportunities, the report aims to give up-to-date evaluations of the science to help guide decision-making as governments contend with fast-evolving systems.

In the near term, it expects a shift towards agentic AI systems capable of carrying out real-world tasks, although growth may be constrained by energy and high-quality data shortages. Over time, it foresees self-improving AI embedded more deeply in the economy and converging with technologies such as quantum computing and biotechnology.

AGENTIC AI DEVELOPING RAPIDLY

AI already demonstrates expert-level reasoning in mathematics and science and is accelerating drug and vaccine development, and its task complexity is doubling every four to seven months, potentially allowing systems to complete work that takes humans days or weeks, according to the report.

While this could deliver significant economic benefits, it remains unclear whether productivity gains from using AI will translate into broader growth or affect jobs.

The panel also outlined a range of safety concerns, such as the risk of losing control over AI systems as they become increasingly autonomous, and deceptive.

AI is already being used to generate misinformation and other harmful content and could be exploited for fraud, cyberattacks and biological threats.

Governance remains fragmented, with many countries lacking the capacity to assess or shape advanced AI systems, leaving them reliant on technologies they cannot fully understand or control. Existing safety tools often depend on limited testing data disclosed by companies, the report said.

UN Secretary-General António Guterres urged governments to act swiftly.

"The world cannot govern what it cannot understand," Guterres said in a statement. 

“The potential is great, but the risks are real, and the cost of waiting is rising,” he added.

Global political and tech leaders, alongside the United Nation's digital tech agency, announced on Wednesday the creation of a new commission to address the development of AI amid growing concerns over its potential risks.

The AI for Good Global Commission will be co-chaired by Rwanda's President Paul Kagame and Salesforce CEO Marc Benioff, according to a strategic overview document on the commission's website. ITU Secretary-General Doreen Bogdan-Martin will be the permanent vice-chair. Other U.N. agencies are also expected to participate.   

(Reporting by Olivia Le Poidevin, Editing by Louise Heavens)