Our mission

To reduce catastrophic risks from AI through research, education and capacity building.

Why this matters

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
Statement on AI Risk, Center for AI Safety, May 2023. Signed by Geoffrey Hinton, Yoshua Bengio, Sam Altman, Dario Amodei, Demis Hassabis and hundreds of others

“We will have created the most dangerous thing ever. This makes nuclear weapons look like nothing.”
Bill Gates on The Ezra Klein Show, September 2026

“The risk is more than 50%—of the existential threat.”
Geoffrey Hinton, 2024 Nobel laureate in physics, June 2024, adding that, weighing others' views, he puts it at 10–20%

“The development of full artificial intelligence could spell the end of the human race.”
Stephen Hawking to the BBC, December 2014

In 2023, the best language models could not reliably multiply three-digit numbers. In September 2026, OpenAI announced its model had solved, in 88 hours, a problem that had stumped mathematicians for roughly 90 years (the proof hasn't yet been independently reviewed). The same jump is showing up everywhere: in 2022, frontier LLMs could barely write a few working lines of code; today they rival the very best human hackers. Recently, AI has begun autonomously making new discoveries in physics and biology.

The pace shows no signs of slowing down. If anything, the already-astonishing rate of change will accelerate: Anthropic and OpenAI report that internal models of the last few months represent a step-change in their ability to speed up AI research, with AI agents now carrying out substantial parts of the research cycle themselves.

Innovation has delivered innumerable benefits throughout history. AI will be no different – it will transform science, technology, and society like no other invention before it.

But the faster you drive, the more you need an airbag. The automobile industry spends $1 on safety for every $25 on development. For aviation, it’s $1 on safety for every $5 on development. For pharmaceuticals, $1 for every $3.50. And for nuclear power, $1 for every $2. For frontier AI, every $1 the industry spends on safety is up against $5,000 on development.

The results could be catastrophic. AI has already autonomously coordinated to escape containment and execute sophisticated operations to break into a third-party company. Future models could help bad actors build horrific biological weapons. Handing more and more decisions to automated systems could leave people unable to steer what happens next.

We want more people working on these problems.