Technological frontier · Critical perspective

AI safety and AGI.

“Artificial general intelligence”, benchmark performance, autonomy and reliability are not interchangeable. This page separates the concepts and explains why safety measures must grow with a system’s capabilities and access.

Primary sourcesLive benchmark linksWork-centred analysis
Power without controlArt · risk · anticipation
Thomas Cole's Destruction: a monumental city invaded, burning and overwhelmed by chaos
Thomas Cole
Destruction · 1836

In the fourth painting of The Course of Empire, a city that appeared all-powerful collapses into chaos. The work is not used here as a catastrophic prediction about AI, but as a reminder: when technical power advances faster than our ability to control it, failures can be amplified rather than contained. AI safety is about identifying those weaknesses, limiting their consequences and preserving the ability to intervene before a crisis. View the work ↗

01

AGI has no universal definition.

Different frameworks propose different thresholds; there is no single recognised test that can declare a system to be AGI.

02

Scaling laws are observations.

They describe measured trends as data, model size and compute increase. They do not guarantee general intelligence.

03

A high score remains local.

Each benchmark measures particular tasks under particular conditions. It does not summarise reliability, usefulness or safety.

04

Control must precede autonomy.

The more a system can act, the more explicit its access limits, approvals, monitoring, shutdown and accountability must be.

01 / Define without oversimplifying

What do people mean by artificial general intelligence?

AGI is a research and forecasting concept, not a settled technical category. Definitions differ on the human reference level, the range of tasks and the degree of autonomy required.

An economic definition

Outperform humans at most economically valuable work.

OpenAI’s Charter defines AGI as highly autonomous systems that outperform humans at most economically valuable work. This framing emphasises broad usefulness and autonomy.

Read the OpenAI Charter ↗
A capability-level framework

Separate breadth, performance and autonomy.

Google DeepMind researchers propose levels based on the range of tasks a system can perform and its level of performance. Autonomy is considered separately because it changes the risk of deployment.

Read “Levels of AGI” ↗
What AGI does not automatically mean

A fluent conversation, an excellent mathematics score or success on one autonomous task does not establish general intelligence, consciousness or human-like understanding of the world.

02 / Scaling laws

Why more compute often improves model performance.

Scaling laws describe empirical relationships: on some training metrics, error decreases fairly regularly when model size, data and compute increase together.

Published experimental dataFigure 1 · 2020
Published charts showing test loss decreasing with compute, data and parameter count on logarithmic axes
How to read this figure. Each point represents a training experiment. Lower test loss means better next-token prediction. The fitted lines show regular improvement with compute, data and parameters, provided another factor does not become the bottleneck. Kaplan et al., Scaling Laws for Neural Language Models
01
Training computeMore operations generally allow larger models or more data, while increasing financial, material and energy costs.
02
Model size and dataCompute-optimal training research shows that increasing model size alone is inefficient: data must scale with it.
03
Data quality and post-trainingData selection, instruction tuning and human or automated feedback strongly shape the final behaviour.
04
Inference-time computeReasoning systems may spend more computation on a problem, try several approaches and use tools, at a higher cost and latency.

03 / Capabilities

What powerful general-purpose models can do — and what still needs checking.

There is no universally “best model”. Suitability depends on the task, allowed tools, cost, latency, error tolerance and the degree of human control required.

01 · Reason

Complex problems

Decompose questions, compare hypotheses, calculate and explain a process across many domains.

02 · Produce

Code and documents

Draft, modify and test code; summarise long records; prepare analyses and professional documents.

03 · Perceive

Several media

Depending on the model, analyse text, images, documents, audio or video within one task.

04 · Act

Tools and interfaces

Search, run code, call software or operate an interface when an application grants access.

05 · Continue

Longer tasks

Plan several steps, maintain context and correct some mistakes without becoming consistently reliable.

06 · Fail

Uneven reliability

Produce excellent work and then fail on a simple step, invent a source or pursue the wrong interpretation of a goal.

04 / Current benchmark families

Measure a specific capability, not a single score for intelligence.

The links below point to maintained project pages and dashboards, so that results can be read with their current protocols rather than frozen into a ranking that will quickly become obsolete.

05 / Why AI safety matters

The more capable a system becomes, the farther an error or misuse can travel.

AI safety covers work to prevent, detect and reduce harm from AI systems, from routine reliability failures to severe risks enabled by greater capability and autonomy.

01 · Misuse

Dual-use capabilities

Code, scientific assistance and automation can support legitimate work or make attacks easier. Access and high-risk use therefore need specific controls.

02 · Reliability

A convincing but wrong action

A system may misunderstand an instruction, fabricate information or use a tool incorrectly. Automation can propagate a small error before it is noticed.

03 · Autonomy

More steps, less direct oversight

When an agent plans and acts for longer, it becomes harder to anticipate each action and detect when the objective has been misread.

04 · Influence

Information and persuasion

Personalised content at scale can amplify fraud, manipulation and disinformation, while polished language may invite excessive trust.

05 · Dependence

Concentration and skill loss

Provider dependence, version changes, outages and gradual deskilling can create systemic organisational risk.

06 · Work

Safety includes occupational health

A technically safe model can still intensify work, reduce autonomy, shift responsibility or impose continuous monitoring. These effects belong in AI safety.

A simple principle

Safety is not about predicting the date of AGI. It is about checking, before each increase in capability or autonomy, that dangerous behaviour can be detected, consequences limited and decisions challenged.

06 / From power to control

Five layers of control before a system is allowed to act.

Model evaluations are necessary but insufficient. They must be combined with technical, organisational and human measures suited to the real context of use.

01 · Evaluate

Test what could go wrong

Capabilities, severe errors, misuse, robustness, cybersecurity and effects on work.

02 · Limit

Grant minimum access

Bound data, tools, budget, duration, number of actions and operational scope.

03 · Approve

Keep human review meaningful

An identified person approves sensitive, irreversible or consequential decisions.

04 · Monitor

Make incidents visible

Logs, alerts, worker feedback, version tracking and an accessible reporting channel.

05 · Correct

Be able to stop and learn

A shutdown process, fallback route, incident analysis and reassessment after major changes.

General risk management

NIST AI Risk Management Framework

A voluntary framework for governing, mapping, measuring and managing AI risks throughout the system lifecycle.

Open the framework ↗
Workplace deployment

Assess what changes before and after.

Use the short questionnaire to structure a discussion about psychosocial risks, opacity, oversight burden and skill retention before a pilot.

Open the short questionnaire →

Independent newsletter

Follow the frontier without losing sight of real work.

Step back from model announcements, understand evaluations and connect technical progress with its effects on occupations and health.

Discover the newsletter ↗