Benchmarking AGI: what to measure when the target keeps moving

Why evaluating general capability is structurally harder

Benchmarking narrow AI is comparatively easy. You have a task, a dataset, a metric, and a held-out split. Benchmarking generality breaks all four. The task is anything, the dataset cannot be enumerated, the metric has to survive across wildly different domains, and there is no held-out split because the internet is the training set and the internet contains your benchmark.

Chollet’s formulation of the problem remains the clearest: if intelligence is skill-acquisition efficiency rather than skill, then any test that measures demonstrated skill is measuring the wrong variable — and can be beaten by memorising enough skills.^[1] That is not a hypothetical failure mode. It is the default outcome for every popular benchmark within about eighteen months of publication.

There is also a quieter problem: the harness is part of the score. The same weights, given different scaffolding, tools, retry policies and prompts, produce materially different results on agentic suites. When two labs publish numbers on the same benchmark, they are frequently not measuring the same system. Reading those two numbers as a ranking is the most common evaluation mistake in the industry, and it survives because it produces such satisfying headlines.

• Contamination: famous benchmarks leak into training corpora; scores drift upward without capability changing.^[11]
• Saturation: once a test is near ceiling it discriminates nothing, but keeps being quoted.
• Harness variance: scaffolding, tools and retries can move agentic scores by many points.
• Construct drift: the test starts measuring the ability to do well on the test.
• Selective reporting: everyone publishes the suites they win.

The four families that matter

Rather than a leaderboard, which is stale within weeks, the useful mental model is a taxonomy. Each family answers a different question, and a claim about general capability needs evidence from more than one of them.

Family one: novel-task generalisation. ARC-AGI is the canonical example — grid puzzles designed so that each task requires inferring a rule from a couple of examples, with tasks authored to resist memorisation. ARC-AGI-2 tightened this further and deliberately targets the gap where tasks remain easy for humans while defeating systems that generalise poorly.^[2] These suites are the closest thing to a direct probe of the general column of the capability grid.^[14]

Two older traditions sit alongside these. Holistic evaluation frameworks score models across many scenarios and metrics at once — accuracy, calibration, robustness, fairness, efficiency — which is a better reporting format than a single headline number even when the underlying tasks are ordinary.^[12] And real-work evaluations grade model output against deliverables produced by experienced professionals in actual occupations, which is expensive, slow, small-sample, and the closest thing anyone has to asking whether the work would have been worth paying for.^[13]

Family two: hard knowledge and reasoning. Humanity’s Last Exam assembles expert-authored questions across many disciplines, explicitly built to resist the saturation that overtook earlier exam-style benchmarks.^[3] These measure depth and breadth of knowledge under reasoning load. They do not measure autonomy, tool use, or whether a system can finish anything.

Family three: agentic and real-work benchmarks. This is where the interesting action has moved. GAIA poses questions that are conceptually simple for humans but require multi-step tool use, browsing and reasoning.^[4] SWE-bench and its human-validated subset test whether a system can resolve real GitHub issues in real repositories.^[5][6] OSWorld measures open-ended computer use in a real desktop environment.^[8] tau-bench measures whether an agent can hold a policy-constrained conversation with a simulated user and complete a transaction correctly.^[9] MLE-bench tests machine-learning engineering end to end.^[10]

Family four: horizon and autonomy. METR’s time-horizon work reframes evaluation entirely: rather than asking how accurate a model is, ask how long a task — measured by how long a skilled human takes — the model can complete at a given success rate.^[7] This is the family most predictive of economic usefulness, and the least gameable, because failure compounds over long tasks in a way it cannot over single questions.

The contamination problem, and how to catch it

The cleanest demonstration of benchmark contamination is also the funniest. Researchers rebuilt a grade-school maths benchmark from scratch, matching the original’s difficulty and format but with entirely new problems, and found that several model families dropped noticeably on the fresh set while others held their scores.^[11] The drop is not proof of cheating; it is proof that the original number was measuring something other than the ability people assumed.

That experiment is the template for the most valuable evaluation habit you can adopt: hold out a private set, and re-author it periodically. If a system’s score on your private set moves in lockstep with its score on public suites, the public numbers are informative for you. If the gap widens with each release, the public numbers have decoupled from reality and you should stop quoting them in planning documents.

Human validation matters for the same reason. SWE-bench Verified exists because a chunk of the original set turned out to be unsolvable or ambiguously specified, meaning models were being penalised for tasks no human could complete from the given information.^[6] A benchmark with a broken ceiling makes every comparison against it slightly wrong in an unpredictable direction.

Why this guide prints no leaderboard

You will notice an absence of numbers so far. That is deliberate, and it is the most useful thing in this article. Any table of current scores would be wrong within a couple of model releases, and worse, it would be quoted long after it stopped being true — screenshots of benchmark tables have a half-life measured in years while their accuracy has a half-life measured in weeks.

What is stable is the shape of results, and the shape is worth stating plainly. Exam-style benchmarks saturate quickly. Novel-task suites built to resist memorisation retain a substantial human-machine gap for far longer. Agentic benchmarks sit in between and are extremely sensitive to scaffolding. And horizon measurements have been rising steadily rather than jumping, which is the pattern you would expect from incremental capability plus better harnesses rather than from a discontinuity.^[7]

If you need current numbers, get them from the benchmark maintainers rather than from a vendor’s launch table, check the grading mode, and check the date. A partial-credit score and a strict-completion score on the same suite can differ by tens of points, and stacking them in one column produces a comparison neither lab published.

Building an internal evaluation that outlives the next release

Public benchmarks answer is this model any good in general. They cannot answer is this model good at my work, and the second question is the one that decides your budget. A serviceable internal eval is smaller than people expect — thirty to fifty tasks is usually enough to separate real differences from noise, provided the tasks are real.

Start from failures, not successes. Collect the last twenty things that went wrong in your workflow, convert each into a task with an unambiguous definition of done, and record what a competent human takes to do it. Then run candidates unsupervised and grade completion, not vibes. The output you want is a table of unsupervised completion rate, median cost, median latency, and cleanup cost per failure — that last column is the one that decides how much autonomy is rational.

Keep half the set private and never paste it into a chat window, an issue tracker, or a vendor support ticket. This is not paranoia about training data; it is the same discipline as not publishing your exam questions. And re-author a portion of it every couple of quarters, because your own set contaminates too, mostly through your team optimising prompts against it.

What an AGI-relevant evaluation would need that today’s do not have

If you wanted to design an evaluation that genuinely tracked progress toward general intelligence rather than toward benchmark performance, the requirements are reasonably well understood even though nothing satisfies all of them at once.

It would need tasks authored after the model’s training cutoff, in domains the model has no special preparation for, with the environment unfamiliar and the tooling non-standard. It would need long horizons, because compounding failure is where generality is revealed. It would need to reward calibrated abstention, because a system that cannot recognise its own incompetence is not general regardless of its peak scores. And it would need to be re-authored continuously, which makes it expensive and unglamorous — the two properties that guarantee it will always be underfunded relative to the leaderboard.

The capability-grid framing gives the reporting format such an evaluation should use: a profile across domains and levels rather than a single number, plus an explicit separate statement of how much autonomy the system was granted.^[14] A single-number AGI score would be a category error even if we could compute one.

• Post-cutoff authorship: tasks written after training data was collected.
• Unfamiliar environments: non-standard tools, so scaffolding cannot be pre-optimised.
• Long horizons: hours to days, where errors compound.^[7]
• Calibrated abstention: credit for correctly declining, penalty for confident nonsense.
• Profile reporting: a grid of level by domain, not one score.^[14]
• Stated autonomy level: what the system was allowed to do, separately from what it could do.

Sources and evidence

Primary and authoritative sources used for factual claims. Company research and executive forecasts are labeled as such in the article.

1. 1
   On the Measure of IntelligenceFrançois Chollet · 2019-11
2. 2
   ARC-AGI-2ARC Prize Foundation · 2025-03
3. 3
   Humanity's Last ExamCenter for AI Safety and Scale AI · 2025-01
4. 4
   GAIA: A Benchmark for General AI AssistantsMeta AI, HuggingFace and collaborators · 2023-11
5. 5
   SWE-bench: Can Language Models Resolve Real-World GitHub Issues?Princeton University · 2023-10
6. 6
   Introducing SWE-bench VerifiedOpenAI · 2024-08
7. 7
   Measuring AI Ability to Complete Long TasksMETR · 2025-03
8. 8
   OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer EnvironmentsUniversity of Hong Kong and collaborators · 2024-04
9. 9
   tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World DomainsSierra AI · 2024-06
10. 10
    MLE-bench: Evaluating Machine Learning Agents on Machine Learning EngineeringOpenAI · 2024-10
11. 11
    A Careful Examination of Large Language Model Performance on Grade School ArithmeticScale AI · 2024-05
12. 12
    Holistic Evaluation of Language Models (HELM)Stanford Center for Research on Foundation Models · 2022
13. 13
    GDPvalOpenAI · 2025-09
14. 14
    Levels of AGI: Operationalizing Progress on the Path to AGIGoogle DeepMind · 2023-11