Agents' Last Exam (ALE)
UC Berkeley RDI benchmark evaluating computer-use agents on 1,500+ expert-sourced, real professional-project tasks across 55 subdomains and 13 industries, with deterministic outcome grading and a brutal 'Last-Exam' tier where frontier agents score ~0%.
Overview
Agents' Last Exam (ALE) is a large-scale benchmark from UC Berkeley's RDI (Center for Responsible, Decentralized Intelligence), led by Dawn Song's team, designed to test whether 'generalist computer-use agents' are actually job-ready rather than merely benchmark-strong. Tasks are sourced from real completed professional projects with the help of 250-300+ industry experts across 100+ institutions, mapped onto the O*NET/SOC 2018 federal occupational taxonomy, and organized into 55 non-physical professional subdomains grouped into 13 industry clusters.
Unlike benchmarks that rely on subjective human or LLM-judge scoring, ALE uses deterministic, code-based evaluators that check agent outputs against verifiable reference artifacts inside full GUI+CLI computer-use environments, following a five-gate pipeline (expert sourcing → submission → peer review → implementation → QC). Tasks are split into three difficulty tiers — Near-Term, Full-Spectrum, and Last-Exam — explicitly designed to separate 'currently automatable' work from long-horizon professional work still out of reach.
Headline results from the June 2026 release show a steep cliff: the best configuration (Codex harness with GPT-5.5) reaches 38.1% on Near-Term tasks but only 22.7% on Full-Spectrum and 0.0% on the Last-Exam tier (24.0% overall); Claude Code with Claude Fable 5 scores similarly (34.3% / 20.9% / 0.0% / 22.0% overall). The paper also highlights that Codex/GPT-5.5 scores 82% on Terminal-Bench yet collapses to near-zero on ALE's hardest professional tasks, and that per-task cost varies 4–12x across similarly-performing agents (Claude Fable 5 ≈ $15.70/task vs. Composer 2.5 ≈ $1.33/task).
How It Works
Task setup, inputs, outputs, and evaluation.
Example Tasks
Real tasks from this evaluation system.
Cross-industry professional workflow task
Domain-specific professional work (one of 55 O*NET/SOC-aligned subfields)
Agent is given a real-world professional task drawn from a completed project in a specific occupation and must produce a verifiable deliverable inside a full computer-use environment.
What the Agent Receives
What It Must Produce
How Success Is Judged
Results
Model performance on this evaluation.
| # | Model | Score |
|---|---|---|
| 1 | Gpt 5 5 | 24.0% |
| 2 | Claude Fable 5 | 22.0% |
Key Findings
On the hardest 'Last-Exam' tier, frontier agent configurations (GPT-5.5, Claude Fable 5, Composer 2.5) score at or near 0% pass rate, versus 34–40% on the Near-Term tier.
Best overall configuration (Codex + GPT-5.5) scores 82% on Terminal-Bench but only 24% overall on ALE, and 0% on Last-Exam — a stark benchmark-to-real-work gap.
Per-task cost varies 4–12x across agents with similar accuracy (Claude Fable 5 ≈$15.70/task vs. Composer 2.5 ≈$1.33/task).
A dominant failure mode is agents declaring success before verifying their own work, producing outputs with missing files or constraint violations.
What Makes It Unique
- ✓
Built with 250-300+ industry experts across 100+ institutions, sourcing tasks from real completed professional projects rather than synthetic or crowdsourced scenarios.
- ✓
Uses deterministic, code-based grading against verifiable reference outcomes instead of subjective human or LLM-judge scoring.
- ✓
Explicit tiered structure (Near-Term / Full-Spectrum / Last-Exam) designed to track 'job-readiness' over time as a living, continuously expanding benchmark.
Controversies & Caveats
Extremely low pass rates on the hardest tier (0%) may partly reflect harness/tooling limitations rather than pure model capability, since headline numbers conflate harness and backbone model choices.
As a very new (June 2026) benchmark, independent replication and long-term leaderboard stability are still limited.