ThinkFacility Sign in
  1. Home
  2. News
  3. Research

METR finds AI time horizons doubling every 3.6 months, but agents still judge worse than experts

Paid specialists mostly couldn't beat the models on contest-style tasks, but agents still overrate work from models as strong as themselves.

On November 20, 2026, METR, the outside group that measures how long a task AI agents can finish on their own, published its quarterly capability update. The headline is that the skills that matter for AI research are improving fast and getting harder to measure.

METR says the length of task frontier models can complete half the time is now doubling roughly every 3.6 months. That's on its old suite plus a new, harder one built because models had used the old one up.

Published by
METR, November 20, 2026
Doubling time
roughly every 3.6 months
Longest time horizon
up to 27 hours, verifiable subset
Human specialists paid
$200–350/hr

Why METR had to build new tasks

A time horizon is METR's yardstick. In March 2025 the best model managed about an hour. Within a year, models had saturated Time Horizon 1.1, a suite with tasks up to 30 hours long.

So METR wrote a second suite, TH2, of 36 tasks in 12 families. Some can be scored automatically, like optimization contests and training a small language model fast. Others hinge on judgment, like investigating an incident or building a new evaluation for METR itself.

Current public frontier models have an estimated time horizon of up to 27 hours on the verifiable subset of our TH2 tasks.

From AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure

Finding people to compare against was the hard part. METR paid task specialists $200 to $350 an hour and still mostly couldn't find anyone who beat the models. On 15 long-running Al Zimmermann programming contests, the strongest models (Fable 5.1, Opus 5.5 and GPT-6 Astra) placed at a median of the 96th to 98th percentile of human entrants.

There are limits on the top end. On a contest to speed up training of a small GPT model, the best agent run gained 15.7% in 48 hours. One strong human record, made with AI help for coding, gained 41% in about 60 hours.

Where models still fall short

Judgment is the gap. When feedback is slow, or a test can't show whether a fix will generalize, METR found agents did worse than human experts. One finding matters for anyone planning to have AI check AI.

model judges overestimate an output’s quality when the output is made by a similarly strong or stronger model.

From AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure

The task that asked agents to build a new METR evaluation is the clearest case. METR calls the results borderline-useful, and says none were useful enough to include in the post. The models leaned toward automatically scored optimization tasks, which are the kind they're already good at.

METR also cautions about its own trend. The doubling time is shorter on the verifiable part of TH2 alone, which it says could fit a recent acceleration, though other explanations are plausible. A naive extrapolation reaches a 200-hour horizon, the longest threshold in its tasks, in July 2027.

What this means for oversight and for work

METR's reason for doing this is a specific worry. It defines drastic AI R&D acceleration as capability gains so quick that humans can't keep up with the risks.

We think that current public evidence is insufficient to rule out drastic AI R&D acceleration in the next couple of years.

AI R&D-relevant model capabilities are improving rapidly and are getting harder to measure - METR

California is already weighing rules on this. Governor Newsom has proposed capping self-improving AI. A cap like that needs a measurement behind it, and METR is saying the measurement is getting expensive and less certain at the same time.

For people who write code or tune systems for a living, the human comparison is the part to read. These are paid specialists, given AI help for implementation, and on contest-style tasks with clear scores they mostly lost. People held their edge where there's no fast answer key, and METR's sources don't say how much of anyone's job looks like that.

METR is careful about that, and so am I. It says its contained tasks may overstate how models do in messy real work, and that its human time estimates are mostly estimates. Of the human times behind its tasks, only 9% come straight from real baselines.

More on Research

All Research stories