Kwa et al. (METR Autonomous Evals) (@metr_evals)
Model Evaluation & Threat Research (METR) team measuring long-horizon software engineering and ML research autonomy (arXiv:2503.14499).
METR Research • arXiv:2503.14499 ('Measuring AI Ability to Complete Long Tasks')
Immutable Forecast Track Record (0)
Authored & Cited Arguments (2)
Autonomous AI task-completion horizons have doubled every ~7 months for six years—crossing the 1-hour software engineering threshold in 2025 puts multi-day coding agents on track for 2026–2027.
In our empirical evaluation across 170+ software engineering, cybersecurity, and ML research tasks (arXiv:2503.14499), the 50%-reliability time horizon of frontier models grew exponentially from under 2 minutes in 2019 to ~50 minutes in early 2025. Even when consumer GUI agents stumble on messy web forms, verifiable coding harnesses scale log-linearly with RL compute.
Extrapolating METR's 7-month horizon doubling time (arXiv:2503.14499) to 2028 reaches week-long autonomous ML engineering tasks—automating >80% of frontier AI R&D.
At a 7-month doubling rate, autonomous task horizons expand from ~1 hour in 2025 to ~32–64 hours (full work-week engineering sprints) by 2028. At that threshold, AI research agents autonomously design, execute, debug, and analyze complete ML architecture experiments.