Test-Time Compute
Test-time compute refers to additional computation a language model performs at inference time — beyond a single fixed-length forward pass — such as extended reasoning, self-verification, or exploring multiple candidate solutions, to…
Definition
Test-time compute refers to additional computation a language model performs at inference time — beyond a single fixed-length forward pass — such as extended reasoning, self-verification, or exploring multiple candidate solutions, to improve answer quality on harder problems.
Overview
For most of the history of deep learning, a model's capability was determined almost entirely by choices made before deployment: how large the model was and how much compute went into training it. Test-time compute introduces a second lever — how much computation a model is allowed to spend while actually answering a given query — and has become a major axis of progress alongside traditional pretraining scale, particularly for reasoning-heavy tasks like math, coding, and multi-step logic. In practice, test-time compute takes several forms: "extended thinking" or chain-of-thought reasoning, where a model generates intermediate reasoning tokens before its final answer, effectively using more forward passes per response; sampling multiple candidate answers and selecting the best one via a verifier or voting scheme; and iterative refinement, where a model critiques and revises its own draft answer. Models like OpenAI's o-series and Claude's extended thinking mode expose this tradeoff directly to users or API callers, letting them dial up reasoning effort — and cost and latency — for problems where accuracy matters more than speed. Research has shown that, for many reasoning tasks, spending more compute at inference time can improve accuracy in ways that are sometimes more cost-effective than simply training a larger model, reshaping how labs think about the capability/cost tradeoff: instead of always needing a bigger pretrained model, a smaller model given more "thinking" budget can match or exceed a larger model answering in one shot on certain problem types. This has made test-time compute one of the more actively studied levers in current frontier model development, alongside continued gains from pretraining scale and post-training techniques like reinforcement learning from human feedback.
Key Concepts
- Additional inference-time computation beyond a single fixed forward pass
- Includes extended chain-of-thought reasoning before a final answer
- Can involve sampling multiple candidates and selecting the best via a verifier
- Supports iterative self-critique and refinement of draft answers
- Exposed to users as adjustable "reasoning effort" in models like Claude and OpenAI's o-series
- Trades increased latency and cost for improved accuracy on hard problems
- Can sometimes be more cost-effective than training a larger base model
- An increasingly important axis of progress alongside pretraining scale
Use Cases
Frequently Asked Questions
From the Blog
Test-Taking Tips That Actually Improve Your Score
Strong test-taking strategy comes down to managing time, reading questions carefully, and reviewing systematically rather than cramming harder. This guide covers the habits that reliably raise scores on technical and certification exams.
Read More AI & TechnologyLLM Evaluation: Building a Test Suite for Generative Output
You can test non-deterministic output by fixing the inputs, grading against a rubric rather than an exact string, and gating releases on aggregate thresholds instead of per-case pass or fail. This guide shows how to assemble that suite: dataset, scorers, run harness, thresholds and the CI wiring that makes it enforceable.
Read More AI & Technology9 Agent Failure Modes to Test Before You Launch
Agents fail in a small number of recognisable ways, and nearly all of them can be provoked deliberately before a user finds them. This article names nine failure modes, from invented tool calls to stale memory and irreversible actions, and gives a concrete test for each that belongs in a pre-launch suite.
Read More AI & TechnologyHow to A/B Test an LLM Feature With Real Users
Judge model changes on behaviour, not on offline scores. Pick one primary behavioural metric, define guardrails that stop the experiment automatically, randomise at the unit users actually experience, and hold the test long enough for the slow signals — retention and follow-up rate — to arrive.
Read More