Long-Running Tools and Asynchronous Results — Tool Use, Function Calling & MCP

A tool-calling loop is built around a simple contract: the model emits a `tool_use` block, your code runs the tool, and a `tool_result` comes back before the model's next turn. That contract works perfectly when the tool answers in milliseconds — a lookup, a calculation, a search. It breaks down the moment the tool represents real work: compiling a large codebase, rendering a video, running a batch data job, fine-tuning a model. Those take minutes or hours, not milliseconds, and the loop has no native concept of 'come back later.'

If you force a long job to run synchronously inside a single tool call, you pay for it twice. First in money: the conversation sits open, tokens accrue, and infrastructure (yours or the API's) holds a connection idle for the full duration of the job. Second in reliability: most transports enforce a request timeout well under an hour, often well under a minute, so a job that takes longer than that ceiling never gets to report its result at all — the call simply dies mid-flight, and the model never learns whether the work even finished.

This lesson is about closing that gap deliberately: designing tools so a long job can be started, tracked, and resolved without ever asking the model — or the API connection carrying the conversation — to sit blocked for the duration. The pattern is old (it predates LLM tool calling by decades, under names like 'async job', 'ticket', or 'operation handle'), but applying it correctly to a model-driven loop has specific constraints that this lesson works through one at a time, using `claude-opus-5` in every example.

Analogy🏏Cricket

🏏 Think of it like cricket: when Jasprit Bumrah bowls the death overs at the Wankhede Stadium, the umpire's job runs on a strict rhythm — ball bowled, decision given, next ball ready within seconds. But when Rohit Sharma is given out caught behind and sends the decision upstairs, that rhythm breaks. The umpire cannot just raise a finger and move on; UltraEdge traces, Hawk-Eye projections, and snickometer replays all have to run in the third umpire's booth, and that takes real seconds, not none. If the on-field umpire tried to stand frozen mid-decision — refusing to let Bumrah mark his run-up or the fielders reset until the review resolved — the over stalls, the broadcast clock burns, and eventually the match referee has to force a resumption whether or not the review is done. That is the exact gap between a normal ball, fast and answered instantly, and a DRS review, slow and only survivable if the game keeps track of where it left off while the review runs in the background. An agent's tool-calling loop has the same two speeds: most tools answer like a legal delivery, but a few — a video render, a data pipeline, a deployment — behave like a DRS review, and forcing them to answer at over-by-over speed either burns the session waiting or times it out before the answer ever lands.