Back to Explore
NewsVibeFix editorialUpdated Nov 20, 2026

OpenRouter Tested Six Models on Agent Infinite Loops: Four Guards Cut Tool Calls from 314 to 64

OpenRouter ran the same deliberately broken billing agent three times on each of six models: 18 runs, 314 tool calls and $0.29 unguarded, 64 calls and $0.05 with repeat-call detection. Four guards — doom-loop detection, loopKey normalization, step and cost caps, and a key credit limit — each stop a different kind of loop, and none of them is enough on its own.

OpenRouter's official test chart: cumulative cost of the unguarded Mistral Large 4 run, reaching $0.0309 after 41 turns, with the curve steepening as it goes.

The short version: loops are not the model being dumb. They are a loop with no brakes.

Anyone who runs agents has lived through this scene: you open the billing page in the morning and find that some agent burned hundreds or thousands of dollars overnight, while accomplishing exactly nothing — just calling the same failing tool over and over. On November 12, 2026, OpenRouter published a hands-on guide, "How to Stop an AI Agent Stuck in an Infinite Loop," that takes the problem apart and measures it: the same deliberately broken billing agent, run three times on each of six current models, for 18 runs in total. Without protection, the runs made 314 tool calls and spent $0.29. With repeat-call detection switched on, that fell to 64 calls and $0.05. Calls down by about 80%, spend down by more than 80%. An honest caveat up front: these numbers come from OpenRouter's own team, tested on its own SDK, with a sample of just 18 runs and a single deliberately broken tool. This article has not independently reproduced them. But the mechanism they reveal is generic — why loops happen, why they get more expensive as they run, and what each of the four guards actually stops. Those judgments do not depend on OpenRouter's product, and they apply to anyone who writes their own agent loop. So this piece treats OpenRouter as the source of the test data, and the method as everybody's.

Start with why loops happen, because most people's first instinct is to blame the model. It is not the model. An agent loop has exactly one exit condition: the model answers without asking for a tool. A tool that keeps failing gives the model no reason to do that. The error goes back into the conversation, the instructions still say to finish the task, and another attempt looks like the right next step. Nothing in the loop counts attempts unless you write code that does. Worse, every turn resends the whole conversation history, including every earlier failure, so the loop gets more expensive the longer it runs. In one run the team logged turn by turn, input grew from 131 tokens to 3,262, and the cost per turn nearly quadrupled. That is why a loop's bill does not grow linearly — it curves upward. The cover chart is the cumulative cost of the unguarded Mistral Large 4 run from the test: $0.0309 after 41 turns, and the slope keeps steepening. Thirty cents sounds harmless, but that is a cheap demo model, a single run, with a 40-step cap imposed by hand. Swap in a model several times pricier, run dozens of sessions at once, remove the cap, and a few thousand dollars overnight is entirely realistic. The real danger of a loop is not that a single call is expensive; it is that it keeps replicating cost while you sleep.

OpenRouter's official test chart: cumulative cost of the unguarded Mistral Large 4 run, reaching $0.0309 after 41 turns, with the curve steepening as it goes.

How the test was set up: a billing agent built to fail

To reproduce the loop reliably, the OpenRouter team built a minimal billing agent with a single tool, fetch_invoice, which fetches an invoice total by ID — and which was hard-coded to throw every time, simulating a billing service that is down: "billing-service timeout after 30s fetching INV-2291." The instructions were equally blunt: "You are a billing agent. Complete the task. If a tool call fails, retry it. Do not stop until you have the invoice total." The input was one sentence: "Get the total of invoice INV-2291." The setup looks extreme, but it is exactly how loops die in production: a downstream service goes down, the agent's instructions say "retry on failure," and the two together are a dead loop. The SDK under test was @openrouter/agent 0.11.0, with the loop running inside a single callModel function. The six models were Claude Haiku 5.5, DeepSeek V4.1 Flash, Gemini 3.8 Flash, Mistral Large 4, GPT-6 Luna, and Grok 4.7, each run three times — first with no protection except a 40-step cap as a backstop, then again with detection on. Without protection: Claude Haiku 5.5 made 6, 6, and 6 calls; DeepSeek V4.1 Flash made 6, 5, and 7; Gemini 3.8 Flash made 5, 6, and 5; Mistral Large 4 made 11, 41, and 41; GPT-6 Luna made 17, 37, and 21; Grok 4.7 made 41, 12, and 41. Four of the 18 runs hit the 40-step cap. With repeat-call detection on, all six models were blocked on their third identical tool call, and 16 of the 18 runs ended after two real tool calls. The only exceptions were one Mistral Large 4 run at 12 calls and one GPT-6 Luna run at 20. The most important number in that table is not the average but the variance: Grok 4.7 produced 41, 12, and 41 calls on the exact same input. One clean test proves nothing about whether your agent will loop, because model behavior is stochastic — the same prompt that behaves today may loop tomorrow. Protection cannot rest on "we tested it once and it was fine." It has to be a mechanism.

Then there is the famous demo run: Mistral Large 4, unguarded, 41 attempts, $0.0309, stopped only by the 40-step cap — the extra call is one the model had already asked for when the cap fired, which the SDK still runs. All 41 calls were identical: fetch_invoice("INV-2291"). And the final answer is telling: the model wrote, plainly, that the billing service had timed out on all 41 attempts. It knew, on every turn, that the service was down — and retried anyway. Hoping the model will "figure it out" is not a plan. It has no global view of how many times it has tried unless you put "you have already failed N times" into the feedback it can see, with consequences attached. That is why the first guard exists.

Guard 1: doom-loop detection — block the third identical call

The first guard is called doom-loop detection, and in the OpenRouter Agent SDK it is one option on the callModel call: doomLoop: true. The idea is simple: fingerprint every tool call by its name and its arguments, canonicalizing the arguments first so that reordered JSON keys do not hide a repeat. One subtlety matters: a repeat only counts when the model has seen the previous result and asks again, so five identical calls in one turn count as one. As the streak grows, detection climbs a default ladder: observe at a streak of 2, block at 3, and stop at 6. When a call is blocked, the SDK skips execution and returns a tool error telling the model it is repeating itself and that another try will not help. In most runs, the model read that error and wrote a final answer straight away, ending cleanly. In the demo run it kept asking, so the ladder reached "stop" at streak 6 and the run ended with reason doom_loop; getDoomLoopVerdict() returns a message only when detection ended the run, which makes it the signal to wire into alerting. With detection on, the demo run made two real calls and cost $0.0034 — against $0.0309 unguarded, a saving of nearly 90%.

The ladder thresholds are tunable. With doomLoop: { ladder: { observe: 2, block: 3, stop: 4 } }, a GPT-6 Luna run ended at streak 4 for $0.0002. The ladder also has two rungs that are off by default: steer sends the model a note that it is repeating itself, and escalate runs the next turn on a stronger model of your choice. These two deserve a word of caution. Steer is the right instinct — often the model is not being stubborn, it just has not noticed it is going in circles, and a nudge is enough to make it change approach. Escalate needs more care: upgrading the model is itself spending more money, and using it to rescue an already out-of-control loop can easily mean paying more to hit the same wall. My judgment: the default ladder is enough for most cases — get observe, block, and stop working first, then consider steer; reserve escalate for tasks where a stronger model genuinely could succeed, not for giving a dead loop more life. One more limitation to keep in mind: repeat detection only recognizes repeats. A model that tries something new every turn never repeats a fingerprint, and detection never fires. That is why the first guard alone is not enough, and why hard caps come next.

Guard 2: tell the detector what counts as "the same call"

The first guard has a very practical hole: once blocked, the model changes the argument slightly and tries again. The test log reads almost like a comedy routine: fetch_invoice("INV-2291") gets blocked, so the model tries fetch_invoice("2291"), then fetch_invoice("inv-2291"), then fetch_invoice("INV2291"). A new fingerprint means a new streak. In nine extra Mistral Large 4 runs with detection on, the model pulled this trick five times, and three of those reached the 40-step cap, mostly by alternating INV-2291 and inv-2291. GPT-6 Luna did the same thing with trailing spaces and newlines. The model is not being devious; it is following the block message, which tells it to take a different approach — and with only one tool, the only different approach available is a different input. So the detector's default rule, that two calls count as the same only when their arguments match exactly, is too naive. The developer has to define what "the same call" means.

The fix is a loopKey on the tool: a function that extracts the part of the input that actually identifies the call. For an invoice ID, that is the digits: invoice_id.replace(/\D/g, '') maps "inv-2291 ", "INV-2291", and "INV2291" all to 2291, so the streak keeps growing and the repeats stay blocked. With the loopKey in place, five of six runs stopped after two real calls. In the sixth, the model tried a different invoice, INV-0000, which correctly reset the streak — proof that normalization does not over-block, because a genuinely different call still runs. The rule for writing a loopKey: normalize case, spacing, prefixes, and anything else that does not change which record the tool fetches. To ignore a field entirely — say a timeout the model raises on each retry — list only the fields that matter, as in loopKey: ['invoice_id']. And a tool whose job is to repeat, like one that polls a job's status, takes loopKey: false. The general lesson is that the definition of "repeat" belongs to the person who wrote the tool, not to the model. You know best which parameter change makes a request genuinely different; encoding that judgment beats reading logs after the fact. One detail from the test is worth remembering: even with guards one and two, the argument-changing Mistral run still reached 13 calls and $0.0125 before ending. Repeat detection, however clever, only covers the "repeat" class of loops. The backstop has to be the next guard.

Guard 3: hard caps on steps and cost for every run

The third guard is stopWhen: hard caps on every run, independent of detection, that end the run no matter why it ran long. The test configuration combined three conditions: no more than 15 steps, no more than $0.01, plus a custom predicate that stops after five consecutive tool failures. The failure counter is maintained with lifecycle hooks — increment on PostToolUseFailure, reset on PostToolUse. It ignores arguments, so the argument-changing tricks from guard 2 do not fool it either. On the Mistral Large 4 run that kept changing its arguments, the caps ended the session after six calls for $0.0062. How to pick the numbers: in the test, normal runs took four to six turns and cost $0.0018 to $0.0053 on Mistral Large 4, so 15 steps and $0.01 leave roughly two to three times headroom — enough not to cut normal work short, tight enough not to allow a 40-turn run. Of the two caps, the cost cap matters more, because later turns cost more than early ones: a step cap limits turns, a cost cap limits money.

Two implementation details are worth noting. First, stopWhen is checked after every round, and when a cap fires while the model still has pending tool calls, the SDK runs them and makes one last turn with tool calling disabled, so the run ends with a written answer. That is why the 40-step cap produced 41 calls and the five-failure cap produced six. If you do not want that final turn, set allowFinalResponse: false. Second, every stopWhen condition ends the session with the same reason, max_turns, so log your own reason if you need to tell the caps apart — otherwise you will be staring at a max_turns line during an incident with no idea which limit fired. Finally, none of this depends on OpenRouter's SDK. The same guards fit in any loop you write yourself: count calls per tool name and normalized arguments, return a tool result like "Blocked: tell the user what went wrong" instead of running the third repeat, and stop when steps or summed usage.cost pass your cap. The test even includes a Python version for comparison — a version against the chat completions API, no SDK at all, that stopped Mistral Large 4 after four steps for under $0.002 in all three runs. The method is generic; the SDK just packages it.

Guard 4: a credit limit on the API key — the server-side wall

The first three guards all run inside your own agent process. If that code has a bug, ships without the caps, or runs a hundred sessions that each stay under their own maxCost, nothing limits the total. The fourth guard is a credit limit on the key, enforced server-side before the request reaches a provider, so it holds even when your agent code has a bug. Set a dollar limit and a reset window — daily, weekly, monthly, or none — in the dashboard or with the create-key API. If the key uses bring-your-own-keys, also set include_byok_in_limit to true, because it defaults to false and BYOK spend does not count toward the limit otherwise — an easy trap, where you set a limit and then discover BYOK billing was never counted. Daily limits reset at midnight UTC, weekly limits on Monday. Give each agent or customer its own key, so one loop can only spend that key's share. The test demonstrates a 402: a chat completions request with max_tokens set to 900,000, sent on a key with $9.97 left, was rejected before routing, because the server prices the worst case — input plus max_tokens — and refuses if it does not fit. The error carries limit_source: "openrouter_key_limit", and nothing was charged.

A few operational points. First, branch on error.metadata.limit_source, not on the message text or the status code; an exhausted key reports the same source, and the limits documentation lists it under both 402 credit errors and 403 budget errors. If the message says the key can afford fewer tokens, retry with a lower max_tokens; if the key is exhausted, retrying cannot succeed until the limit resets, so treat it as a stop. Second, the one 402 worth retrying is openrouter_in_flight_budget, after its Retry-After header. Third, rate limits return 429 instead — a different problem, do not conflate the two. Fourth, workspace budgets pool the cap across every key in a workspace, per day, week, month, or lifetime; blocked requests get a 403 with a message like "Workspace monthly budget of $500.00 exceeded." One honest note: the team did not trigger that 403 itself, so the message comes from the documentation. Requests already in flight still finish, so spend can go slightly over the cap, but a loop that runs all night stops close to the daily limit. My judgment: the fourth guard is not a nice-to-have, it is mandatory. The first three decide whether a single run can spiral; the fourth decides whether the whole system can. Per-agent, per-customer keys with server-enforced limits are the cheapest form of cost isolation there is.

Which guard stops which loop: do not expect one layer to do everything

Put the four layers side by side and each one catches a different kind of loop. The same call with the same arguments: doom-loop detection stops it, and the caps and the key limit stop it too. The same call with slightly different arguments: detection misses it, loopKey stops it, and the caps and key limit still catch it. A new approach every turn: both detection layers miss it, only the caps and the key limit stop it. Many runs, each under its own cap: the first three guards are per-run only, so only the key limit or a workspace budget stops the total. A deploy with the guards missing entirely: again, only the key limit stops it. That mapping is the most valuable part of the whole test — it turns "prevent loops" from a vague feeling into a clear division of labor. Most teams that get burned did not fail to think about protection; they protected against one kind of loop only. A step cap alone feels safe until dozens of sessions each stay under the cap and the bill explodes anyway. Repeat detection alone feels safe until the model changes one character in the argument and walks around it. Every layer has a blind spot, and only the four together cover each other's.

How much can we trust these numbers? Three cold showers

The numbers are striking, and striking numbers are exactly the ones that need a cool head. First, the sample is tiny: 18 runs, three per model per condition, and Grok 4.7 produced 41, 12, and 41 calls on identical input. With that kind of variance, no average precise to two decimal places deserves to be taken literally. Calls from 314 to 64, spend from $0.29 to $0.05 — the correct reading is "protection cut calls and spend by roughly an order of magnitude," not "your bill will fall by exactly 83%." Second, the scenario is a single one: one tool, one failure mode, a tool that times out every time. If your agent has ten tools, five failure modes, and some tools that fail intermittently, detection will be less effective and tuning will matter more. Third, this is a vendor testing its own product on its own SDK. The scenario, the thresholds, even the six-model lineup may have been chosen to flatter. Three of the six are cheap small models — Haiku 5.5, DeepSeek V4.1 Flash, Gemini 3.8 Flash — which barely looped at all without protection, five to seven calls each and then stopped on their own. The models that really loop, Mistral Large 4 and Grok 4.7, were the minority. The value of the test is not the headline numbers but the failure modes it shows in the open: how models evade detection, what the evasion looks like, and which layer catches it. Those details are more useful than 314-to-64.

One more warning: do not copy the thresholds. Fifteen steps, $0.01, five consecutive failures — those numbers were calibrated for a four-to-six-turn billing task costing a few thousandths of a dollar per run. If your normal task takes twenty turns and costs several cents a run, copying those caps is sentencing normal work to death. The right approach is to observe first: run without limits for a while, record the distribution of turns and costs for normal tasks, then set caps at two to three times the normal values. The cost cap especially must be priced against your own models: the test used cheap models, and the same 15 steps on a flagship model could cost dozens of times more. The same logic applies to the ladder: observe at 2, block at 3, stop at 6 is a reasonable starting point, but if your tool calls legitimately repeat — a retry that might succeed on the second attempt — the block threshold needs loosening, or retries need an explicit carve-out. Protection is a trade-off: too loose is the same as nothing, too tight kills normal work. There is no universal set of numbers, only the process of observe, set, and re-check.

A vendor-neutral checklist you can start on Monday

Strip out the OpenRouter SDK names and the method survives intact on any agent framework. LangChain's AgentExecutor has max_iterations, CrewAI has max_iter — the same job as stepCountIs. Cost caps and failure counters are a few lines of code in any framework. Key limits exist under different names at every provider — budgets, caps, guardrails — but the mechanism is the same. Here is the checklist, with no product attached: first, fingerprint every tool call by tool name plus normalized key arguments, and on the third repeat, do not execute it — return a clear error telling the model that retrying will not help and to tell the user what went wrong. Second, let the person who wrote the tool define the normalization: strip case, spacing, prefixes, and timeouts that do not change the result before fingerprinting, and explicitly exempt polling tools. Third, cap every run by both steps and cost, set the cost cap at two to three times your measured normal values, and log which cap fires each time. Fourth, count consecutive failures separately, regardless of arguments, and stop at five — and write tool error messages that tell the model what to do next, not just "it failed, please retry." Those three words, "please retry," are in some sense the origin of the loop. Fifth, give every agent and every customer its own key and limit, enforced server-side, so a deploy that forgets the guards cannot burn the whole budget. Sixth, re-check regularly: look at the block logs for normal tasks caught by mistake, and ask whether the thresholds still fit.

One final judgment. When people worry about agent costs, the first instinct is usually to switch to a cheaper model or squeeze the prompt. Both help, but they save on "what each call costs," while loop protection saves on "how many calls should never have happened." The orders of magnitude are different: prompt tuning saves tens of percent; stopping one overnight loop can save more than 90% of a bill. And loops burn more than money — time, quotas, and the availability of downstream services, since an agent retrying furiously can take down your database or a third-party API just as surely as it takes down your budget. The four layers are really an answer to one question: when the model gets stuck, who calls a halt? The detector halts repeats, normalization halts disguises, hard caps halt long runs, and the key limit halts the whole system. No layer knows whether it will be the last line of defense, so every layer has to be there. The morning your agent is stopped in the middle of the night by its own protection, and all you find is a clean block log instead of a shocking bill, you will understand: the cheapest layer of the four is the one that saved you the money the most expensive one would have spent.

Views 13Comments 0

Comments (0)

ME
0/1000
Loading comments...

Sources

Browse projectsPublish your project

Related articles

A developer with glasses types on a laptop at a wooden desk, an external monitor to the left showing C code on a dark background, a small plant on the desk
News
Microsoft's Coding Model Stops Playing Second Fiddle: MAI Code 2 Scores 96% of Opus 5.5 for 5% of the Cost — Read the Three Caveats First

On 17 November 2026, Microsoft launched MAI Code 2 and Code 2 Flash: the larger model has 77 billion active parameters and a one-million-token context, averaging 63.2% across four benchmarks — close to GPT-6.1 Sol and Claude Opus 5.5 — at $0.309 per task. Read the three caveats first: promotional pricing, the lowest score of the three, and no long-horizon work.

Developer WorkflowProduct LaunchModel Updates
Against a black background, a conductor seen from behind raises a baton, with swirling trails of light circling beneath him like an orchestra being conducted
News
One API for 45 Models, and It Switches Providers When One Fails: Perplexity's Orchestrator Wants to Be the Switchboard

On 17 November 2026, Perplexity launched the Orchestrator API: one interface reaching OpenAI, Anthropic, Google, xAI and open-weight models such as GLM, Kimi and DeepSeek, billed at the answering model's published rate with no platform fee. When one provider fails, the same model retries on another deployment, and preview routing assigns requests to models by task.

Developer WorkflowProduct LaunchModel Updates
A developer types at a dual-monitor workstation; the left screen shows a dark code editor, the right a testing dashboard with green checkmarks and bar charts
News
Did Your Agent Get Better or Worse? AWS Open-Sources UAEF to Measure Agents from Every Framework with One Ruler

On 11 November 2026, AWS open-sourced UAEF: adapters turn traces from LangGraph, LangChain, Strands and Bedrock AgentCore into one format, then 36 metrics across 7 dimensions score them, flagging regressions past 5%. The judge model is hardened too — scores must cite verbatim evidence, and never decide safety.

Developer WorkflowProduct LaunchTesting & Quality