A real root-cause investigation is not one prompt. It’s the agent pulling logs, correlating a metric spike, reading a recent deploy, checking a dashboard, forming a hypothesis, testing it, again and again for hundreds of steps. We raised our loop ceiling from 10 iterations to 500, and to 1,000 for long-running work. Two problems arrive long before you hit that number: the conversation blows past any model’s context window, and the bill grows without bound. And somewhere in a multi-hour run, a pod will restart.
// agent-loop configuration
DEFAULT_AGENT_LOOP_MAX_ITERATIONS = 500 // was 10
LONG_RUNNING_AGENT_LOOP_MAX_ITERATIONS = 1000 // was 50
CONTEXT_BUDGET_WARNING_RATIO = 0.50 // "write down key IDs now"
CONTEXT_BUDGET_COMPRESSION_RATIO = 0.65 // compact the historyWhy a plain loop breaks
Start with the obvious design: a while loop that appends every tool call and result to a message array, then re-sends the whole array to the model each turn. It survives about a dozen iterations. After that, three things break at once. The context window overflows, and the model starts silently dropping findings it made an hour ago. Cost climbs superlinearly, because every step re-sends the entire history. And a single crash loses the whole investigation, or worse, replays it and opens a duplicate PR.
Managing context as a budget
At the top of each iteration, we compute one number: the last prompt’s token count divided by the model’s real context window. Two thresholds act on it. At 0.50, we inject a one-time nudge that tells the agent to write its key findings and IDs into its own reply right now, so they survive the next step and any later compaction. At 0.65, we compact. Everything after the system prompt is summarized by a small, fast model, and the working history is replaced with that summary plus the original request.
That summary becomes the agent’s memory. It is the most dangerous operation in the system, because everything the agent does next depends on what the summary kept and what it dropped. So we engineer the compression prompt to be faithful rather than pithy. It carries a MUST-PRESERVE list: exact IDs, counts, timestamps, file paths, error messages, decisions made, and dead-ends already tried. It forbids the model from inventing, rounding, or truncating. And it runs at temperature zero.
We also defend this step in depth. The compaction call gets a 90-second timeout and up to three attempts. If all three fail, we do not silently drop context. We append one instruction, “wrap up now, compression unavailable,” and stop retrying, so a single broken summary can’t burn every remaining iteration.
A second optimizer works at a finer grain, and it leans on something the agent already has: a sandbox with a real file system. Instead of keeping a large tool result in the context, we store it on disk and leave a [context optimized] reference in its place, which the agent fetches back on demand.
Surviving the restart
A run that lasts hours will outlive at least one pod. When one dies, we can’t start the investigation over, and we especially can’t redo a step that already touched the outside world. If the agent pushed a commit or opened a PR before the crash, running that step again would create a duplicate.
So the loop is built as a durable execution. The rule is that every step which acts on the outside world, or could come out differently a second time, is recorded the moment it finishes. When a pod dies, the loop restarts and replays that record: steps that already completed hand back their saved result instead of running again. The agent resumes exactly where it left off, without paying twice for a model call it already made or repeating an action it already took.
This is why the identity of an action has to be fixed the first time it runs, not made up again on resume. Picture a step that opens a PR and stamps it with a new ID. If that ID were generated again on replay, it wouldn’t match the one already recorded, and the recovered run would look like a second, different action, the exact duplicate we were trying to prevent. Recording an action’s identity when it first happens, and reusing it on replay, is what keeps a resumed run from drifting into a subtly different reality. The compaction summary follows the same rule: a resumed run reuses the summary it already had, instead of re-summarizing into something slightly different.
State isn’t the whole story, though. The agent itself has to know it’s continuing rather than starting over. So on resume we tell it directly: pick up where you left off, and don’t repeat analysis you’ve already done.
Knowing when to stop
The loop needs to know when the agent is finished. Our rule is simple: the agent is done when it stops asking to use tools. No tool calls means nothing left to do in the outside world, so the loop ends.
That rule has a failure mode. Models sometimes announce a tool call in plain prose (“I’ll now run the log search…”) instead of issuing it in the structured format the loop watches for. The loop sees no real tool call, decides the agent is finished, and cuts the run off in the middle of an investigation. So we catch it: when the text reads like an intended action, we ask the model once to use the proper format, and if it still won’t, we pull the tool call out of the prose and run it anyway.
The opposite problem is hitting the iteration ceiling. We don’t want a silent cutoff there either. On the final turn we disable tools and send a text-only prompt that tells the agent its tool calls won’t run anymore, so it has to report in plain text what it finished and what is still pending, like an unpushed commit or an unopened PR. Whether the agent stops on its own or hits the cap, it should never leave you guessing what it did and didn’t finish.
Build-it-yourself reality check
It’s tempting to treat context compaction and durable execution as plumbing. In our experience, each is closer to a research problem. Lossy summarization of an agent’s working memory degrades answers in ways your evals won’t catch for weeks. The run still finishes; it just quietly forgot the one ID that mattered.
Determinism on replay is worse, because it isn’t a feature you can bolt on. It’s a discipline that every side effect in the codebase has to follow, forever. One unguarded clock read, one random value generated the wrong way, one action that escapes the recovery path, and a crash restores the wrong state instead of failing loudly. Both problems are easy to underestimate, because the happy path looks identical whether you got them right or not. The difference only surfaces at hour three, under load, right after a deploy.
That is the real reason this is hard. A resumed agent must never redo work, double-charge, or open the same PR twice, and that single invariant reaches into every side effect in the system. It looks like a weekend project in a demo, and it becomes a permanent obligation in production. Worth knowing before you decide to build it yourself.
