Agents That Run for Hours: Checkpoints, Progress Files and Budgets

An agent that answers in thirty seconds can keep everything in one context window and one process. A three-hour session has more opportunities to exceed context or process limits. Its context fills and gets compacted or reset. The process gets redeployed, or the container is rescheduled. And over hundreds of steps, small errors compound until the agent is confidently doing the wrong thing.

Long-running agents work when the harness assumes all three will happen. The model matters, but the harness decides whether a long run is recoverable. Here are the four pieces that matter most.

1. Durable execution, so a crash loses one step

If the process dies at step 140, restarting from step 1 is expensive, and it may repeat side effects. The loop needs to record progress somewhere that outlives the process.

Pick one of these:

• A durable execution engine: Temporal, Step Functions or Lambda durable functions. Model calls and tool calls become recorded activities. After a crash, the engine replays the recorded results and continues from the first incomplete step.
• A managed agent runtime such as Bedrock AgentCore Runtime, whose isolated sessions currently last up to eight hours. Session lifetime is not durable recovery: persist task and operation state externally for restarts.
• Your own checkpointing: after every step, write the conversation state and a step counter to a database, keyed by session. On start, load and resume.

Whichever you choose, the rule from every durable-execution system applies: side effects must be idempotent, because at some point one will run twice.

2. A progress file the agent maintains itself

Durable execution saves the process state. It doesn’t save the agent’s understanding. When the context is compacted or a fresh session picks up the work, the new context needs to know what’s done, what’s in progress and what’s been learned.

The pattern Anthropic describes in Effective harnesses for long-running agents is plain and effective. The agent keeps its own notes in files:

• A task list in a structured format (JSON works well because the agent is less inclined to rewrite it wholesale), with each item marked pending, done or failed, plus how it was verified.
• A progress log: short, dated entries saying what was done, what was found, and what to try next.
• Version control as memory. For code, committing after each completed item gives both a record and a rollback point.

The harness’s first instruction to every new session is to read those files before doing anything else. A fresh context that starts by reading “items 1–14 done and verified; 15 failed because the test fixture is stale; next: fix the fixture” gets back on course in one step, instead of re-exploring for twenty.

{
  "goal": "Migrate billing service from API v1 to v2",
  "items": [
    {"id": 14, "task": "Port refund endpoint", "status": "done",
     "verified_by": "tests/refund_v2_test.py passes", "commit": "a1c9e3f"},
    {"id": 15, "task": "Port invoice listing", "status": "failed",
     "note": "fixture invoices.json uses v1 schema; regenerate first"},
    {"id": 16, "task": "Port webhook handler", "status": "pending"}
  ]
}

3. Small increments with a check after each

The most common way long runs go wrong isn’t a crash. The agent does a lot of work, declares success, and is wrong. Across a long run, “I think this is done” compounds badly.

Structure the work so every increment ends with a check the agent can’t talk its way past:

• one item at a time, never “the whole migration”;
• each item has a verification defined before the work starts: a test that must pass, a command whose output must match, a page that must render;
• the harness, not the model, decides whether the check passed, by running the command and reading the exit code.

Measure whether these checks catch incomplete work before relying on the progress file.

4. Budgets enforced from outside the loop

A long-running agent has the time to run up a large bill or take a lot of actions. Limits need to live in the harness, where the model can’t argue with them:

• total tokens and total spend per run, with an alert at 50% and a hard stop at 100%;
• wall-clock time, which also bounds how stale the agent’s view of the world can become;
• steps without progress: if N consecutive steps complete no task-list item, stop and escalate. This catches loops sooner than a raw step limit does;
• action limits for side-effecting tools, per run and in aggregate.

When a budget runs out, the run should stop cleanly: write a final progress entry, leave the task list accurate, and notify someone. A run that ends with an accurate record of where it stopped can be resumed. One that ends mid-thought usually can’t.

Where this stops

These pieces make long runs survivable and resumable. They don’t make the agent’s decisions good. A well-checkpointed agent can still spend three hours doing the wrong task very reliably. The per-item checks are your defence against that, so spend most of your design effort on what counts as verified.