Designing AI for Long-Running Tasks: Looking Beyond More Context

Model capability is moving in that direction quickly. In its February – March 2026 Frontier Risk Report, METR estimated how long a software task could take before a frontier model’s chance of completing it successfully fell to 50%. Claude Opus 4.6 reached about 17.8 hours on cleaner tasks but only 6.4 hours on messier ones involving changing environments, constrained resources, expensive feedback, or mistakes that could not easily be undone.

The difference suggests that task length alone is not the main challenge. Long-running work also requires the system to preserve progress, adapt when conditions change, and recover when earlier assumptions no longer hold.

Recent benchmarks show how difficult that remains. Microsoft’s WeaveBench evaluates realistic workflows spanning graphical interfaces, command-line tools, and code, yet its strongest evaluated model-runtime pairing reached only a 41.2% pass rate.

These failures are not simply a matter of running out of context.

Long-Horizon Is Not the Same as Long-Context

The August 2026 survey The Horizon Gap reviewed more than 1,500 papers and draws an important distinction between three concepts that are often conflated:

• Long-horizon – the task requires many dependent steps.
• Long-context – the model can process a large amount of information in a single invocation.
• Long-term memory – the system can preserve useful information across steps or sessions.

These capabilities solve different problems. A larger context window lets the model see more at once but does not ensure that it knows what has been completed, what remains, or which earlier decisions still matter. Long-term memory can preserve information across time, but remembering more does not guarantee that the right information will be available for the next decision.

For long-running work, the system therefore needs an explicit representation of task state — what has been completed, what remains, what assumptions are active, and what it is waiting on. That state can persist outside the model, while each model call receives only the information needed for the next step.

A useful way to think about a long-running AI task is as a sequence of model calls connected by persistent state. At each step, the system loads where the task stands, gives the model the information needed for the next decision, takes an action, and records what changed. Once that process stretches over minutes, hours, or days, several additional problems appear.

1. Make Progress Explicit

Consider an AI system given this task:

Plan and book a three-day Tokyo trip under $2,500, including flights, a hotel, activities and dinner reservations.

After several steps, the useful state is not a transcript containing every hotel considered and every restaurant rejected. It is something closer to:

Goal: Plan and book a three-day Tokyo trip under $2,500.

Completed:

• Flights booked
• Hotel reserved in Shinjuku
• TeamLab tickets purchased
• First two days planned

Remaining:

• Plan the final day
• Book Saturday dinner
• Arrange airport transportation

Constraints:

• Total budget - $2,500
• Vegetarian-friendly restaurants
• No activities before 10 AM

Current spend: $1,940

Known issue: The preferred Saturday restaurant is fully booked.

This gives the next model invocation a compact representation of the task rather than asking it to reconstruct the trip from its own history. The 2026 paper Remember When It Matters describes a related failure mode as behavioral state decay — requirements, previous attempts, and open goals can remain somewhere in a long trajectory but stop influencing the model when they matter.

The important idea is not simply to remember more. It is to maintain a representation of where the task currently stands.

2. Make Progress Recoverable

Now suppose the system books the hotel and then the process crashes. When execution resumes, it needs to know that the hotel has already been booked.

This is where checkpoints and durable execution enter the picture. A checkpoint might preserve the current task state, completed actions, booking IDs, intermediate artifacts, pending work, and the external state last observed by the system.

Production runtimes are already being built around this requirement. LangChain’s runtime for long-running agents uses durable execution and checkpointing so work can resume after interruptions rather than restarting from the beginning. Recovery also changes how actions have to be designed. If the system crashes immediately after making a reservation, retrying the previous step should not accidentally make a second reservation.

Long-running AI therefore inherits familiar distributed-systems concerns - retries, idempotency, resumability, and careful handling of side effects.

A useful question when designing state for any long-running step is: if execution stopped immediately after this action, could the system safely figure out what happened and continue?

3. Handle Waiting as Part of the Task

Not every unfinished task has an action the system can take. Suppose the user says, "Book Saturday dinner after my wife confirms whether she wants sushi or Italian." The trip is not complete, but the system cannot make progress on that step yet. It has to wait.

Microsoft’s SentinelBench studies long-running tasks where systems have to remain responsive to changes over time while avoiding unnecessary work. For practitioners, the important distinction is between active work and waiting. A long-running system needs to know when progress is possible through its own actions and when progress depends on something outside the system. Waiting is also part of the task state.

4. Revalidate What May Have Changed

Six hours later, the user replies in Italian. The system can resume the dinner step, but should it reuse the restaurant availability it found earlier?

Probably not.

The fact that Saturday dinner still needs to be booked is a durable task state. The fact that Restaurant A had a 7 PM table available six hours ago is an observation about the external world.

Those two types of information should not be treated the same way.

Before continuing, the system may need to refresh the restaurant’s availability and price. The same problem appears with inventory, account balances, permissions, software deployments, ticket availability, or any other state that can change while a task is paused.

A long-running execution therefore starts to look like:

Load task state → identify the next step → refresh information that may be stale → act → record what changed

The challenge here is identifying which assumptions remain safe to reuse and which should be checked again because time has passed.

5. Verify That the Task Is Actually Done

Eventually the system announces:

Your Tokyo trip is all set.

Before the task is closed, the system needs to check that against what the user actually asked for.

For our Tokyo trip, that might look like:

Flights - confirmed
Hotel - confirmed
Activities - confirmed
Saturday dinner - confirmed
Airport transportation - confirmed
Total spend - $2,380 / $2,500

This matters more as tasks get longer. With many intermediate steps, it is easy to complete most of the work while missing one requirement. Completion should therefore be tied to the task’s actual requirements and observable outcomes. For a code migration, that could mean verifying that the required files were changed and the tests pass. For the Tokyo trip, it means checking that everything requested was booked and that the total remains within budget.

The same task state used to track progress can help here — anything still marked as remaining means the task is not finished.

Putting the Pieces Together

At this point, the lifecycle of a long-running task becomes clearer:

Load task state
      ↓
Determine the next unfinished step
      ↓
Build the context needed for that step
      ↓
Refresh external state if needed
      ↓
Decide and act
      ↓
Record what changed
      ↓
Wait, recover or replan when necessary
      ↓
Verify completion

The model is only one part of this loop — task state records where the system is, context gives the model what it needs for the current decision, checkpoints make progress recoverable, waiting allows execution to span external events, revalidation prevents stale information from driving new actions, completion checks determine whether the requested outcome actually happened.

This is why long-horizon AI needs more than longer context windows.

What Is Still Unsolved

The pieces are becoming clearer, but several design questions remain open.

What should survive over time? Saving the full trajectory creates noise and cost. Summarizing too aggressively can discard information that becomes important later. Systems still need to decide what belongs in structured task state, memory, searchable history, or transient context.

When should a system replan? Replanning after every change wastes work and can destabilize a good plan. Never replanning allows stale assumptions to persist.

When should external information be revalidated? There is no universal expiration time for a fact. Some observations remain valid for months while others can change in seconds.

How should partial progress be evaluated? Binary success becomes increasingly crude as tasks get longer. A system may accomplish useful work without reaching the final goal, and understanding where it failed matters for both recovery and evaluation.

These are increasingly systems questions rather than model questions. Larger context windows will help. Better models will help. Neither removes the need to manage what happens between model calls.

Building for long-running tasks means thinking beyond each individual model call. The system must preserve enough state to continue, handle interruptions and changes along the way, and reliably know when the work is done.