When an AI Agent Forgets, Compaction Decides What It Still Means

Imagine an operations agent that has spent three days coordinating a factory repair. It has read maintenance logs, compared supplier quotes, called inventory tools, learned that one replacement part is unsafe, and received a final instruction from a manager: keep the production line running at all costs, except by using that part. Then its context window fills. What happens next is no longer a minor implementation detail. The way the system compresses its past determines whether the agent resumes as a reliable colleague or as a capable stranger with a dangerously incomplete memory.

On July 29, 2026, OpenAI published a striking result from ARC-AGI-3, a benchmark of interactive 2D logic games. With its standard harness, GPT-5.6 Sol scored 13.3 percent on the public task set. With retained reasoning and compaction enabled, the score reached 38.3 percent while using roughly six times fewer output tokens. The underlying model did not change. Its continuity of work did.

That result makes compaction harder to dismiss as a way to reduce inference bills. It is becoming an API primitive that shapes execution itself.

A long-running agent is not simply a model with a large prompt. It is a process that repeatedly decides what its earlier work means, what still matters, and what can safely become a smaller representation. That is close to memory, but not identical to memory. It is closer to creating a checkpoint in a running program, except the program is also interpreting ambiguous language, changing plans, and operating tools in the real world.

The practical question for builders is therefore not, “Does the agent support a million tokens?” It is, “What state survives the journey across context windows, and how will we know when that state has changed the job?”

The moment a conversation becomes a checkpoint

A context limit is easy to picture as a full notebook. New observations, tool outputs, plans, and messages keep arriving until there is no blank space left. The naive solution is sliding-window truncation: tear out the oldest pages and keep writing.

That can work for a casual chat. It is a poor default for an agent whose current decision depends on something learned hours ago.

OpenAI described the ARC-AGI-3 harness as using sliding truncation when its conversation passed 175,000 characters, removing older messages. The company identified two consequences: the model lost earlier observations and actions, and it spent much of the task working in a crowded context window.

Compaction takes a different route. Instead of simply dropping the past, the system transforms the prior session into a compact state that can seed later inference. The original record may still exist in application storage, but the next model call does not need to carry every old token in working context.

The useful mental model is not summary as prose. It is checkpointing as a state transition.

Before compaction, an agent may possess a chaotic but richly detailed trail: five competing hypotheses, raw database records, browser navigation steps, a failed attempt, a manager’s correction, a tool error, and an unresolved question. After compaction, it needs a smaller package that preserves the live logic of the task. What is the goal? What commitments have already been made? Which facts are verified? Which assumptions are provisional? What action is next? What must never be done?

If that package is correct, the agent wakes up with a coherent working identity. If it is incomplete, the agent may still sound confident while pursuing a subtly different assignment.

Not every part of state deserves equal treatment

The most important design choice is deciding what must remain lossless.

Some state should survive word for word, or through an equivalent structured representation that cannot blur its meaning. This category includes user instructions, policy constraints, explicit approvals, transaction identifiers, file versions, security scopes, monetary amounts, legal terms, and irreversible actions already taken. A procurement agent cannot replace “do not buy from vendor B” with “prefer vendors with good safety records.” A healthcare workflow cannot turn a contraindication into a general caution. A coding agent cannot substitute “deploy commit 9f3c” with “deploy the latest approved change.”

Other state can be semantically compressed. Repeated tool traces, routine status checks, intermediate calculations that are reproducible, duplicate search results, and abandoned explorations often belong here. The compressed representation should preserve conclusions, evidence references, uncertainty, and any reason an earlier path was rejected.

Then there is state that should move out of model context entirely. Large documents, source files, raw observations, images, tables, and extensive tool output belong in durable storage with stable identifiers. The compacted context should contain a useful map, not a miniature copy of the entire warehouse.

Finally, some material can be discarded: transient formatting, duplicate retries, stale telemetry, verbose tool chatter, and reasoning branches that neither changed the plan nor supplied evidence for a decision.

That classification cannot be left entirely to a generic compactor. It is a product and safety decision.

There is also a critical distinction between preserving an event and preserving its interpretation. A raw event might say, “The customer accepts a one-day delay.” Its interpretation may be, “Delay is now preferred over a substitute part.” But that interpretation could become wrong when a later message adds, “Only if the substitute part is safe.” A good state contract retains both the durable fact and the dependency that gives it meaning.

This is why compacted agents need explicit open questions and invalidation conditions. The state should say not only “Supplier quote selected,” but “Quote selected pending confirmation that part is not from restricted batch.” It should not only say “Customer accepts delay,” but “Delay approval expires if outage exceeds 24 hours.”

The black box at the center of continuity

There is an uncomfortable but useful fact about native compaction: developers should not assume they can inspect exactly what was retained as if it were a human-written summary.

OpenAI’s Responses API documents compaction as an operation that returns a compacted response object. Its output contains the user messages followed by a single compaction item with encrypted content. The documentation also reports token accounting for the compaction pass, including cached, reasoning, and total tokens.

That means the platform can offer continuity without exposing a neat ledger of which reasoning details survived verbatim, which tool traces were abstracted, or which internal associations were removed. This is often sensible for privacy and system design. But it changes the builder’s responsibility.

You cannot audit opaque compacted state by asking whether its summary “looks good.” You need to audit outcomes.

Build a visible application-level state ledger alongside native compaction. Record the task objective, nonnegotiable constraints, key entities and IDs, decisions, approvals, evidence pointers, open questions, next action, and state version. Treat this ledger as the agent’s explicit contract with the product. Native compaction can preserve useful cognitive continuity. The ledger protects the details that the business must be able to inspect, recover, or litigate.

That separation also makes recovery possible. If an agent begins behaving strangely after a checkpoint, you can rebuild a clean context from the contract, the recent events, and retrieved evidence rather than hoping an opaque state object will explain itself.

Compaction is not RAG, caching, or durable memory

These systems are often discussed as though they are interchangeable ways to give an agent more memory. They solve different problems.

Retrieval-augmented generation, commonly called RAG, finds relevant information from an external corpus at the moment it is needed. It is excellent for answering, “What does the contract say about cancellation?” It can fail when retrieval misses the one document that changes the answer, when relevance ranking favors topical similarity over a binding constraint, or when the agent does not realize it should search.

Prompt caching reuses matching prompt prefixes to reduce latency and cost. It does not by itself give the model a better account of what happened. OpenAI’s current documentation describes prompt cache breakpoints and a 30-minute default cache lifetime for supported models. That is an execution optimization, not a durable narrative of the agent’s work.

Durable memory stores information across sessions, often as profiles, notes, databases, or event histories. Its main risks are staleness, permission errors, unwanted persistence, and retrieving an old preference after the user has changed their mind.

Compaction is different. It reshapes the state passed across context windows within a continuing task. Its primary risk is semantic drift: the agent retains a plausible version of the past that subtly revises priorities, constraints, or causal links.

A robust agent architecture uses all four, but refuses to confuse their guarantees. Cache the stable prompt prefix. Retrieve primary evidence when decisions require it. Store durable facts under permissions and expiration rules. Compact the active task so the agent can continue without carrying every footstep forever.

Measure continuity, not just capacity

The phrase “million-token continuity” can create the wrong expectation. A model can work across a million tokens of task history without holding a million raw tokens in every inference call. Effective context is not perfect recall, and it should not be marketed as such.

OpenAI says GPT-5.6 was trained with architectural interventions including persisted reasoning and native compaction, intended to maintain coherence across longer task horizons. It also describes moving deterministic filtering, aggregation, and tool orchestration outside the context window so model tokens can be reserved for judgment.

That is meaningful progress. It is not a promise that every detail remains available forever.

The right benchmark is checkpoint fidelity. Create long tasks with planted facts and dependencies. Include an early hard constraint, a later exception, a rejected option, a partially completed tool action, and a source document that must be revisited. Run the task with and without compaction. Force checkpoints at different intervals. Then measure whether the resumed agent preserves the goal, constraint satisfaction, entity identity, tool state, evidence provenance, and calibration of uncertainty.

Instrumentation should make compaction visible in production. Log the checkpoint ID, original event range, state-contract version, count of unresolved questions, retrievals made after resume, policy checks passed, and whether the agent reversed a previous decision. Watch for regressions that appear only after the first or third checkpoint. Those are often not model failures in isolation. They are continuity failures.

The agent becomes a stateful collaborator

The future experience of working with agents may feel less like restarting a chat and more like returning to a colleague’s desk after lunch. The colleague does not need to reread every email. They need the active brief, the commitments made, the evidence behind the important calls, and a way to pull the original document when challenged.

That is the promise of compaction. It makes long work possible without forcing every new inference to drag the full past behind it.

But the design standard must be higher than “the agent kept going.” Continuity is not measured by a smooth sentence after the context limit. It is measured by whether the agent still knows what it is responsible for, what it is forbidden to do, what remains uncertain, and where its confidence came from.

Once compaction becomes an API primitive, agents stop being stateless answer machines with longer prompts. They become running systems with checkpoints. The best ones will not merely remember more. They will make the boundaries of memory legible, testable, and safe.