Context and prompt caching

Cereal compacts long conversations and keeps stable prompt content reusable for providers that support caching. Both happen during the session; your app does not need to rebuild conversation history for each request.

Compaction

As context approaches the model's window, Cereal first removes eligible older tool results. If that is not enough, it asks a model to summarize the older conversation. Recent messages remain verbatim. Each change is recorded as a compaction event.

Set compaction on the agent definition:

compaction: {
  thresholdPercent: 80,
  keepRecentMessages: 10,
  contextTokens: 200_000,
  model: { model: "claude-haiku-4.6" },
  summaryWords: 600,
  prompt: "Keep the ticket ID and unresolved customer requests."
}

All fields are optional. Set contextTokens only when Cereal's model catalogue does not have the correct window. The summarization model uses the configured provider unless you specify another; choose a model your provider supports.

keepRecentTokens defaults to the smaller of 20,000 tokens and one tenth of the context window. When combined with keepRecentMessages, the rule preserving more context wins. Old sandbox reads can also be removed when a later read or edit makes them obsolete.

prompt adds instructions to the summary format. Use it for details your app needs to preserve. replaceInstructions: true replaces the default summarization instructions. Summarization adds a model request and its associated cost. If it fails, Cereal records session.error and continues with the existing transcript; that context may still be too large for a later request.

Prompt caching

Cereal keeps tool schemas and the agent's instructions ahead of changing session context. Appended session instructions and compaction summaries come later, so they can change without invalidating the stable prefix. Tool ordering stays consistent within a session.

Use overrides.system: "append" for session-specific instructions. Changing the agent's tools or replacing its system prompt can reduce cache reuse. Prefer session-level tool narrowing when a capability restriction applies to the whole conversation, rather than changing it on each turn.

Caching support, minimum prompt sizes, expiry, and pricing depend on your provider and model. A stable prefix helps reuse; it does not guarantee a cache hit.

Inspect the result

model.request.completed.usage reports total, uncached, and, when the provider supplies them, cacheRead and cacheWrite. total includes cached tokens. An absent cache counter means it was not reported; a reported zero is different.

Use the session's Trace tab to inspect model requests and cache counters. A cache.prefix event records a prefix change and helps explain a drop in reuse. See traces and usage.