# Context Management

URL: https://cogitator.app/docs/advanced/context-management

> Automatically manage conversation length with truncation, summarization, sliding window, and smart hybrid compression strategies.

## Overview

Long-running agents accumulate conversation history that can exceed a model's context window. The `ContextManager` automatically detects when messages are approaching the limit and compresses them using one of four strategies -- keeping agents running indefinitely without losing critical context.

```typescript
import { Cogitator } from '@cogitator-ai/core';

const cog = new Cogitator({
  llm: { providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } } },
  context: {
    enabled: true,
    strategy: 'hybrid',
    compressionThreshold: 0.8,
    outputReserve: 0.15,
    windowSize: 10,
  },
});
```

Context management is built into the runtime: whenever `context` is set (and `enabled` is not `false`), the run checks the conversation before each model call and compresses it when needed. Compression changes only what is sent; memory keeps the full history. After compression, tool results whose call was dropped (and calls without their results) are removed too, so the provider never sees a broken tool exchange.

## Configuration

```typescript
interface ContextManagerConfig {
  enabled?: boolean; // default: true
  strategy?: 'truncate' | 'sliding-window' | 'summarize' | 'hybrid';
  compressionThreshold?: number; // 0-1, triggers at this % of max tokens (default: 0.8)
  outputReserve?: number; // fraction of context reserved for output (default: 0.15)
  summaryModel?: string; // provider/model id for LLM summaries (see below)
  windowSize?: number; // messages to keep in sliding window (default: 10)
}
```

The `compressionThreshold` controls when compression kicks in. At `0.8`, compression triggers when token usage exceeds 80% of the available context (the model's context window minus `outputReserve`). Compression then brings the conversation back under the available context.

Which strategies call a model depends on `summaryModel`: with it, sliding-window, summarize and hybrid compression summarize older messages with that model. Without it, only `strategy: 'summarize'` calls a model (the running agent's); sliding-window and hybrid fall back to an extractive summary without an LLM call.

## Token Counting

Cogitator uses a fast heuristic token counter (4 characters per token plus 4 per message) for real-time decisions. Context windows come from the [model registry](/docs/core/model-registry), with built-in fallbacks for well-known model families; an unknown model counts as 8,192 tokens:

```typescript
import { ContextManager } from '@cogitator-ai/core';

const manager = new ContextManager({
  enabled: true,
  strategy: 'hybrid',
  compressionThreshold: 0.8,
});

manager.getModelContextLimit('anthropic/claude-sonnet-5-5'); // 1000000
manager.getModelContextLimit('ollama/my-finetune'); // 8192
```

## Checking Context State

Before compressing, you can inspect the current state:

```typescript
const state = manager.checkState(messages, 'openai/gpt-6.1-sol');

console.log(state.currentTokens); // estimated tokens used
console.log(state.maxTokens); // effective limit (minus output reserve)
console.log(state.availableTokens); // remaining space
console.log(state.utilizationPercent); // e.g. 85.3
console.log(state.needsCompression); // true if above threshold
```

## Strategies

### Truncate

Drops the oldest non-system messages to fit within the token budget. Fast and stateless -- no LLM call required.

```typescript
const cog = new Cogitator({
  context: { strategy: 'truncate', compressionThreshold: 0.8 },
  // ...
});
```

System messages are always preserved. History is trimmed from the beginning, keeping the most recent exchanges intact. Best for scenarios where older context is rarely needed.

### Sliding Window

Keeps a fixed window of recent messages and optionally summarizes older ones. Provides a good balance between context preservation and token efficiency.

```typescript
const cog = new Cogitator({
  context: {
    strategy: 'sliding-window',
    windowSize: 15,
    summaryModel: 'openai/gpt-6-luna',
  },
  // ...
});
```

Messages outside the last `windowSize` are summarized into a single system message (`[Previous conversation summary]`): by `summaryModel` when it is set, else as a basic extractive summary of the last few user and assistant messages. If even the window does not leave room for a summary, the older messages are dropped.

### Summarize

Uses an LLM to generate a comprehensive summary of older conversation history, preserving key facts, decisions, and pending tasks.

```typescript
const cog = new Cogitator({
  context: {
    strategy: 'summarize',
    summaryModel: 'openai/gpt-6-luna',
    windowSize: 5,
  },
  // ...
});
```

The summarizer keeps the most recent 20% of messages as is (at least 2, at most `windowSize`) and replaces everything older with one summary message. It uses `summaryModel`, or the running agent's model when that is not set. The summary prompt instructs the LLM to preserve user preferences, key decisions, and critical context. Falls back to an extractive summary if the LLM call fails.

### Hybrid (Default)

Automatically selects the best strategy based on how far over the limit the conversation is:

| Size vs. available context    | Strategy       | Reasoning                                    |
| ----------------------------- | -------------- | -------------------------------------------- |
| Up to 1.5x                    | Sliding Window | Mild overflow, window is sufficient          |
| Over 1.5x, `summaryModel` set | Summarize      | Heavy overflow, need intelligent compression |
| 1.5x–2x, no `summaryModel`    | Sliding Window | Best effort without LLM                      |
| Over 2x, no `summaryModel`    | Truncate       | Extreme overflow, aggressive trimming needed |

```typescript
const cog = new Cogitator({
  context: { strategy: 'hybrid', summaryModel: 'openai/gpt-6-luna' },
  // ...
});
```

This is the recommended default -- it adapts to the situation without manual tuning.

## Compression Results

Every compression call returns detailed metrics:

```typescript
const result = await manager.compress(messages, 'openai/gpt-6.1-sol');

console.log(result.messages); // compressed message array
console.log(result.originalTokens); // tokens before compression
console.log(result.compressedTokens); // tokens after compression
console.log(result.strategy); // which strategy was used
console.log(result.truncated); // messages dropped (truncate)
console.log(result.summarized); // messages summarized (summarize/sliding-window)
```

## Standalone Usage

You can use `ContextManager` outside of the Cogitator runtime for custom pipelines:

```typescript
import { ContextManager } from '@cogitator-ai/core';

const manager = new ContextManager(
  {
    enabled: true,
    strategy: 'sliding-window',
    windowSize: 10,
    summaryModel: 'openai/gpt-6-luna',
  },
  { route: (model) => cog.route(model) }
);

if (manager.shouldCompress(messages, 'openai/gpt-6.1-sol')) {
  const { messages: compressed } = await manager.compress(messages, 'openai/gpt-6.1-sol');
  messages = compressed;
}
```

The second constructor argument tells the manager how to reach a model for summaries: `route` returns the backend and the model name for a `provider/model` id (`cog.route()` does exactly that), and `getBackend` is the older form that returns only the backend. Without either, summaries are extractive. The strategy classes (`TruncateStrategy`, `SlidingWindowStrategy`, `SummarizeStrategy`, `HybridStrategy`) are exported too, for custom pipelines that call `compress()` on a strategy directly.