Long Conversations

medium10 mincontext-manager.ts

A conversation has outgrown the model’s context window. You want to know how full it is and compress it before the request fails.

What it shows

Measure usage with checkState()
Compare the truncate and sliding-window strategies with compress()
Look up a model’s window with getModelContextLimit()

The code

context-manager.ts

import { ContextManager } from '@cogitator-ai/core';
import type { Message } from '@cogitator-ai/types';

const MODEL = 'ollama:llama3.1:8b';

function longConversation(turns: number): Message[] {
  const messages: Message[] = [{ role: 'system', content: 'You are a senior software engineer.' }];
  for (let i = 0; i < turns; i++) {
    messages.push({ role: 'user', content: `Turn ${i + 1}: tell me about distributed systems. ${'x'.repeat(200)}` });
    messages.push({ role: 'assistant', content: `Turn ${i + 1}: here is what I know. ${'x'.repeat(200)}` });
  }
  return messages;
}

const conversation = longConversation(80);

for (const strategy of ['truncate', 'sliding-window'] as const) {
  const manager = new ContextManager({
    strategy,
    compressionThreshold: 0.3,
    outputReserve: 0.15,
    windowSize: 10,
  });

  const before = manager.checkState(conversation, MODEL);
  console.log(`${strategy}: ${before.utilizationPercent.toFixed(1)}% of the window used, needs compression: ${before.needsCompression}`);

  const result = await manager.compress(conversation, MODEL);
  const savings = (1 - result.compressedTokens / result.originalTokens) * 100;
  console.log(
    `  ${conversation.length} -> ${result.messages.length} messages, ` +
      `${result.originalTokens} -> ${result.compressedTokens} tokens (${savings.toFixed(1)}% saved)`
  );
}

const limits = new ContextManager({ strategy: 'truncate' });
console.log('gemini-3.8-flash window:', limits.getModelContextLimit('google:gemini-3.8-flash'));

Run it

install · bash

pnpm add @cogitator-ai/core @cogitator-ai/types

run · bash

npx tsx context-manager.ts

Or run the full example from a clone of the repository (keys in .env):

repository · bash

npx tsx examples/core/04-context-manager.ts

Long Conversations

What it shows

The code

Run it

Learn more