Cogitator
Advanced

Time-Travel Debugging

Capture execution snapshots at any step, replay from checkpoints, fork alternate timelines, and compare runs side-by-side to debug agent behavior.

Overview

Time-travel debugging lets you capture snapshots of agent execution at any point, then replay, fork, or compare runs. Instead of re-running an entire agent from scratch to debug a failure at step 47, you load the checkpoint at step 46 and explore what happens with different inputs, tools, or context.

Setting Up TimeTravel

import { Cogitator, TimeTravel } from '@cogitator-ai/core';

const cogitator = new Cogitator({/* ... */});

const timeTravel = new TimeTravel(cogitator, {
  config: {
    maxCheckpointsPerTrace: 50, // oldest checkpoints of a trace go first
    checkpointRetention: 24 * 60 * 60 * 1000, // ms; older checkpoints of the agent are deleted
  },
});

Checkpoints and the traces used for comparisons live in process memory (InMemoryCheckpointStore, InMemoryTraceStore). Pass checkpointStore (any TimeTravelCheckpointStore) and traceStore (any TraceStore, the interface AgentOptimizer from Agent Learning uses as well) to keep them elsewhere; getCheckpointStore() and getTraceStore() return the ones in use. The limits above are the defaults; checkpoint(), checkpointAll() and checkpointEvery() apply them.

Creating Checkpoints

Checkpoints are cut from a finished RunResult. The steps checkpoints count are the run's tool calls: a checkpoint at step i is anchored on the run's i-th tool call (counting from 0), so checkpoint(result, n) captures the state after n tool calls have returned.

const result = await cogitator.run(agent, { input: 'Research quantum computing advances' });

// checkpoint a specific step
const checkpoint = await timeTravel.checkpoint(result, 3, 'after-web-search');

// checkpoint every tool call
const allCheckpoints = await timeTravel.checkpointAll(result, 'research-run');

// checkpoint every N tool calls
const sparseCheckpoints = await timeTravel.checkpointEvery(result, 2, 'sparse');

Each checkpoint stores:

  • messages — the conversation up to that tool call, without its result
  • toolResults — the results of the earlier tool calls, by call id
  • pendingToolCalls — the tool call the step is anchored on
  • label — for retrieval with getByLabel() on the store; checkpointAll and checkpointEvery label them <prefix>_step_<i>

Browsing Checkpoints

Retrieve checkpoints by trace, agent, or label:

const checkpoints = await timeTravel.getCheckpoints(result.trace.traceId);

for (const cp of checkpoints) {
  console.log(`Step ${cp.stepIndex}: ${cp.label ?? 'unlabeled'}`);
  console.log(`  Messages: ${cp.messages.length}`);
  console.log(`  Tool results: ${Object.keys(cp.toolResults).length}`);
  console.log(`  Created: ${cp.createdAt.toISOString()}`);
}

const specific = await timeTravel.getCheckpoint('ckpt_abc123');
await timeTravel.deleteCheckpoint('ckpt_abc123');

Replaying from a Checkpoint

Replay continues from a saved checkpoint. Two modes are available:

Deterministic Replay

Returns the state at the checkpoint without making any model or tool calls: output is the last assistant message in the checkpoint, and messages carry the checkpoint's cached tool results:

const replay = await timeTravel.replayDeterministic(agent, checkpoint.id);

console.log('Output:', replay.output);
console.log('Steps replayed:', replay.stepsReplayed);
console.log('Steps executed:', replay.stepsExecuted); // 0 in deterministic mode

stepsReplayed is the checkpoint's stepIndex: the number of tool calls before it.

Live Replay

Runs the agent again from the checkpoint with fresh model calls. The replayed agent is a copy named <name>_replay whose instructions — the checkpoint's system message, or the agent's own instructions when it has none — carry the conversation up to the checkpoint (its assistant and tool messages); it answers the checkpoint's last user message in a thread of its own (replay_<runId>):

const replay = await timeTravel.replayLive(agent, checkpoint.id);

console.log('Output:', replay.output);
console.log('Steps replayed:', replay.stepsReplayed);
console.log('Steps executed:', replay.stepsExecuted);

if (replay.divergedAt !== undefined) {
  console.log(`Execution diverged at step ${replay.divergedAt}`);
}

stepsExecuted counts the replay's tool calls. divergedAt is numbered like the original run's steps: stepIndex when the replay's first tool call differs from the call the checkpoint is anchored on (another tool, or the same tool with different arguments) or when it calls no tool, and stepIndex + 1 when it matches but the replay goes on to call more tools.

Live replays take options to change what the agent sees:

const replay = await timeTravel.replayLive(agent, checkpoint.id, {
  modifiedToolResults: { web_search: { results: [] } }, // the tool answers this instead of running
  skipTools: ['send_email'], // removed from the replayed agent
  modifiedMessages: editedMessages, // replaces the checkpoint's messages
});

replay(agent, checkpointId, { mode }) is the general form of both.

Forking Execution

Forking creates an alternate timeline from a checkpoint with modified conditions. This is the core debugging primitive. Each fork saves a copy of the checkpoint (fork.checkpoint, id ckpt_fork_..., with metadata.forkedFrom) and runs a live replay from it (fork.result).

Fork with Additional Context

Append new information to the system prompt:

const fork = await timeTravel.forkWithContext(
  agent,
  checkpoint.id,
  'The user is a premium subscriber with access to advanced features',
  'premium-context'
);

console.log('Fork output:', fork.result.output);

Fork with Mocked Tools

Make tools answer with fixed results instead of running, to test "what if" scenarios:

const fork = await timeTravel.forkWithMockedTool(
  agent,
  checkpoint.id,
  'web_search',
  { results: [], error: 'Service unavailable' },
  'search-failure'
);

// mock multiple tools at once
const multiFork = await timeTravel.forkWithMockedTools(
  agent,
  checkpoint.id,
  {
    web_search: { results: [] },
    database_query: { rows: [], error: 'Connection timeout' },
  },
  'all-services-down'
);

Fork with New Input

Replace the checkpoint's last user message, keeping the context before it:

const fork = await timeTravel.forkWithNewInput(
  agent,
  checkpoint.id,
  'Now focus specifically on quantum error correction',
  'refined-question'
);

Fork Multiple Variants

Explore several alternatives from the same checkpoint in one call:

const forks = await timeTravel.forkMultiple(agent, checkpoint.id, [
  { additionalContext: 'Focus on practical applications', label: 'practical' },
  { additionalContext: 'Focus on theoretical foundations', label: 'theoretical' },
  { mockToolResults: { web_search: { results: [] } }, label: 'no-search' },
  { input: 'Explain quantum entanglement instead', label: 'different-topic' },
]);

for (const fork of forks) {
  console.log(`${fork.checkpoint.label}: ${fork.result.output.slice(0, 100)}...`);
}

Variants run one after another. A variant with both additionalContext and input uses only the context; mockToolResults combines with either. fork(agent, checkpointId, options) runs a single variant.

Comparing Runs

The TraceComparator produces structured diffs between two execution traces. compare() reads them from the TimeTravel's trace store, which holds the trace of every run you checkpointed and of every replay and fork; an unknown id throws Trace not found:

const diff = await timeTravel.compare(result.trace.traceId, fork.result.trace.traceId);

console.log(`Common steps: ${diff.commonSteps}`);
console.log(`Only in original: ${diff.trace1OnlySteps}`);
console.log(`Only in replay: ${diff.trace2OnlySteps}`);

if (diff.divergencePoint !== undefined) {
  console.log(`Diverged at step ${diff.divergencePoint}`);
}

console.log(`Score delta: ${diff.metricsDiff.score.delta}`);
console.log(`Token delta: ${diff.metricsDiff.tokens.delta}`);
console.log(`Duration delta: ${diff.metricsDiff.duration.delta}ms`);

After a replay, compare directly against the original:

const replayDiff = await timeTravel.compareWithOriginal(replay);
console.log(timeTravel.formatDiff(replayDiff));

The formatted diff output looks like:

═══════════════════════════════════════════
              TRACE COMPARISON
═══════════════════════════════════════════

Trace 1: trace_abc123
Trace 2: trace_def456

⚠ Traces diverged at step 3

─── Summary ───
Common steps:     3
Only in trace 1:  2
Only in trace 2:  1

─── Metrics ───
Success:  true → true
Score:    0.850 → 0.920 (+0.070)
Tokens:   4200 → 3800 (-400)
Duration: 2300ms → 1900ms (-400ms)

─── Step Differences ───
✗ Step 3: different
   └─ Tool: web_search → database_query
≈ Step 4: similar
   └─ LLM response differs

Step diff statuses: identical (exact match), similar (same structure, different LLM text), different (structural change), only_in_1, or only_in_2.

Debugging Workflow

A typical debugging session:

  1. Run the agent and observe the failure
  2. Create checkpoints for the entire run with checkpointAll()
  3. Identify the last good step by browsing checkpoints
  4. Fork from that step with different conditions to isolate the cause
  5. Compare the fork against the original to quantify the improvement
  6. Use insights to fix the agent's instructions or tool configuration

Cost-Aware Routing

Route tasks to the cheapest capable model, enforce spending budgets, estimate token usage before calling, and track costs across agents and runs.

Causal Reasoning

Build causal graphs, run d-separation analysis, perform counterfactual reasoning, and predict intervention effects.

On this page

OverviewSetting Up TimeTravelCreating CheckpointsBrowsing CheckpointsReplaying from a CheckpointDeterministic ReplayLive ReplayForking ExecutionFork with Additional ContextFork with Mocked ToolsFork with New InputFork Multiple VariantsComparing RunsDebugging Workflow