# Self-Modifying Agents

URL: https://cogitator.app/docs/advanced/self-modifying

> Build agents that generate new tools at runtime, adapt reasoning strategies, and evolve their architecture through multi-armed bandits.

## Overview

The `@cogitator-ai/self-modifying` package provides agents that evolve at runtime. They detect capability gaps and generate new tools, monitor their own reasoning to switch strategies, and optimize architecture parameters using bandit algorithms.

```bash
pnpm add @cogitator-ai/self-modifying
```

## Quick Start

```typescript
import { Agent, createLLMBackend } from '@cogitator-ai/core';
import { SelfModifyingAgent } from '@cogitator-ai/self-modifying';

const llm = createLLMBackend('openai', {
  providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
});

const agent = new Agent({
  name: 'adaptive-assistant',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Solve problems adaptively.',
});

const selfModifying = new SelfModifyingAgent({
  agent,
  llm,
  config: {
    toolGeneration: { enabled: true, autoGenerate: true },
    metaReasoning: { enabled: true },
    architectureEvolution: { enabled: true },
    constraints: { enabled: true, autoRollback: true },
  },
});

const result = await selfModifying.run('Analyze this CSV and visualize trends');

console.log('Output:', result.output);
console.log('Tools generated:', result.toolsGenerated.length);
console.log('Adaptations:', result.adaptationsMade.length);
```

The backend's provider prefix (`openai/` here) is stripped from the agent model, the agent's `temperature` and `maxTokens` form the baseline configuration, and concurrent `run()` calls on one instance are serialized. Set `config.enabled: false` to run as a plain tool-calling agent. Pass `modificationConstraints` to add constraints and `availableModels` to let architecture evolution switch models.

`config.maxInternalTokens` bounds the output tokens of every internal LLM call (gap analysis, tool generation and review, architecture evolution), which keeps reasoning models from spending minutes on those steps. Unset leaves the backend default. Meta-reasoning has its own bound, `metaReasoning.maxMetaTokens`. The components used on their own (`GapAnalyzer`, `ToolGenerator`, `ToolValidator`, `CapabilityAnalyzer`, `ParameterOptimizer`) take it as `maxTokens`.

## Tool Self-Generation

When the agent encounters a task requiring capabilities it doesn't have, it generates new tools at runtime.

### How It Works

1. **Gap Analysis** - LLM compares user intent and the agent's instructions with available tools (including tools generated earlier), identifies missing capabilities
2. **Code Synthesis** - Generates a plain JavaScript `execute(params)` implementation, a description of what the tool does, and a few example inputs
3. **Validation** - Static security scanning, test cases in the sandbox, optional LLM review
4. **Constraint Check** - The tool must satisfy the modification constraints (sandboxing, size, depth)
5. **Registration** - Accepted tools are stored as `active`, used immediately and loaded into later runs

A generated tool's `description` is what the model wrote about the tool. When it is empty or only repeats the gap text, the gap's `requiredCapability` is used, so the next gap analysis sees what the tool does and reuses it instead of generating a duplicate.

### Validation

* **Static checks** are authoritative: a security rule hit (`eval`, `require`, `process.`, prototype tricks, shell or file system access) makes the tool invalid with score 0, whatever a review says.
* **Sandbox tests** are authoritative too: any failed case makes the tool invalid. Without explicit test cases, the tool runs on the example inputs it was generated with (they must succeed) and on inputs synthesized from its parameters schema. Synthesized inputs only prove the tool runs, so a descriptive error is accepted for them, while leaving out a required parameter must throw.
* **LLM review** (`requireLLMValidation`) follows the reviewer's verdict. An approving review keeps the tool valid and its security and logic notes become `suggestions`. A review that recommends `revise` or `reject` makes the tool invalid, and its findings are what the next iteration fixes.

Generated code runs in a worker thread inside a fresh `vm` context without host objects (`process`, `require`, timers, host constructors) and with string code generation disabled. Parameters and results cross the boundary as JSON.

### Configuration

```typescript
const selfModifying = new SelfModifyingAgent({
  agent,
  llm,
  config: {
    toolGeneration: {
      enabled: true,
      autoGenerate: true,
      maxToolsPerSession: 3,
      minConfidenceForGeneration: 0.7,
      maxIterationsPerTool: 3,
      requireLLMValidation: true,
      sandboxConfig: {
        enabled: true,
        maxExecutionTime: 5000,
        maxMemory: 50 * 1024 * 1024,
        allowedModules: [],
        isolationLevel: 'strict',
      },
    },
  },
});
```

### Manual Tool Generation

```typescript
import { GapAnalyzer, ToolGenerator } from '@cogitator-ai/self-modifying';

const gapAnalyzer = new GapAnalyzer({ llm, config: toolGenConfig, model: 'gpt-6.1-sol' });
const toolGenerator = new ToolGenerator({ llm, config: toolGenConfig, model: 'gpt-6.1-sol' });

const analysis = await gapAnalyzer.analyze(
  'Calculate compound interest over 10 years',
  existingTools,
  { instructions: agent.instructions }
);

for (const gap of analysis.gaps) {
  const result = await toolGenerator.generate(gap, existingTools);
  if (result.success && result.tool) {
    console.log('Generated:', result.tool.name);
  }
}
```

### Generated Tool Store

```typescript
import { InMemoryGeneratedToolStore } from '@cogitator-ai/self-modifying';

const store = new InMemoryGeneratedToolStore();

await store.save(generatedTool);

await store.recordUsage({
  toolId: tool.id,
  timestamp: new Date(),
  success: true,
  executionTime: 150,
});

const tools = await store.list({ status: 'active' });
const similar = await store.findSimilar('calculate interest');
```

## Meta-Reasoning

The meta-reasoning layer monitors the agent's reasoning process and makes strategic adjustments — switching between analytical, creative, systematic, and other modes.

### Reasoning Modes

| Mode          | Temperature | Use Case                    |
| ------------- | ----------- | --------------------------- |
| `analytical`  | 0.3         | Logical analysis, debugging |
| `creative`    | 0.9         | Brainstorming, ideation     |
| `systematic`  | 0.2         | Step-by-step procedures     |
| `intuitive`   | 0.6         | Quick decisions, heuristics |
| `reflective`  | 0.4         | Self-assessment, learning   |
| `exploratory` | 0.7         | Open-ended exploration      |

### Configuration

```typescript
const selfModifying = new SelfModifyingAgent({
  agent,
  llm,
  config: {
    metaReasoning: {
      enabled: true,
      defaultMode: 'analytical',
      allowedModes: ['analytical', 'creative', 'systematic'],
      modeProfiles: {
        analytical: { temperature: 0.2, depth: 4 },
        creative: { temperature: 1.0 },
      },
      maxMetaAssessments: 5,
      maxAdaptations: 3,
      triggers: ['on_failure', 'on_low_confidence', 'periodic'],
      triggerAfterIterations: 3,
      triggerOnConfidenceDrop: 0.3,
      enableRollback: true,
    },
  },
});
```

`modeProfiles` overrides the temperature, depth or parameters of individual modes; each given field is merged over the mode's default in `DEFAULT_MODE_PROFILES`, so list only what you change. `mergeModeProfiles(base, overrides)` applies the same merge to profiles of your own.

Inside `SelfModifyingAgent`, meta-reasoning only intervenes when a step is incomplete (empty or truncated answer). Only triggers listed in `triggers` fire, and the next attempt uses the adapted mode temperature, parameters or injected context. LLM assessments are validated; recommendations without a valid confidence are ignored.

### Direct MetaReasoner Usage

```typescript
import { MetaReasoner } from '@cogitator-ai/self-modifying';

const metaReasoner = new MetaReasoner({ llm, model: 'gpt-6.1-sol', config: metaReasoningConfig });

const modeConfig = metaReasoner.initializeRun(runId);

const observation = metaReasoner.observe(
  {
    runId,
    iteration: 3,
    goal: 'Analyze data',
    currentMode: 'analytical',
    tokensUsed: 1500,
    timeElapsed: 5000,
    iterationsRemaining: 2,
    budgetRemaining: 0.6,
  },
  insights
);

const assessment = await metaReasoner.assess(observation);

if (assessment.requiresAdaptation) {
  const adaptation = await metaReasoner.adapt(runId, assessment);
  console.log('Switched to:', adaptation?.after?.mode);
}
```

## Architecture Evolution

Optimizes agent parameters (temperature, max tokens, tool strategy, reflection depth and — with `availableModels` — the model) using multi-armed bandit algorithms. Every run records its outcome for the configuration that was actually used, LLM-proposed candidates are sanitized, and high-risk candidates are discarded.

### Strategies

| Strategy            | Description                                   |
| ------------------- | --------------------------------------------- |
| `ucb`               | Upper Confidence Bound — balanced exploration |
| `thompson_sampling` | Thompson Sampling — probabilistic selection   |
| `epsilon_greedy`    | Epsilon-Greedy — random exploration           |

### Parameter Optimizer

```typescript
import { ParameterOptimizer } from '@cogitator-ai/self-modifying';

const optimizer = new ParameterOptimizer({
  llm,
  model: 'gpt-6.1-sol',
  config: evolutionConfig,
  baseConfig: {
    model: 'gpt-6.1-sol',
    temperature: 0.7,
    maxTokens: 4096,
    toolStrategy: 'sequential',
    reflectionDepth: 0,
  },
  availableModels: ['gpt-6.1-sol', 'gpt-6-luna'],
});

const result = await optimizer.optimize('Complex reasoning task');

console.log('Should adopt:', result.shouldAdopt);
console.log('Recommended config:', result.recommendedConfig);

await optimizer.recordOutcome(result.candidate!.id, result.taskProfile, {
  successRate: 1,
  latency: 1200,
  tokenUsage: 850,
  qualityScore: 0.9,
});
```

### Capability Analyzer

```typescript
import { CapabilityAnalyzer } from '@cogitator-ai/self-modifying';

const analyzer = new CapabilityAnalyzer({ llm, enableLLMAnalysis: true, model: 'gpt-6.1-sol' });

const profile = await analyzer.analyzeTask('Build a REST API with authentication');

console.log('Complexity:', profile.complexity);
console.log('Domain:', profile.domain);
console.log('Tool intensity:', profile.toolIntensity);
```

## Constraints & Safety

All self-modifications are validated against safety constraints before being applied.

```typescript
import {
  ModificationValidator,
  DEFAULT_SAFETY_CONSTRAINTS,
  DEFAULT_CAPABILITY_CONSTRAINTS,
  DEFAULT_RESOURCE_CONSTRAINTS,
} from '@cogitator-ai/self-modifying';

const validator = new ModificationValidator({
  constraints: {
    safety: DEFAULT_SAFETY_CONSTRAINTS,
    capability: DEFAULT_CAPABILITY_CONSTRAINTS,
    resource: DEFAULT_RESOURCE_CONSTRAINTS,
    custom: [
      {
        id: 'no-external-apis',
        name: 'No External APIs',
        description: 'External API calls not allowed',
        predicate: (request) => !JSON.stringify(request.changes).includes('fetch'),
      },
    ],
  },
});

const result = await validator.validate({
  type: 'tool_creation',
  target: 'tools',
  changes: { name: 'new-tool', code: '...' },
  reason: 'User requested capability',
});
```

The default safety constraints carry `appliesTo: ['tool_generation', 'tool_creation']`, so configuration changes are not checked against tool-only rules. Rules support `=`, `!=`, `<`, `<=`, `>`, `>=`, `AND`, `OR`, numbers, booleans, `null` and quoted strings.

### Rollback Manager

```typescript
import { RollbackManager } from '@cogitator-ai/self-modifying';

const rollbackManager = new RollbackManager({ maxCheckpoints: 10 });

const checkpoint = await rollbackManager.createCheckpoint(
  agentName,
  agentConfig,
  currentTools,
  modifications
);

const restored = await rollbackManager.rollbackTo(checkpoint.id);
```

## Events

Subscribe to self-modification events for observability. Handlers receive typed event data and `on()` returns an unsubscribe function.

```typescript
selfModifying.on('tool_generation_started', (e) => {
  console.log('Generating:', e.data.gap.suggestedToolName);
});

selfModifying.on('strategy_changed', (e) => {
  console.log(`Mode: ${e.data.previousMode} → ${e.data.newMode}`);
});

selfModifying.on('run_completed', (e) => {
  console.log('Success:', e.data.success);
});
```

| Event                       | Description                     |
| --------------------------- | ------------------------------- |
| `run_started`               | Self-modifying run started      |
| `run_completed`             | Run completed (success/failure) |
| `tool_generation_started`   | Started generating a new tool   |
| `tool_generation_completed` | Tool generation finished        |
| `meta_assessment`           | Meta-reasoning assessment made  |
| `strategy_changed`          | Reasoning mode switched         |
| `architecture_evolved`      | Architecture config changed     |
| `checkpoint_created`        | Rollback checkpoint created     |
| `rollback_performed`        | Rolled back to checkpoint       |

## Utilities

### extractJson

Extracts the first valid JSON object from a string using balanced-brace matching.

```typescript
import { extractJson } from '@cogitator-ai/self-modifying';

const raw = 'Here is my analysis: {"onTrack": true} end.';
const json = extractJson(raw); // '{"onTrack": true}'
```

### llmChat

Normalizes LLM backend calls — uses `complete()` if available, falls back to `chat()`.

```typescript
import { llmChat } from '@cogitator-ai/self-modifying';

const response = await llmChat(llm, [{ role: 'user', content: 'Analyze this data' }], {
  model: 'gpt-6.1-sol',
});
```