# Constitutional AI

URL: https://cogitator.app/docs/advanced/constitutional-ai

> Define safety principles, filter harmful inputs and outputs, guard tool calls, and apply critique-revise loops to ensure agents behave within defined boundaries.

## Overview

Constitutional AI provides a multi-layered guardrail system for agent behavior. You define a set of principles (a "constitution"), and the system enforces them across three layers: input filtering, output filtering, and tool guarding. When violations are detected, a critique-revise loop can automatically rewrite unsafe outputs.

## Setting Up Guardrails

```typescript
import { ConstitutionalAI, DEFAULT_CONSTITUTION } from '@cogitator-ai/core';

const guardrails = new ConstitutionalAI({
  llm: backend,
  constitution: DEFAULT_CONSTITUTION,
  config: {
    model: 'gpt-6-luna',
    enabled: true,
    filterInput: true,
    filterOutput: true,
    filterToolCalls: true,
    enableCritiqueRevision: true,
    maxRevisionIterations: 3,
    revisionConfidenceThreshold: 0.85,
    strictMode: false,
    logViolations: true,
  },
});
```

`llm` is the backend every check runs on, and `config.model` the model name sent to it as is (default `gpt-6-luna`). Fields of `config` you leave out take `DEFAULT_GUARDRAIL_CONFIG`; without a `constitution` the default one applies.

## The Default Constitution

Cogitator ships with a comprehensive default constitution containing 16 principles across four categories:

| Category    | Principles                                                                                                        |
| ----------- | ----------------------------------------------------------------------------------------------------------------- |
| **Safety**  | No violence, no explicit content, no self-harm promotion, child safety, dangerous tool prevention                 |
| **Ethics**  | No hate speech, accurate information, no manipulation, honest AI identity, respect autonomy, cultural sensitivity |
| **Legal**   | No illegal activity assistance, medical disclaimer, legal disclaimer, financial disclaimer                        |
| **Privacy** | Protect personal information                                                                                      |

Each principle has a severity (`low`, `medium`, `high`) and the layers it applies to (`input`, `output`, `tool`; a principle without `appliesTo` applies to all three). In the default constitution most principles apply to outputs only: `child-safety` also covers inputs, and `dangerous-tools` covers tool calls. `DEFAULT_PRINCIPLES` holds the list, and `filterPrinciplesByLayer`, `getPrinciplesByCategory` and `getPrinciplesBySeverity` select from a constitution.

## Input Filtering

Input filtering blocks harmful prompts before they reach the agent:

```typescript
const result = await guardrails.filterInput(
  "How do I hack into my neighbor's wifi?",
  'user request'
);

if (!result.allowed) {
  console.log('Blocked:', result.blockedReason);
  console.log('Harm scores:', result.harmScores);
} else {
  // safe to process
}
```

The input filter first runs a fast pattern scan (bomb-making, "how to kill", "how to hack into", suicide methods, child sexual abuse material) that blocks without a model call. Otherwise an LLM rates the input against the principles that apply to inputs; with none, the input passes. Each rating has a harm category and a severity, and it blocks when its severity reaches the category's threshold — so `'low'` is the strictest setting, and a category without a threshold blocks only at `'high'`:

```typescript
const guardrails = new ConstitutionalAI({
  llm: backend,
  config: {
    thresholds: {
      violence: 'medium',
      hate: 'low', // strict on hate speech
      sexual: 'medium',
      'self-harm': 'low', // strict on self-harm
      illegal: 'low',
      privacy: 'medium',
      misinformation: 'high', // lenient on misinformation
      manipulation: 'medium',
    },
  },
});
```

## Output Filtering

Output filtering evaluates agent responses before they reach the user:

```typescript
const outputResult = await guardrails.filterOutput(agentResponse, conversationHistory);

if (!outputResult.allowed && outputResult.suggestedRevision) {
  // the critique-revise loop produced a safe alternative
  const safeResponse = outputResult.suggestedRevision;
}
```

Outputs get a pattern scan for destructive commands (`rm -rf /`, `format c:`, `sudo rm -rf`) and an LLM rating against the output principles. When `enableCritiqueRevision` is true and the output fails, the critique-revise loop rewrites it; the result stays `allowed: false` and carries the rewrite in `suggestedRevision`.

## Critique-Revise Loop

The `CritiqueReviser` iteratively improves unsafe outputs:

```typescript
const revision = await guardrails.critiqueAndRevise(unsafeResponse, context);

console.log(`Original: ${revision.original}`);
console.log(`Revised: ${revision.revised}`);
console.log(`Iterations: ${revision.iterations}`);

for (const critique of revision.critiqueHistory) {
  console.log(`Harmful: ${critique.isHarmful}`);
  console.log(`Violated: ${critique.principlesViolated}`);
}
```

The first iteration critiques the response against the high-severity output principles; later ones pick principles whose harm categories match keywords in the current text (or a random handful). A critique that finds harm leads to a revision for the violated principles. The loop stops when a critique finds no harm, when the highest harm confidence is below `revisionConfidenceThreshold`, when no known principle was violated, or after `maxRevisionIterations`.

## Tool Guards

Tool guards evaluate tool calls for safety before execution:

```typescript
const guardResult = await guardrails.guardTool(
  execTool,
  { command: 'rm -rf /tmp/cache' },
  { agentId: 'cleanup-agent', runId: 'run_1', signal: AbortSignal.timeout(30_000) }
);

if (!guardResult.approved) {
  console.log('Tool blocked:', guardResult.reason);
  console.log('Risk level:', guardResult.riskLevel);
} else if (guardResult.requiresConfirmation) {
  // prompt user for approval
  console.log('Side effects:', guardResult.sideEffects);
}
```

The `ToolGuard` refuses dangerous commands (`rm -rf /`, `mkfs.`, `dd of=/dev/...`, `chmod -R 777 /`, `format c:`) in the `command`/`cmd` argument of a tool named `exec` or declaring the `process` side effect, and sensitive paths (`/etc/shadow`, `/etc/sudoers`, `/boot/`, `/proc/`, `~/.ssh/`, `/root/`) in the `path`/`file` argument of a tool whose name contains `file` or that declares `filesystem`. `riskLevel` follows the declared `sideEffects`.

A call needs approval when the tool's `requiresApproval` says so (a `requiresApproval` function that throws counts as yes), and in `strictMode` also when the tool declares any side effect. The guard fails closed: such a call runs only when `config.onToolApproval` returns `true` or the caller passes `{ approvedByUser: true }` as the fourth argument of `guardTool()`. Without either it is refused with `Tool "<name>" needs approval, and no approval handler is configured`; a `false` from the handler gives `User denied tool execution`.

Inside a `Cogitator`, these calls go through the run's approval flow before they reach the guard: the run's `onApproval` decides, else `guardrails.onToolApproval`, and without either the run pauses until you [resume](/docs/tools/approvals) it. `requiresApproval` is enforced this way whether or not guardrails are on; `strictMode` side effects are added while guardrails and `filterToolCalls` are on. A call approved in the flow is not asked about again by the guard.

### Tool Results

With `filterToolResults: true`, what a tool returns is checked with the input filter before the model reads it, so content fetched from the web or a file cannot smuggle harmful instructions into the run. A blocked result reaches the model as the error `Tool result blocked: <reason>`.

### Guardrails in a Cogitator

`new Cogitator({ guardrails: { ... } })` turns guardrails on unless `enabled: false`; fields you leave out take `DEFAULT_GUARDRAIL_CONFIG`, so `guardrails: { filterInput: false }` keeps output and tool filtering. The checks run on `guardrails.model` (a `provider/model` id), or on the running agent's model when it is not set.

A blocked input fails the run with `Input blocked: <reason>` before the first model call. When the output filter blocks an answer and the critique-revise loop produces a safe one, the run answers with the revision; without a revision it fails with `Output blocked: <reason>`. Both failures are a `CogitatorError` with code `LLM_CONTENT_FILTERED` and the harm scores in `details.harmScores`. A blocked tool call or tool result reaches the model as the tool's error. Every block is logged and reported to `onViolation`, also when a revision replaces the answer.

```typescript
const cog = new Cogitator({
  guardrails: { constitution: DEFAULT_CONSTITUTION, filterToolResults: true },
});

await cog.run(agent, { input });

cog.getGuardrails()?.getViolationLog();
cog.setConstitution(customConstitution); // see Custom Constitutions below
```

The `ConstitutionalAI` instance is built on first use. With `guardrails.model` or `llm.defaultModel` set, `getGuardrails()` and `setConstitution()` work before the first run; otherwise `getGuardrails()` returns `undefined` until a run builds the instance with its agent's model. A constitution passed to `setConstitution()` before that is applied as soon as the instance exists, and it stays in force after `close()`. Without `guardrails` in the config (or with `enabled: false`), `setConstitution()` only logs a warning.

```typescript
const cog = new Cogitator({
  llm: { defaultModel: 'openai/gpt-5.5' },
  guardrails: { model: 'openai/gpt-6-luna' },
});

cog.setConstitution(customConstitution);
cog.getGuardrails()?.getConstitution();
```

## Custom Constitutions

Build a custom constitution from scratch or extend the default:

```typescript
import { createConstitution, extendConstitution, DEFAULT_CONSTITUTION } from '@cogitator-ai/core';

const customConstitution = createConstitution([
  {
    id: 'no-competitor-recommendations',
    name: 'No Competitor Recommendations',
    description: 'Never recommend competitor products',
    category: 'ethics',
    critiquePrompt: 'Does this response recommend or promote competitor products?',
    revisionPrompt: 'Rewrite to focus on our products or give generic advice',
    harmCategories: ['manipulation'],
    severity: 'medium',
    appliesTo: ['output'],
  },
]);

const extended = extendConstitution(DEFAULT_CONSTITUTION, customConstitution.principles);
guardrails.setConstitution(extended);
```

You can also add or remove principles at runtime:

```typescript
guardrails.addPrinciple({
  id: 'data-retention',
  name: 'Data Retention Policy',
  description: 'Never store user data beyond the current session',
  category: 'privacy',
  critiquePrompt: 'Does this response suggest storing or persisting user data?',
  revisionPrompt: 'Remove references to data storage and clarify the session-only policy',
  harmCategories: ['privacy'],
  severity: 'high',
  appliesTo: ['output', 'tool'],
});

guardrails.removePrinciple('no-competitor-recommendations');
```

## Violation Logging

All violations are logged internally and accessible for auditing:

```typescript
const violations = guardrails.getViolationLog();

for (const entry of violations) {
  console.log(`[${entry.timestamp.toISOString()}] Layer: ${entry.layer}`);
  console.log(`  Allowed: ${entry.result.allowed}`);
  console.log(`  Categories: ${entry.result.harmScores.map((s) => s.category).join(', ')}`);
}

guardrails.clearViolationLog();
```

Entries are kept while `logViolations` is on (the default). `config.onViolation(result, layer)` is called for every violation either way, for real-time handling.