Constitutional AI

OverviewCopy Anchor Link

Constitutional AI provides a multi-layered guardrail system for agent behavior. You define a set of principles (a "constitution"), and the system enforces them across three layers: input filtering, output filtering, and tool guarding. When violations are detected, a critique-revise loop can automatically rewrite unsafe outputs.

Setting Up GuardrailsCopy Anchor Link

import { ConstitutionalAI, DEFAULT_CONSTITUTION } from '@cogitator-ai/core';

const guardrails = new ConstitutionalAI({
  llm: backend,
  constitution: DEFAULT_CONSTITUTION,
  config: {
    model: 'gpt-6-luna',
    enabled: true,
    filterInput: true,
    filterOutput: true,
    filterToolCalls: true,
    enableCritiqueRevision: true,
    maxRevisionIterations: 3,
    revisionConfidenceThreshold: 0.85,
    strictMode: false,
    logViolations: true,
  },
});

llm is the backend every check runs on, and config.model the model name sent to it as is (default gpt-6-luna). Fields of config you leave out take DEFAULT_GUARDRAIL_CONFIG; without a constitution the default one applies.

The Default ConstitutionCopy Anchor Link

Cogitator ships with a comprehensive default constitution containing 16 principles across four categories:

Each principle has a severity (low, medium, high) and the layers it applies to (input, output, tool; a principle without appliesTo applies to all three). In the default constitution most principles apply to outputs only: child-safety also covers inputs, and dangerous-tools covers tool calls. DEFAULT_PRINCIPLES holds the list, and filterPrinciplesByLayer, getPrinciplesByCategory and getPrinciplesBySeverity select from a constitution.

Input FilteringCopy Anchor Link

Input filtering blocks harmful prompts before they reach the agent:

const result = await guardrails.filterInput(
  "How do I hack into my neighbor's wifi?",
  'user request'
);

if (!result.allowed) {
  console.log('Blocked:', result.blockedReason);
  console.log('Harm scores:', result.harmScores);
} else {
  // safe to process
}

The input filter first runs a fast pattern scan (bomb-making, "how to kill", "how to hack into", suicide methods, child sexual abuse material) that blocks without a model call. Otherwise an LLM rates the input against the principles that apply to inputs; with none, the input passes. Each rating has a harm category and a severity, and it blocks when its severity reaches the category's threshold — so 'low' is the strictest setting, and a category without a threshold blocks only at 'high':

const guardrails = new ConstitutionalAI({
  llm: backend,
  config: {
    thresholds: {
      violence: 'medium',
      hate: 'low', // strict on hate speech
      sexual: 'medium',
      'self-harm': 'low', // strict on self-harm
      illegal: 'low',
      privacy: 'medium',
      misinformation: 'high', // lenient on misinformation
      manipulation: 'medium',
    },
  },
});

Output FilteringCopy Anchor Link

Output filtering evaluates agent responses before they reach the user:

const outputResult = await guardrails.filterOutput(agentResponse, conversationHistory);

if (!outputResult.allowed && outputResult.suggestedRevision) {
  // the critique-revise loop produced a safe alternative
  const safeResponse = outputResult.suggestedRevision;
}

Outputs get a pattern scan for destructive commands (rm -rf /, format c:, sudo rm -rf) and an LLM rating against the output principles. When enableCritiqueRevision is true and the output fails, the critique-revise loop rewrites it; the result stays allowed: false and carries the rewrite in suggestedRevision.

Critique-Revise LoopCopy Anchor Link

The CritiqueReviser iteratively improves unsafe outputs:

const revision = await guardrails.critiqueAndRevise(unsafeResponse, context);

console.log(`Original: ${revision.original}`);
console.log(`Revised: ${revision.revised}`);
console.log(`Iterations: ${revision.iterations}`);

for (const critique of revision.critiqueHistory) {
  console.log(`Harmful: ${critique.isHarmful}`);
  console.log(`Violated: ${critique.principlesViolated}`);
}

The first iteration critiques the response against the high-severity output principles; later ones pick principles whose harm categories match keywords in the current text (or a random handful). A critique that finds harm leads to a revision for the violated principles. The loop stops when a critique finds no harm, when the highest harm confidence is below revisionConfidenceThreshold, when no known principle was violated, or after maxRevisionIterations.

Tool GuardsCopy Anchor Link

Tool guards evaluate tool calls for safety before execution:

const guardResult = await guardrails.guardTool(
  execTool,
  { command: 'rm -rf /tmp/cache' },
  { agentId: 'cleanup-agent', runId: 'run_1', signal: AbortSignal.timeout(30_000) }
);

if (!guardResult.approved) {
  console.log('Tool blocked:', guardResult.reason);
  console.log('Risk level:', guardResult.riskLevel);
} else if (guardResult.requiresConfirmation) {
  // prompt user for approval
  console.log('Side effects:', guardResult.sideEffects);
}

The ToolGuard refuses dangerous commands (rm -rf /, mkfs., dd of=/dev/..., chmod -R 777 /, format c:) in the command/cmd argument of a tool named exec or declaring the process side effect, and sensitive paths (/etc/shadow, /etc/sudoers, /boot/, /proc/, ~/.ssh/, /root/) in the path/file argument of a tool whose name contains file or that declares filesystem. riskLevel follows the declared sideEffects.

A call needs approval when the tool's requiresApproval says so (a requiresApproval function that throws counts as yes), and in strictMode also when the tool declares any side effect. The guard fails closed: such a call runs only when config.onToolApproval returns true or the caller passes { approvedByUser: true } as the fourth argument of guardTool(). Without either it is refused with Tool "<name>" needs approval, and no approval handler is configured; a false from the handler gives User denied tool execution.

Inside a Cogitator, these calls go through the run's approval flow before they reach the guard: the run's onApproval decides, else guardrails.onToolApproval, and without either the run pauses until you resume it. requiresApproval is enforced this way whether or not guardrails are on; strictMode side effects are added while guardrails and filterToolCalls are on. A call approved in the flow is not asked about again by the guard.

Tool ResultsCopy Anchor Link

With filterToolResults: true, what a tool returns is checked with the input filter before the model reads it, so content fetched from the web or a file cannot smuggle harmful instructions into the run. A blocked result reaches the model as the error Tool result blocked: <reason>.

Guardrails in a CogitatorCopy Anchor Link

new Cogitator({ guardrails: { ... } }) turns guardrails on unless enabled: false; fields you leave out take DEFAULT_GUARDRAIL_CONFIG, so guardrails: { filterInput: false } keeps output and tool filtering. The checks run on guardrails.model (a provider/model id), or on the running agent's model when it is not set.

A blocked input fails the run with Input blocked: <reason> before the first model call. When the output filter blocks an answer and the critique-revise loop produces a safe one, the run answers with the revision; without a revision it fails with Output blocked: <reason>. Both failures are a CogitatorError with code LLM_CONTENT_FILTERED and the harm scores in details.harmScores. A blocked tool call or tool result reaches the model as the tool's error. Every block is logged and reported to onViolation, also when a revision replaces the answer.

const cog = new Cogitator({
  guardrails: { constitution: DEFAULT_CONSTITUTION, filterToolResults: true },
});

await cog.run(agent, { input });

cog.getGuardrails()?.getViolationLog();
cog.setConstitution(customConstitution); // see Custom Constitutions below

The ConstitutionalAI instance is built on first use. With guardrails.model or llm.defaultModel set, getGuardrails() and setConstitution() work before the first run; otherwise getGuardrails() returns undefined until a run builds the instance with its agent's model. A constitution passed to setConstitution() before that is applied as soon as the instance exists, and it stays in force after close(). Without guardrails in the config (or with enabled: false), setConstitution() only logs a warning.

const cog = new Cogitator({
  llm: { defaultModel: 'openai/gpt-5.5' },
  guardrails: { model: 'openai/gpt-6-luna' },
});

cog.setConstitution(customConstitution);
cog.getGuardrails()?.getConstitution();

Custom ConstitutionsCopy Anchor Link

Build a custom constitution from scratch or extend the default:

import { createConstitution, extendConstitution, DEFAULT_CONSTITUTION } from '@cogitator-ai/core';

const customConstitution = createConstitution([
  {
    id: 'no-competitor-recommendations',
    name: 'No Competitor Recommendations',
    description: 'Never recommend competitor products',
    category: 'ethics',
    critiquePrompt: 'Does this response recommend or promote competitor products?',
    revisionPrompt: 'Rewrite to focus on our products or give generic advice',
    harmCategories: ['manipulation'],
    severity: 'medium',
    appliesTo: ['output'],
  },
]);

const extended = extendConstitution(DEFAULT_CONSTITUTION, customConstitution.principles);
guardrails.setConstitution(extended);

You can also add or remove principles at runtime:

guardrails.addPrinciple({
  id: 'data-retention',
  name: 'Data Retention Policy',
  description: 'Never store user data beyond the current session',
  category: 'privacy',
  critiquePrompt: 'Does this response suggest storing or persisting user data?',
  revisionPrompt: 'Remove references to data storage and clarify the session-only policy',
  harmCategories: ['privacy'],
  severity: 'high',
  appliesTo: ['output', 'tool'],
});

guardrails.removePrinciple('no-competitor-recommendations');

Violation LoggingCopy Anchor Link

All violations are logged internally and accessible for auditing:

const violations = guardrails.getViolationLog();

for (const entry of violations) {
  console.log(`[${entry.timestamp.toISOString()}] Layer: ${entry.layer}`);
  console.log(`  Allowed: ${entry.result.allowed}`);
  console.log(`  Categories: ${entry.result.harmScores.map((s) => s.category).join(', ')}`);
}

guardrails.clearViolationLog();

Entries are kept while logViolations is on (the default). config.onViolation(result, layer) is called for every violation either way, for real-time handling.