Structured Outputs

OverviewCopy Anchor Link

Structured outputs let you constrain the LLM to return data in a specific JSON shape. Cogitator supports three response format modes and handles the provider-specific differences for you.

import { Agent } from '@cogitator-ai/core';
import { z } from 'zod';

const agent = new Agent({
  name: 'extractor',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Extract structured data from the given text.',
  responseFormat: {
    type: 'json_schema',
    schema: z.object({
      title: z.string(),
      topics: z.array(z.string()),
      sentiment: z.enum(['positive', 'negative', 'neutral']),
    }),
  },
});

Response Format ModesCopy Anchor Link

text (default)Copy Anchor Link

The LLM responds with free-form text. This is the default when no responseFormat is set, and the same as responseFormat: { type: 'text' }. result.structured stays undefined.

jsonCopy Anchor Link

The LLM is instructed to return valid JSON, but without a specific schema. Useful when you need flexible JSON output.

const agent = new Agent({
  name: 'json-agent',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Respond with a JSON object containing your analysis.',
  responseFormat: { type: 'json' },
});

const result = await cog.run(agent, { input: 'Analyze this sentence: "I love TypeScript"' });
const data = result.structured; // the parsed JSON, or undefined if the answer was not JSON

json_schemaCopy Anchor Link

The LLM is constrained to a specific JSON structure defined by a Zod schema. This is the most powerful mode -- providers with native structured outputs enforce the schema, and every answer is validated against it (see Reading the Result).

const sentimentSchema = z.object({
  text: z.string(),
  sentiment: z.enum(['positive', 'negative', 'neutral']),
  confidence: z.number(),
  keywords: z.array(z.string()),
  entities: z.array(
    z.object({
      name: z.string(),
      type: z.enum(['person', 'organization', 'location', 'other']),
    })
  ),
});

const agent = new Agent({
  name: 'sentiment-analyzer',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Analyze the sentiment and extract entities from the input text.',
  responseFormat: {
    type: 'json_schema',
    schema: sentimentSchema,
  },
});

How Providers Handle ItCopy Anchor Link

Each LLM provider implements structured outputs differently. Cogitator abstracts these differences away, but understanding them helps pick the right provider.

OpenAICopy Anchor Link

OpenAI natively supports all three modes. The official OpenAI backend uses the Responses API, where json_schema is sent as text.format with strict schema enforcement:

// what Cogitator sends under the hood (Responses API)
text: {
  format: {
    type: 'json_schema',
    name: 'response',
    schema: { /* your Zod schema converted to JSON Schema */ },
    strict: true, // when the schema allows it, see below
  },
}

OpenAI-compatible providers (and requests with stop sequences) use Chat Completions, where the same schema goes into response_format: { type: 'json_schema', json_schema: { ... } }.

When the agent also has tools, OpenAI-compatible servers other than OpenAI and Azure OpenAI (OpenRouter, DeepSeek, Groq, Together, Mistral, vLLM, a custom OpenAIBackend with a baseUrl) get the schema as an instruction in the system prompt instead of response_format. Many of them apply response_format from the first turn, so the model answers in JSON without calling its tools, or ignore the schema altogether, while the same models follow a schema in the prompt reliably. Requests without tools keep response_format.

AnthropicCopy Anchor Link

• json mode: Appends "respond with valid JSON only" to the system prompt
• json_schema mode on Claude 4.5+ and Fable: uses native structured outputs (output_config.format). The schema is adapted to the subset Claude accepts: every object gets additionalProperties: false, oneOf becomes anyOf, and unsupported constraints (numeric bounds, string lengths, maxItems, unknown formats) are moved into the description so the model still sees them
• json_schema mode on older models: Cogitator uses a tool trick -- it creates a synthetic tool named __json_response with your schema as its parameters and forces the LLM to call it. The output is extracted from the tool call arguments. When the agent has tools of its own, forcing that call would keep it from using them, so the schema goes into the system prompt instead

Either way you get the same structured response regardless of provider.

AWS BedrockCopy Anchor Link

• json mode: Appends "respond with valid JSON only" to the system prompt
• json_schema mode: enforced with Converse outputConfig.textFormat on Claude 4.5 – 4.6; other models get a system-prompt instruction containing the schema

Google (Gemini)Copy Anchor Link

Google supports structured outputs via responseMimeType and responseSchema:

• json mode: Sets responseMimeType: 'application/json'
• json_schema mode: Sets both responseMimeType and responseSchema (keywords Gemini does not accept are dropped). Gemini has no null type, so nullable fields (.nullable() in Zod, a null in type, anyOf or oneOf) are sent as nullable: true, in tool parameter schemas too
• Gemini 1.x and 2.x refuse a JSON mime type together with function calling, so for an agent with tools the schema goes into the system instruction instead. Gemini 3 takes both in one request

OllamaCopy Anchor Link

Ollama supports structured outputs via the format field:

• json mode: Sets format: 'json'
• json_schema mode: Passes the full JSON Schema as the format value

Azure OpenAICopy Anchor Link

Azure uses the same format as OpenAI since it's backed by the same API.

Defining Schemas with ZodCopy Anchor Link

Use the full power of Zod to define your output shapes:

const analysisSchema = z.object({
  summary: z.string().describe('A brief summary of the content'),

  categories: z
    .array(z.enum(['tech', 'business', 'science', 'politics', 'entertainment']))
    .describe('Relevant categories'),

  metrics: z.object({
    readability: z.number().min(0).max(100),
    complexity: z.enum(['simple', 'moderate', 'complex']),
    wordCount: z.number().int().positive(),
  }),

  references: z
    .array(
      z.object({
        title: z.string(),
        url: z.string().url().optional(),
        relevance: z.number().min(0).max(1),
      })
    )
    .optional(),
});

Field descriptions (.describe()) are included in the JSON Schema sent to the LLM, which improves output quality. A .describe() on the root schema becomes the schema's description, and names the schema in a serialized agent snapshot.

Reading the ResultCopy Anchor Link

result.output is always the raw text of the final answer. For json and json_schema agents, result.structured holds the parsed value: the JSON (a ```json fence around it is tolerated), run through your Zod schema in json_schema mode, so transforms and defaults apply.

const result = await cog.run(agent, { input: 'Analyze this article...' });
const analysis = result.structured as z.infer<typeof analysisSchema> | undefined;

JSON wrapped in prose or a code fence is still read. When the final answer is not valid JSON or does not match the schema, the run asks the model once more, telling it what was wrong (for example celsius: expected number, received string); the rejected answer is not saved to the thread. Streamed runs skip that second attempt, since the client has already seen the first answer.

A turn that ends with neither text nor tool calls (some models, Gemini in particular, occasionally stop empty after a tool result) is not treated as an answer: the run asks again, up to twice, and the empty turn never reaches the thread. This applies to every run, with or without a response format, and the retries count towards maxIterations.

structured is undefined when even that answer does not fit, so check it before use: result.structuredError then says why, in the words the model was given (the answer is not valid JSON, celsius: expected number, received string). Agents with tools can call them as usual: the format applies to the final answer.

Strict mode is requested only when the schema allows it: OpenAI rejects strict schemas with optional fields, so a schema with .optional() properties is sent with strict: false.

Common PatternsCopy Anchor Link

ClassificationCopy Anchor Link

const classifier = new Agent({
  name: 'classifier',
  model: 'openai/gpt-6-luna',
  instructions: 'Classify the support ticket into a category and priority.',
  temperature: 0,
  responseFormat: {
    type: 'json_schema',
    schema: z.object({
      category: z.enum(['billing', 'technical', 'account', 'feature_request', 'other']),
      priority: z.enum(['low', 'medium', 'high', 'critical']),
      reasoning: z.string(),
    }),
  },
});

Data ExtractionCopy Anchor Link

const extractor = new Agent({
  name: 'extractor',
  model: 'anthropic/claude-sonnet-5-5',
  instructions: 'Extract all contact information from the given text.',
  responseFormat: {
    type: 'json_schema',
    schema: z.object({
      contacts: z.array(
        z.object({
          name: z.string(),
          email: z.string().email().optional(),
          phone: z.string().optional(),
          company: z.string().optional(),
        })
      ),
    }),
  },
});