# Structured Outputs

URL: https://cogitator.app/docs/core/structured-outputs

> Get typed JSON responses from LLMs using Zod schemas and response format modes.

## Overview

Structured outputs let you constrain the LLM to return data in a specific JSON shape. Cogitator supports three response format modes and handles the provider-specific differences for you.

```typescript
import { Agent } from '@cogitator-ai/core';
import { z } from 'zod';

const agent = new Agent({
  name: 'extractor',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Extract structured data from the given text.',
  responseFormat: {
    type: 'json_schema',
    schema: z.object({
      title: z.string(),
      topics: z.array(z.string()),
      sentiment: z.enum(['positive', 'negative', 'neutral']),
    }),
  },
});
```

## Response Format Modes

### `text` (default)

The LLM responds with free-form text. This is the default when no `responseFormat` is set, and the same as `responseFormat: { type: 'text' }`. `result.structured` stays `undefined`.

### `json`

The LLM is instructed to return valid JSON, but without a specific schema. Useful when you need flexible JSON output.

```typescript
const agent = new Agent({
  name: 'json-agent',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Respond with a JSON object containing your analysis.',
  responseFormat: { type: 'json' },
});

const result = await cog.run(agent, { input: 'Analyze this sentence: "I love TypeScript"' });
const data = result.structured; // the parsed JSON, or undefined if the answer was not JSON
```

### `json_schema`

The LLM is constrained to a specific JSON structure defined by a Zod schema. This is the most powerful mode -- providers with native structured outputs enforce the schema, and every answer is validated against it (see [Reading the Result](#reading-the-result)).

```typescript
const sentimentSchema = z.object({
  text: z.string(),
  sentiment: z.enum(['positive', 'negative', 'neutral']),
  confidence: z.number(),
  keywords: z.array(z.string()),
  entities: z.array(
    z.object({
      name: z.string(),
      type: z.enum(['person', 'organization', 'location', 'other']),
    })
  ),
});

const agent = new Agent({
  name: 'sentiment-analyzer',
  model: 'openai/gpt-6.1-sol',
  instructions: 'Analyze the sentiment and extract entities from the input text.',
  responseFormat: {
    type: 'json_schema',
    schema: sentimentSchema,
  },
});
```

## How Providers Handle It

Each LLM provider implements structured outputs differently. Cogitator abstracts these differences away, but understanding them helps pick the right provider.

### OpenAI

OpenAI natively supports all three modes. The official OpenAI backend uses the Responses API, where `json_schema` is sent as `text.format` with strict schema enforcement:

```typescript
// what Cogitator sends under the hood (Responses API)
text: {
  format: {
    type: 'json_schema',
    name: 'response',
    schema: { /* your Zod schema converted to JSON Schema */ },
    strict: true, // when the schema allows it, see below
  },
}
```

OpenAI-compatible providers (and requests with stop sequences) use Chat Completions, where the same schema goes into `response_format: { type: 'json_schema', json_schema: { ... } }`.

When the agent also has tools, OpenAI-compatible servers other than OpenAI and Azure OpenAI (OpenRouter, DeepSeek, Groq, Together, Mistral, vLLM, a custom `OpenAIBackend` with a `baseUrl`) get the schema as an instruction in the system prompt instead of `response_format`. Many of them apply `response_format` from the first turn, so the model answers in JSON without calling its tools, or ignore the schema altogether, while the same models follow a schema in the prompt reliably. Requests without tools keep `response_format`.

### Anthropic

* `json` mode: Appends "respond with valid JSON only" to the system prompt
* `json_schema` mode on Claude 4.5+ and Fable: uses **native structured outputs** (`output_config.format`). The schema is adapted to the subset Claude accepts: every object gets `additionalProperties: false`, `oneOf` becomes `anyOf`, and unsupported constraints (numeric bounds, string lengths, `maxItems`, unknown formats) are moved into the description so the model still sees them
* `json_schema` mode on older models: Cogitator uses a **tool trick** -- it creates a synthetic tool named `__json_response` with your schema as its parameters and forces the LLM to call it. The output is extracted from the tool call arguments. When the agent has tools of its own, forcing that call would keep it from using them, so the schema goes into the system prompt instead

Either way you get the same structured response regardless of provider.

### AWS Bedrock

* `json` mode: Appends "respond with valid JSON only" to the system prompt
* `json_schema` mode: enforced with Converse `outputConfig.textFormat` on Claude 4.5 – 4.6; other models get a system-prompt instruction containing the schema

### Google (Gemini)

Google supports structured outputs via `responseMimeType` and `responseSchema`:

* `json` mode: Sets `responseMimeType: 'application/json'`
* `json_schema` mode: Sets both `responseMimeType` and `responseSchema` (keywords Gemini does not accept are dropped). Gemini has no `null` type, so nullable fields (`.nullable()` in Zod, a `null` in `type`, `anyOf` or `oneOf`) are sent as `nullable: true`, in tool parameter schemas too
* Gemini 1.x and 2.x refuse a JSON mime type together with function calling, so for an agent with tools the schema goes into the system instruction instead. Gemini 3 takes both in one request

### Ollama

Ollama supports structured outputs via the `format` field:

* `json` mode: Sets `format: 'json'`
* `json_schema` mode: Passes the full JSON Schema as the `format` value

### Azure OpenAI

Azure uses the same format as OpenAI since it's backed by the same API.

## Defining Schemas with Zod

Use the full power of Zod to define your output shapes:

```typescript
const analysisSchema = z.object({
  summary: z.string().describe('A brief summary of the content'),

  categories: z
    .array(z.enum(['tech', 'business', 'science', 'politics', 'entertainment']))
    .describe('Relevant categories'),

  metrics: z.object({
    readability: z.number().min(0).max(100),
    complexity: z.enum(['simple', 'moderate', 'complex']),
    wordCount: z.number().int().positive(),
  }),

  references: z
    .array(
      z.object({
        title: z.string(),
        url: z.string().url().optional(),
        relevance: z.number().min(0).max(1),
      })
    )
    .optional(),
});
```

Field descriptions (`.describe()`) are included in the JSON Schema sent to the LLM, which improves output quality. A `.describe()` on the root schema becomes the schema's description, and names the schema in a [serialized](/docs/core/agents#serialization) agent snapshot.

## Reading the Result

`result.output` is always the raw text of the final answer. For `json` and `json_schema` agents, `result.structured` holds the parsed value: the JSON (a ` ```json ` fence around it is tolerated), run through your Zod schema in `json_schema` mode, so transforms and defaults apply.

```typescript
const result = await cog.run(agent, { input: 'Analyze this article...' });
const analysis = result.structured as z.infer<typeof analysisSchema> | undefined;
```

JSON wrapped in prose or a code fence is still read. When the final answer is not valid JSON or does not match the schema, the run asks the model once more, telling it what was wrong (for example `celsius: expected number, received string`); the rejected answer is not saved to the thread. Streamed runs skip that second attempt, since the client has already seen the first answer.

A turn that ends with neither text nor tool calls (some models, Gemini in particular, occasionally stop empty after a tool result) is not treated as an answer: the run asks again, up to twice, and the empty turn never reaches the thread. This applies to every run, with or without a response format, and the retries count towards `maxIterations`.

`structured` is `undefined` when even that answer does not fit, so check it before use: `result.structuredError` then says why, in the words the model was given (`the answer is not valid JSON`, `celsius: expected number, received string`). Agents with tools can call them as usual: the format applies to the final answer.

Strict mode is requested only when the schema allows it: OpenAI rejects strict schemas with optional fields, so a schema with `.optional()` properties is sent with `strict: false`.

## Common Patterns

### Classification

```typescript
const classifier = new Agent({
  name: 'classifier',
  model: 'openai/gpt-6-luna',
  instructions: 'Classify the support ticket into a category and priority.',
  temperature: 0,
  responseFormat: {
    type: 'json_schema',
    schema: z.object({
      category: z.enum(['billing', 'technical', 'account', 'feature_request', 'other']),
      priority: z.enum(['low', 'medium', 'high', 'critical']),
      reasoning: z.string(),
    }),
  },
});
```

### Data Extraction

```typescript
const extractor = new Agent({
  name: 'extractor',
  model: 'anthropic/claude-sonnet-5-5',
  instructions: 'Extract all contact information from the given text.',
  responseFormat: {
    type: 'json_schema',
    schema: z.object({
      contacts: z.array(
        z.object({
          name: z.string(),
          email: z.string().email().optional(),
          phone: z.string().optional(),
          company: z.string().optional(),
        })
      ),
    }),
  },
});
```