# OpenAI Compatibility

URL: https://cogitator.app/docs/integrations/openai-compat

> Drop-in OpenAI Assistants API compatibility layer that lets existing OpenAI SDK clients interact with Cogitator agents.

## Overview

The `@cogitator-ai/openai-compat` package exposes Cogitator as an OpenAI-compatible API server. Any application built on the OpenAI SDK can point at your Cogitator instance and work without code changes -- assistants, threads, messages, and runs all map to Cogitator concepts under the hood.

```bash
pnpm add @cogitator-ai/openai-compat @cogitator-ai/core
```

## Quick Start

Spin up a compatibility server in a few lines:

```typescript
import { Cogitator } from '@cogitator-ai/core';
import { createOpenAIServer } from '@cogitator-ai/openai-compat';

const cog = new Cogitator({
  llm: {
    defaultProvider: 'openai',
    providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  },
});

const server = createOpenAIServer(cog, {
  port: 8080,
  apiKeys: ['sk-my-secret-key'],
  tools: [calculator, webSearch],
});

await server.start();
```

Now use the standard OpenAI SDK against your local server:

```typescript
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'sk-my-secret-key',
  baseURL: 'http://localhost:8080/v1',
});

const assistant = await openai.beta.assistants.create({
  model: 'gpt-6.1-sol',
  name: 'My Assistant',
  instructions: 'You are a helpful assistant with access to tools.',
});

const thread = await openai.beta.threads.create();

await openai.beta.threads.messages.create(thread.id, {
  role: 'user',
  content: 'What is the square root of 144?',
});

const run = await openai.beta.threads.runs.createAndPoll(thread.id, {
  assistant_id: assistant.id,
});

const messages = await openai.beta.threads.messages.list(thread.id); // newest first
```

Assistant `model` values are Cogitator model strings (`gpt-6.1-sol` uses the default provider, `ollama/llama3.2` selects one explicitly). The `cogitator` id listed by `GET /v1/models` maps to the server's `defaultModel`.

## OpenAIServer

The server is built on Fastify and implements the full Assistants API surface:

```typescript
const server = new OpenAIServer(cogitator, config);
await server.start();

console.log(server.getBaseUrl());
console.log(server.isRunning());

await server.stop();
```

### OpenAIServerConfig

| Option         | Type                  | Default            | Description                                 |
| -------------- | --------------------- | ------------------ | ------------------------------------------- |
| `port`         | `number`              | `8080`             | Port to listen on                           |
| `host`         | `string`              | `'0.0.0.0'`        | Host to bind to                             |
| `apiKeys`      | `string[]`            | `[]`               | API keys for auth (empty = no auth)         |
| `tools`        | `Tool[]`              | `[]`               | Cogitator tools available to all assistants |
| `defaultModel` | `string`              | —                  | Model used for the `cogitator` model id     |
| `storage`      | `ThreadStorage`       | in-memory          | Persistence backend (connect it first)      |
| `maxFileSize`  | `number`              | 512 MB             | Upload limit for `POST /v1/files`           |
| `logging`      | `boolean`             | `false`            | Enable request logging via pino (JSON)      |
| `cors`         | `{ origin, methods }` | `{ origin: true }` | CORS configuration                          |

`start()` waits for plugin registration, so it can be called right after construction. With `port: 0` the OS picks a port and `getUrl()` / `getBaseUrl()` report the bound one. `/health` stays public when `apiKeys` are set.

### Authentication and Errors

With `apiKeys`, every request except `GET /health` needs `Authorization: Bearer <key>`; a missing or unknown key answers `401` with code `invalid_api_key`. Keys only gate access: every key sees the same assistants, threads, runs and files, and runs carry no `userId`, so give each tenant its own server (or storage) when their data must stay apart.

Errors use OpenAI's shape, `{ "error": { "message", "type", "param", "code" } }`, with `invalid_request` (400), `invalid_api_key` (401), `permission_denied` (403), `not_found` (404), `rate_limit_exceeded` (429) and otherwise `internal_error`. Every 5xx answer says `Internal server error` and the details are only logged. Requests the adapter refuses throw `InvalidRequestError` (exported from the package), which the server answers as `400 invalid_request`. A run that fails ends with `status: 'failed'` and `last_error: { code: 'server_error', message }`, where `message` is `Internal server error` unless the cause is a `CogitatorError` or an `InvalidRequestError`.

## Supported Endpoints

The server implements these OpenAI API routes:

| Method   | Endpoint                                        | Description                                         |
| -------- | ----------------------------------------------- | --------------------------------------------------- |
| `GET`    | `/v1/models`                                    | List available models                               |
| `POST`   | `/v1/assistants`                                | Create assistant                                    |
| `GET`    | `/v1/assistants`                                | List assistants                                     |
| `GET`    | `/v1/assistants/:id`                            | Get assistant                                       |
| `POST`   | `/v1/assistants/:id`                            | Update assistant                                    |
| `DELETE` | `/v1/assistants/:id`                            | Delete assistant                                    |
| `POST`   | `/v1/threads`                                   | Create thread                                       |
| `POST`   | `/v1/threads/runs`                              | Create thread and run                               |
| `GET`    | `/v1/threads/:id`                               | Get thread                                          |
| `POST`   | `/v1/threads/:id`                               | Update thread metadata                              |
| `DELETE` | `/v1/threads/:id`                               | Delete thread                                       |
| `POST`   | `/v1/threads/:id/messages`                      | Add message                                         |
| `GET`    | `/v1/threads/:id/messages`                      | List messages                                       |
| `GET`    | `/v1/threads/:id/messages/:mid`                 | Get message                                         |
| `POST`   | `/v1/threads/:id/runs`                          | Create run                                          |
| `GET`    | `/v1/threads/:id/runs`                          | List runs                                           |
| `GET`    | `/v1/threads/:id/runs/:rid`                     | Get run status                                      |
| `POST`   | `/v1/threads/:id/runs/:rid/cancel`              | Cancel run                                          |
| `POST`   | `/v1/threads/:id/runs/:rid/submit_tool_outputs` | Submit tool outputs                                 |
| `POST`   | `/v1/files`                                     | Upload file                                         |
| `GET`    | `/v1/files`                                     | List files (`?purpose=`, `limit`, `order`, `after`) |
| `GET`    | `/v1/files/:id`                                 | Get file metadata                                   |
| `GET`    | `/v1/files/:id/content`                         | Download file                                       |
| `DELETE` | `/v1/files/:id`                                 | Delete file                                         |

`GET /v1/files` takes `limit` (1-10 000, default 10 000), `order` (`asc` / `desc`, default `desc`) and `after`.

## OpenAIAdapter

For in-process usage without a network server, use `OpenAIAdapter` directly. It provides the same Assistants API as programmatic methods:

```typescript
import { createOpenAIAdapter } from '@cogitator-ai/openai-compat';

const adapter = createOpenAIAdapter(cog, { tools: [calculator] });

const assistant = await adapter.createAssistant({
  model: 'gpt-6.1-sol',
  name: 'Math Helper',
  instructions: 'Help with math problems.',
});

const thread = await adapter.createThread();

await adapter.addMessage(thread.id, {
  role: 'user',
  content: 'What is 15! (factorial)?',
});

const run = await adapter.createRun(thread.id, {
  assistant_id: assistant.id,
});
```

Runs see the whole thread (earlier messages are replayed to the agent), are aborted on cancel, and honour `additional_instructions`, `response_format`, `max_completion_tokens`, `top_p`, `tool_choice` (`none` or a specific function), `parallel_tool_calls` and `truncation_strategy`. `max_prompt_tokens` is best-effort (about 4 characters per token, instructions included): the oldest replayed messages are dropped until the prompt fits, and a run whose last user message alone does not fit ends `incomplete` with `incomplete_details.reason: 'max_prompt_tokens'`. A thread can have one active run at a time (a second `createRun` returns HTTP 400), and cancelling a run that has already finished returns HTTP 400. Messages are listed in insertion order (`order: 'desc'`, the default, returns the newest first). Image parts are passed to the model.

### Function Calling

Assistant tools of type `function` are executed by the API client. When the model calls one, the run moves to `requires_action`; submit the outputs to resume (runs expire after 10 minutes without outputs). Server-side `tools` run inside Cogitator and do not need to be declared on the assistant.

```typescript
let run = await openai.beta.threads.runs.createAndPoll(thread.id, { assistant_id });

if (run.status === 'requires_action') {
  const calls = run.required_action!.submit_tool_outputs.tool_calls;
  run = await openai.beta.threads.runs.submitToolOutputsAndPoll(run.id, {
    thread_id: thread.id,
    tool_outputs: calls.map((call) => ({
      tool_call_id: call.id,
      output: JSON.stringify(myFunctions[call.function.name](JSON.parse(call.function.arguments))),
    })),
  });
}
```

### Streaming Runs

Pass `stream: true` to stream OpenAI-style events (`thread.run.*`, `thread.message.created|delta|completed`, `done`). With the SDK use `runs.stream(...)` / `submitToolOutputsStream(...)`. In-process, iterate the run's event log:

```typescript
const run = await adapter.createRun(thread.id, {
  assistant_id: assistant.id,
  stream: true,
});

for await (const { event, data } of adapter.streamRunEvents(run.id)) {
  if (event === 'thread.message.delta' && typeof data !== 'string' && 'delta' in data) {
    for (const part of data.delta.content ?? []) {
      if (part.type === 'text') process.stdout.write(part.text?.value ?? '');
    }
  }
}
```

A stream ends with `done` when the run finishes or pauses on `requires_action`.

## Thread Storage

By default, threads and messages are stored in memory. For production, plug in Redis or PostgreSQL:

```typescript
import { RedisThreadStorage, createOpenAIServer } from '@cogitator-ai/openai-compat';

const storage = new RedisThreadStorage({
  url: 'redis://localhost:6379',
  keyPrefix: 'cogitator:',
  ttl: 86400,
});
await storage.connect();

const server = createOpenAIServer(cog, { storage, defaultModel: 'openai/gpt-6.1-sol' });
// or in-process: createOpenAIAdapter(cog, { storage })
```

Storage is the single source of truth (no per-process cache), so several server instances can share it. Runs are not stored: they live in the process that created them, so with several instances route each thread to the same one (sticky routing). `ioredis` and `pg` are optional peer dependencies loaded on `connect()`; Redis `ttl: 0` disables expiry.

Or use the factory function:

```typescript
import { createThreadStorage } from '@cogitator-ai/openai-compat';

const storage = createThreadStorage({
  type: 'postgres',
  connectionString: 'postgresql://user:pass@localhost/mydb',
  tableName: 'assistant_threads',
});
await storage.connect?.();
```

### Storage Options

| Backend    | Package Required | Config                                   |
| ---------- | ---------------- | ---------------------------------------- |
| In-memory  | (none)           | `{ type: 'memory' }`                     |
| Redis      | `ioredis`        | `{ type: 'redis', url, keyPrefix, ttl }` |
| PostgreSQL | `pg`             | `{ type: 'postgres', connectionString }` |

## Full Example: Migration from OpenAI

If you have an existing OpenAI-powered app, migration is straightforward. Just change the `baseURL`:

```typescript
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.COGITATOR_API_KEY,
  baseURL: process.env.COGITATOR_URL + '/v1',
});

const thread = await openai.beta.threads.create();

await openai.beta.threads.messages.create(thread.id, {
  role: 'user',
  content: 'Analyze the sentiment of this review: "Absolutely loved this product!"',
});

const run = await openai.beta.threads.runs.createAndPoll(thread.id, {
  assistant_id: 'asst_abc123',
});

if (run.status === 'completed') {
  const messages = await openai.beta.threads.messages.list(thread.id);
  console.log(messages.data[0].content[0].text.value);
}
```