OpenAI Compatibility

OverviewCopy Anchor Link

The @cogitator-ai/openai-compat package exposes Cogitator as an OpenAI-compatible API server. Any application built on the OpenAI SDK can point at your Cogitator instance and work without code changes -- assistants, threads, messages, and runs all map to Cogitator concepts under the hood.

pnpm add @cogitator-ai/openai-compat @cogitator-ai/core

Quick StartCopy Anchor Link

Spin up a compatibility server in a few lines:

import { Cogitator } from '@cogitator-ai/core';
import { createOpenAIServer } from '@cogitator-ai/openai-compat';

const cog = new Cogitator({
  llm: {
    defaultProvider: 'openai',
    providers: { openai: { apiKey: process.env.OPENAI_API_KEY! } },
  },
});

const server = createOpenAIServer(cog, {
  port: 8080,
  apiKeys: ['sk-my-secret-key'],
  tools: [calculator, webSearch],
});

await server.start();

Now use the standard OpenAI SDK against your local server:

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: 'sk-my-secret-key',
  baseURL: 'http://localhost:8080/v1',
});

const assistant = await openai.beta.assistants.create({
  model: 'gpt-6.1-sol',
  name: 'My Assistant',
  instructions: 'You are a helpful assistant with access to tools.',
});

const thread = await openai.beta.threads.create();

await openai.beta.threads.messages.create(thread.id, {
  role: 'user',
  content: 'What is the square root of 144?',
});

const run = await openai.beta.threads.runs.createAndPoll(thread.id, {
  assistant_id: assistant.id,
});

const messages = await openai.beta.threads.messages.list(thread.id); // newest first

Assistant model values are Cogitator model strings (gpt-6.1-sol uses the default provider, ollama/llama3.2 selects one explicitly). The cogitator id listed by GET /v1/models maps to the server's defaultModel.

OpenAIServerCopy Anchor Link

The server is built on Fastify and implements the full Assistants API surface:

const server = new OpenAIServer(cogitator, config);
await server.start();

console.log(server.getBaseUrl());
console.log(server.isRunning());

await server.stop();

OpenAIServerConfigCopy Anchor Link

start() waits for plugin registration, so it can be called right after construction. With port: 0 the OS picks a port and getUrl() / getBaseUrl() report the bound one. /health stays public when apiKeys are set.

Authentication and ErrorsCopy Anchor Link

With apiKeys, every request except GET /health needs Authorization: Bearer <key>; a missing or unknown key answers 401 with code invalid_api_key. Keys only gate access: every key sees the same assistants, threads, runs and files, and runs carry no userId, so give each tenant its own server (or storage) when their data must stay apart.

Errors use OpenAI's shape, { "error": { "message", "type", "param", "code" } }, with invalid_request (400), invalid_api_key (401), permission_denied (403), not_found (404), rate_limit_exceeded (429) and otherwise internal_error. Every 5xx answer says Internal server error and the details are only logged. Requests the adapter refuses throw InvalidRequestError (exported from the package), which the server answers as 400 invalid_request. A run that fails ends with status: 'failed' and last_error: { code: 'server_error', message }, where message is Internal server error unless the cause is a CogitatorError or an InvalidRequestError.

Supported EndpointsCopy Anchor Link

The server implements these OpenAI API routes:

GET /v1/files takes limit (1-10 000, default 10 000), order (asc / desc, default desc) and after.

OpenAIAdapterCopy Anchor Link

For in-process usage without a network server, use OpenAIAdapter directly. It provides the same Assistants API as programmatic methods:

import { createOpenAIAdapter } from '@cogitator-ai/openai-compat';

const adapter = createOpenAIAdapter(cog, { tools: [calculator] });

const assistant = await adapter.createAssistant({
  model: 'gpt-6.1-sol',
  name: 'Math Helper',
  instructions: 'Help with math problems.',
});

const thread = await adapter.createThread();

await adapter.addMessage(thread.id, {
  role: 'user',
  content: 'What is 15! (factorial)?',
});

const run = await adapter.createRun(thread.id, {
  assistant_id: assistant.id,
});

Runs see the whole thread (earlier messages are replayed to the agent), are aborted on cancel, and honour additional_instructions, response_format, max_completion_tokens, top_p, tool_choice (none or a specific function), parallel_tool_calls and truncation_strategy. max_prompt_tokens is best-effort (about 4 characters per token, instructions included): the oldest replayed messages are dropped until the prompt fits, and a run whose last user message alone does not fit ends incomplete with incomplete_details.reason: 'max_prompt_tokens'. A thread can have one active run at a time (a second createRun returns HTTP 400), and cancelling a run that has already finished returns HTTP 400. Messages are listed in insertion order (order: 'desc', the default, returns the newest first). Image parts are passed to the model.

Function CallingCopy Anchor Link

Assistant tools of type function are executed by the API client. When the model calls one, the run moves to requires_action; submit the outputs to resume (runs expire after 10 minutes without outputs). Server-side tools run inside Cogitator and do not need to be declared on the assistant.

let run = await openai.beta.threads.runs.createAndPoll(thread.id, { assistant_id });

if (run.status === 'requires_action') {
  const calls = run.required_action!.submit_tool_outputs.tool_calls;
  run = await openai.beta.threads.runs.submitToolOutputsAndPoll(run.id, {
    thread_id: thread.id,
    tool_outputs: calls.map((call) => ({
      tool_call_id: call.id,
      output: JSON.stringify(myFunctions[call.function.name](JSON.parse(call.function.arguments))),
    })),
  });
}

Streaming RunsCopy Anchor Link

Pass stream: true to stream OpenAI-style events (thread.run.*, thread.message.created|delta|completed, done). With the SDK use runs.stream(...) / submitToolOutputsStream(...). In-process, iterate the run's event log:

const run = await adapter.createRun(thread.id, {
  assistant_id: assistant.id,
  stream: true,
});

for await (const { event, data } of adapter.streamRunEvents(run.id)) {
  if (event === 'thread.message.delta' && typeof data !== 'string' && 'delta' in data) {
    for (const part of data.delta.content ?? []) {
      if (part.type === 'text') process.stdout.write(part.text?.value ?? '');
    }
  }
}

A stream ends with done when the run finishes or pauses on requires_action.

Thread StorageCopy Anchor Link

By default, threads and messages are stored in memory. For production, plug in Redis or PostgreSQL:

import { RedisThreadStorage, createOpenAIServer } from '@cogitator-ai/openai-compat';

const storage = new RedisThreadStorage({
  url: 'redis://localhost:6379',
  keyPrefix: 'cogitator:',
  ttl: 86400,
});
await storage.connect();

const server = createOpenAIServer(cog, { storage, defaultModel: 'openai/gpt-6.1-sol' });
// or in-process: createOpenAIAdapter(cog, { storage })

Storage is the single source of truth (no per-process cache), so several server instances can share it. Runs are not stored: they live in the process that created them, so with several instances route each thread to the same one (sticky routing). ioredis and pg are optional peer dependencies loaded on connect(); Redis ttl: 0 disables expiry.

Or use the factory function:

import { createThreadStorage } from '@cogitator-ai/openai-compat';

const storage = createThreadStorage({
  type: 'postgres',
  connectionString: 'postgresql://user:pass@localhost/mydb',
  tableName: 'assistant_threads',
});
await storage.connect?.();

Storage OptionsCopy Anchor Link

Full Example: Migration from OpenAICopy Anchor Link

If you have an existing OpenAI-powered app, migration is straightforward. Just change the baseURL:

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.COGITATOR_API_KEY,
  baseURL: process.env.COGITATOR_URL + '/v1',
});

const thread = await openai.beta.threads.create();

await openai.beta.threads.messages.create(thread.id, {
  role: 'user',
  content: 'Analyze the sentiment of this review: "Absolutely loved this product!"',
});

const run = await openai.beta.threads.runs.createAndPoll(thread.id, {
  assistant_id: 'asst_abc123',
});

if (run.status === 'completed') {
  const messages = await openai.beta.threads.messages.list(thread.id);
  console.log(messages.data[0].content[0].text.value);
}