# STT Providers

URL: https://cogitator.app/docs/voice/stt-providers

> Speech-to-text providers for @cogitator-ai/voice — OpenAI (gpt-transcribe) for batch transcription and Deepgram (nova-3) for real-time streaming with interim results.

## Overview

STT providers convert audio to text. Every provider implements the `STTProvider` interface:

```typescript
interface STTProvider {
  readonly name: string;
  transcribe(audio: Buffer, options?: STTOptions): Promise<TranscribeResult>;
  createStream(options?: STTStreamOptions): STTStream;
}
```

Both providers support batch transcription via `transcribe()` and streaming via `createStream()`. The key difference is that Deepgram provides true real-time streaming over WebSocket, while OpenAI buffers audio and transcribes on `close()`.

## Provider Comparison

| Feature          | OpenAI STT                                                 | Deepgram STT                 |
| ---------------- | ---------------------------------------------------------- | ---------------------------- |
| Default model    | `gpt-transcribe`                                           | `nova-3`                     |
| Streaming        | Buffered (transcribes on close)                            | Real-time WebSocket          |
| Interim results  | No                                                         | Yes                          |
| Word timestamps  | `whisper-1` only                                           | Yes                          |
| Endpointing      | No                                                         | Configurable                 |
| Auto-punctuation | Yes                                                        | Yes                          |
| Other models     | `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `whisper-1` | `nova-2`, `enhanced`, `base` |

## OpenAI STT

Uses the OpenAI Audio Transcriptions API. Default model is `gpt-transcribe`.

Requires the optional `openai` peer (`pnpm add openai`). It is loaded on the first `transcribe` call (or the first stream `close()`), not when you import the package or construct `OpenAISTT`; if it is missing, that call throws with an install hint.

```typescript
import { OpenAISTT } from '@cogitator-ai/voice';

const stt = new OpenAISTT({
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'gpt-transcribe',
});
```

### Batch Transcription

```typescript
const result = await stt.transcribe(audioBuffer, { language: 'en' });
console.log(result.text);

const whisper = new OpenAISTT({ apiKey: process.env.OPENAI_API_KEY!, model: 'whisper-1' });
const detailed = await whisper.transcribe(audioBuffer);
console.log(detailed.duration, detailed.words);
// [{ word: 'Hello', start: 0.0, end: 0.32, confidence: 1 }, ...]
```

`gpt-transcribe` and `gpt-4o(-mini)-transcribe` only support the `json` response format, so word timestamps and duration are returned for `whisper-1` only. For `gpt-transcribe`, `language` is sent as a `languages` list and the language the API detects is returned in `result.language`. Input can be a containerized file (wav, mp3, ogg, flac, webm, mp4 — detected from magic bytes) or headerless PCM16 mono 16kHz, which is wrapped into WAV automatically.

### Streaming

OpenAI STT streaming collects raw PCM16 chunks (at `sampleRate`, default 16kHz) and transcribes the complete buffer as WAV when `close()` is called. Calling `close()` twice returns the same result; empty streams do not hit the API.

```typescript
const stream = stt.createStream({ language: 'en' });

stream.on('final', (result) => {
  console.log('Transcription:', result.text);
});

stream.write(audioChunk1);
stream.write(audioChunk2);
const result = await stream.close();
```

### Configuration

| Field     | Type     | Default          | Description         |
| --------- | -------- | ---------------- | ------------------- |
| `apiKey`  | `string` | required         | OpenAI API key      |
| `model`   | `string` | `gpt-transcribe` | Model ID            |
| `baseURL` | `string` | —                | Custom API base URL |

## Deepgram STT

Uses the Deepgram API with real-time WebSocket streaming. Default model is `nova-3`.

```typescript
import { DeepgramSTT } from '@cogitator-ai/voice';

const stt = new DeepgramSTT({
  apiKey: process.env.DEEPGRAM_API_KEY!,
  model: 'nova-3',
  language: 'en',
});
```

### Batch Transcription

```typescript
const result = await stt.transcribe(audioBuffer);

console.log(result.text);
console.log(result.duration);
console.log(result.words);
// [{ word: 'Hello', start: 0.0, end: 0.32, confidence: 0.99 }, ...]
```

Files with a container (wav, mp3, ogg, flac, webm, mp4) are sent as-is; headerless input is sent as raw PCM16 mono at `sampleRate`. Requests time out after 30 seconds.

### Streaming

Deepgram streaming opens a WebSocket connection and provides real-time results as audio arrives. Interim results (on by default) update continuously; a `final` event is emitted for every finalized segment, and `close()` resolves with all final segments joined, their words and the total duration. Audio written before the socket opens is queued. `close()` waits up to 10 seconds for Deepgram to flush; if the connection drops before `close()`, an `error` is emitted.

```typescript
const stream = stt.createStream({
  interimResults: true,
  endpointing: 500,
  language: 'en',
});

stream.on('partial', (text) => {
  console.log('Interim:', text);
});

stream.on('final', (result) => {
  console.log('Final:', result.text);
});

stream.on('error', (err) => {
  console.error('Stream error:', err);
});

stream.write(audioChunk1);
stream.write(audioChunk2);
await stream.close();
```

### Configuration

| Field        | Type     | Default  | Description                                                   |
| ------------ | -------- | -------- | ------------------------------------------------------------- |
| `apiKey`     | `string` | required | Deepgram API key                                              |
| `model`      | `string` | `nova-3` | Model ID                                                      |
| `language`   | `string` | —        | Default language code                                         |
| `sampleRate` | `number` | `16000`  | Sample rate of raw PCM16 (streams and headerless batch input) |

## STT Options

Options for `transcribe()` (and `createStream()`):

| Field      | Type     | Description                                                  |
| ---------- | -------- | ------------------------------------------------------------ |
| `language` | `string` | Language code (e.g. `"en"`, `"es"`, `"ja"`)                  |
| `prompt`   | `string` | Context hint to improve transcription accuracy (OpenAI only) |

Additional options for `createStream()`:

| Field            | Type      | Description                                                        |
| ---------------- | --------- | ------------------------------------------------------------------ |
| `interimResults` | `boolean` | Partial/interim results (Deepgram, default `true`)                 |
| `endpointing`    | `number`  | Silence duration (ms) for Deepgram endpointing                     |
| `sampleRate`     | `number`  | Sample rate of the raw PCM16 written to the stream (default 16000) |

## TranscribeResult

Both providers return the same result shape:

```typescript
interface TranscribeResult {
  text: string;
  language?: string;
  duration?: number;
  words?: Array<{
    word: string;
    start: number;
    end: number;
    confidence: number;
  }>;
}
```

## Custom STT Provider

Implement `STTProvider` to integrate any speech recognition service:

```typescript
import { EventEmitter } from 'node:events';
import type {
  STTProvider,
  STTOptions,
  STTStream,
  STTStreamOptions,
  TranscribeResult,
} from '@cogitator-ai/voice';

class BufferedStream extends EventEmitter implements STTStream {
  private chunks: Buffer[] = [];

  constructor(
    private readonly stt: STTProvider,
    private readonly options?: STTStreamOptions
  ) {
    super();
  }

  write(chunk: Buffer): void {
    this.chunks.push(chunk);
  }

  async close(): Promise<TranscribeResult> {
    const result = await this.stt.transcribe(Buffer.concat(this.chunks), this.options);
    this.emit('final', result);
    return result;
  }
}

class WhisperLocalSTT implements STTProvider {
  readonly name = 'whisper-local';

  async transcribe(audio: Buffer, options?: STTOptions): Promise<TranscribeResult> {
    // call your local Whisper instance
    return { text: 'transcribed text' };
  }

  createStream(options?: STTStreamOptions): STTStream {
    return new BufferedStream(this, options);
  }
}
```

The pipeline writes raw PCM16 (16kHz) chunks to the stream, listens for `partial` / `final` / `error`, and uses the result of `close()` as the user's turn.