# Streaming

URL: https://cogitator.app/docs/channels/streaming

> Progressive message delivery with StreamBuffer and message editing.

## Overview

Cogitator uses a **send-then-edit** pattern for streaming responses in chat. Instead of waiting for the full LLM response (2-10 seconds), the user sees the answer appearing progressively.

## How It Works

Streaming is on when the Gateway gets a `stream` config; without it every answer is sent once, complete (chunked to the platform limit).

1. LLM generates tokens via `cogitator.run()` with `stream: true`
2. Tokens accumulate in a `StreamBuffer`
3. Every `flushInterval` ms (measured from the end of the previous flush), the buffer flushes:
   * **First flush** → `channel.sendText()` creates a new message (or a Telegram draft via `sendDraft`)
   * **Subsequent flushes** → `channel.editText()` updates the message
4. When the text reaches `maxMessageChars` (platform limit by default) the current message is finalized at a line/word boundary and a new one is started — no text is lost
5. After generation completes → final edit (or final message in draft mode)

All channel calls are serialized, so a slow `sendText` never races with an edit and every created message is tracked (for `deleteOnAbort`).

## Configuration

```typescript
interface StreamConfig {
  flushInterval: number; // ms between flushes, e.g. 500
  minChunkSize: number; // skip a flush while the current message text is shorter than this
  minInitialChars?: number; // hold back the first message until N chars
  maxMessageChars?: number; // split threshold, default: platform limit
  deleteOnAbort?: boolean; // delete partial messages when the run fails or is interrupted
}
```

```typescript
const gateway = new Gateway({
  // ...
  stream: {
    flushInterval: 500,
    minChunkSize: 20,
  },
});
```

## Platform Support

| Platform | Edit Support                    | Rate Limit     | Max Length   |
| -------- | ------------------------------- | -------------- | ------------ |
| Telegram | Yes (`editMessageText`)         | \~30 edits/sec | 4,096 chars  |
| Discord  | Yes (`message.edit`)            | \~5 edits/sec  | 2,000 chars  |
| Slack    | Yes (`chat.update`)             | \~20 edits/sec | 40,000 chars |
| WhatsApp | Yes (message edit)              | —              | 65,536 chars |
| WebChat  | Yes (WebSocket `edit` event)    | No limit       | No limit     |
| Terminal | Yes (output rewritten in place) | No limit       | No limit     |

`RuntimeBuilder` streams with `flushInterval: 600` and `minChunkSize: 30` unless `stream` in `cogitator.yml` overrides them. Telegram streams through rich drafts (`sendRichMessageDraft`, Markdown rendered as written) with a stop button, and falls back to `editMessageText` in groups or if a draft fails; a press of the stop button stops the run and keeps what was written as the reply (`stream.stopButton: false` turns it off); Discord edits keep their continuation messages in sync when an answer grows past 2,000 chars.

If the model produced no tokens (e.g. a provider without streaming), the Gateway sends the complete response as regular chunked messages.

## StreamBuffer API

The `StreamBuffer` class is used internally by Gateway, but can be used standalone:

```typescript
import { StreamBuffer, adaptMarkdown } from '@cogitator-ai/channels';

const buffer = new StreamBuffer(
  channel, // any Channel implementation
  chatId,
  { flushInterval: 500, minChunkSize: 20, maxMessageChars: 4096 },
  replyToMessageId, // optional, used for the first message only
  true, // use channel.sendDraft when available
  (text) => adaptMarkdown(text, 'telegram') // optional formatter applied to every send/edit
);

buffer.start();

// append tokens as they arrive
buffer.append('Hello');
buffer.append(' world');
buffer.append('!');

// when generation is done — rethrows if the final delivery failed
const lastMessageId = await buffer.finish();
buffer.getMessageIds(); // every message created for this answer

// or, on failure
await buffer.abort(); // deletes partial messages when deleteOnAbort is set
```

`forceNewMessage()` finalizes the current message with the buffered text and continues in a new one.

## Markdown Formatting

The Gateway passes `adaptMarkdown(text, channelType)` as the formatter, so every flush is converted for the platform (code spans and fences are never touched):

* **Telegram**: `# Heading` and `**bold**` → `*bold*`, `* item` → `• item`
* **Discord**: Standard markdown passes through
* **Slack**: `**bold**` → `*bold*`, `~~strike~~` → `~strike~`, links → `<url|text>`, `&<>` escaped
* **WhatsApp**: `**bold**` → `*bold*`, headings → bold, links → `text (url)`

Non-streamed answers are chunked with `chunkMessage()` at paragraph → line → word boundaries, closing and reopening code fences across chunks.