Chunking Strategies

Why Chunking MattersCopy Anchor Link

Embedding models have token limits and perform best on focused text segments. Chunking splits large documents into smaller pieces that can be individually embedded and retrieved. The chunking strategy you choose directly affects retrieval quality.

Every chunker returns DocumentChunk objects: id, documentId, content, startOffset / endOffset (character offsets into the document) and order. Sizes are measured in characters, not tokens.

Fixed-Size ChunkingCopy Anchor Link

Splits text into chunks of exactly chunkSize characters with a configurable overlap. Fast and predictable, but splits can land in the middle of sentences or paragraphs.

import { FixedSizeChunker } from '@cogitator-ai/rag';

const chunker = new FixedSizeChunker({
  chunkSize: 500,
  chunkOverlap: 50,
});

const chunks = chunker.chunk(documentText, 'doc-1');

FixedSizeChunker and RecursiveChunker throw when chunkSize is not positive, chunkOverlap is negative, or chunkOverlap >= chunkSize.

When to use: Uniform-length data (logs, code), or when you need predictable chunk sizes and don't care about semantic boundaries.

Recursive ChunkingCopy Anchor Link

Splits text hierarchically using a list of separators — first by double newlines (paragraphs), then single newlines, then sentences, then words. Produces chunks that respect natural text boundaries while staying under the size limit.

import { RecursiveChunker } from '@cogitator-ai/rag';

const chunker = new RecursiveChunker({
  chunkSize: 512,
  chunkOverlap: 50,
  separators: ['\n\n', '\n', '. ', ' '],
});

const chunks = chunker.chunk(documentText, 'doc-1');

Default separators: ['\n\n', '\n', '. ', ' ', '']

Guarantees: chunk.content === text.slice(startOffset, endOffset) for every chunk (even with runs of repeated separators), no chunk exceeds chunkSize, chunks are trimmed, sentence punctuation stays with its sentence, every non-whitespace character is covered, and overlap never produces a chunk fully contained in the previous one.

When to use: General-purpose documents (articles, documentation, books). This is the recommended default for most use cases.

Semantic ChunkingCopy Anchor Link

Groups sentences by semantic similarity using embeddings. Text is split into sentences (after ., ?, !) and blank-line separated blocks (headings, list blocks), adjacent units are embedded, and a new chunk boundary is created when the cosine similarity between consecutive sentences drops below a threshold. This is an async chunker since it calls the embedding service.

import { SemanticChunker } from '@cogitator-ai/rag';
import { OpenAIEmbeddingService } from '@cogitator-ai/memory';

const embeddings = new OpenAIEmbeddingService({
  apiKey: process.env.OPENAI_API_KEY!,
});

const chunker = new SemanticChunker({
  embeddingService: embeddings,
  breakpointThreshold: 0.5,
  minChunkSize: 100,
  maxChunkSize: 2000,
});

const chunks = await chunker.chunk(documentText, 'doc-1');

minChunkSize defaults to min(100, maxChunkSize). Groups longer than maxChunkSize are split at sentence boundaries, and single sentences longer than that are cut into maxChunkSize pieces. Semantic chunks never overlap. Every sentence is embedded once (embedBatch), so ingestion costs roughly twice the embedding calls of the other strategies.

When to use: Documents where topic boundaries don't align with formatting (transcripts, long-form prose, research papers). Produces the highest quality chunks but is slower due to embedding calls.

createChunker FactoryCopy Anchor Link

Create a chunker from a config object without importing individual classes:

import { createChunker } from '@cogitator-ai/rag';

const chunker = createChunker({
  strategy: 'recursive',
  chunkSize: 512,
  chunkOverlap: 50,
});

For semantic chunking, pass the embedding service as the second argument; chunkSize becomes maxChunkSize, and the other semantic options keep their defaults (construct SemanticChunker directly to change them):

const chunker = createChunker(
  { strategy: 'semantic', chunkSize: 2000, chunkOverlap: 0 },
  embeddingService
);

ComparisonCopy Anchor Link

Custom ChunkersCopy Anchor Link

Implement Chunker for synchronous chunking or AsyncChunker for async:

import { randomUUID } from 'node:crypto';
import type { Chunker, DocumentChunk } from '@cogitator-ai/rag';

class ParagraphChunker implements Chunker {
  chunk(text: string, documentId: string): DocumentChunk[] {
    const chunks: DocumentChunk[] = [];
    for (const match of text.matchAll(/\S(?:[\s\S]*?\S)?(?=\s*\n\s*\n|\s*$)/g)) {
      chunks.push({
        id: randomUUID(),
        documentId,
        content: match[0],
        startOffset: match.index,
        endOffset: match.index + match[0].length,
        order: chunks.length,
      });
    }
    return chunks;
  }
}

Keep content === text.slice(startOffset, endOffset) so offsets stay meaningful in the stored metadata. The pipeline drops whitespace-only chunks.

Pass it to the builder:

const pipeline = new RAGPipelineBuilder()
  .withChunker(new ParagraphChunker())
  // ...
  .build();