Fall Back to Another Model
easy5 minmodel-fallback.ts
Your provider has an outage, the account runs out of credits or a model is retired, and every run fails with it. You want the turn answered by another model instead.
What it shows
- List
fallbackModelson the agent: a failing turn moves to the next one - Rate limits are retried first, an exhausted quota or a missing model moves at once
- A circuit breaker skips a model that keeps failing,
onModelFallbackreports each move
The code
model-fallback.ts
import { Agent, Cogitator } from '@cogitator-ai/core';
const cog = new Cogitator({
llm: { modelCircuitBreaker: { failureThreshold: 3, resetTimeout: 60_000 } },
});
const agent = new Agent({
name: 'support',
model: 'ollama/retired-model:7b',
fallbackModels: ['ollama/qwen2.5:0.5b'],
instructions: 'Answer in one short sentence.',
});
const result = await cog.run(agent, {
input: 'What is a circuit breaker in software?',
onModelFallback: ({ from, to, error }) => console.log(`${from} failed (${error.message}), using ${to}`),
});
console.log(result.modelUsed, result.output);
await cog.close();Run it
install · bash
pnpm add @cogitator-ai/core
ollama pull qwen2.5:0.5brun · bash
npx tsx model-fallback.tsOr run the full example from a clone of the repository (keys in .env):
repository · bash
npx tsx examples/core/19-model-fallback.ts