Cost-Aware Routing

OverviewCopy Anchor Link

Cost-aware routing analyzes incoming tasks, picks a model based on capability requirements and price, enforces budget limits, and tracks spending. Turn it on for a Cogitator with costRouting, or use its parts on their own.

In a CogitatorCopy Anchor Link

import { Cogitator } from '@cogitator-ai/core';

const cog = new Cogitator({
  llm: {
    providers: {
      openai: { apiKey: process.env.OPENAI_API_KEY! },
      anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },
    },
  },
  costRouting: {
    enabled: true,
    autoSelectModel: true,
    budget: { maxCostPerRun: 0.5, maxCostPerDay: 50 },
  },
});

const result = await cog.run(agent, { input });

cog.getCostSummary(); // totals by model and agent
cog.getCostRouter()?.getBudgetStatus();

With enabled: true, the budget is checked twice. Before a run starts, its estimated cost is checked, so a run that would clearly go over never makes a call. Then, before every model call, what has really been spent is checked: the run's own cost against maxCostPerRun, and every run's spending over the last hour and day against maxCostPerHour and maxCostPerDay. A run that reaches a limit stops before its next call with a CogitatorError whose code is BUDGET_EXCEEDED (statusCode 429) and whose message is Budget exceeded: <reason>. A call already in flight can take spending past a limit by its own cost, never further.

The cost of each model call is recorded as soon as it is answered, from the cost the provider reports or, when it reports none, from the model's price. So a run that fails or is cancelled still counts toward the hourly and daily budget, and runs side by side see each other's spending at once. Costs are recorded unless trackCosts: false; a budget keeps its records either way, since its limits are read from them.

With autoSelectModel: true, the router analyzes the input and picks a model from the model registry, considering only providers the runtime can call: a backend in llm.backends, a registered plugin, llm.defaultProvider, the provider of the agent's own model, or a built-in provider configured in llm.providers with its credentials. The budget is then checked against that model's estimated cost. When no callable provider has a fitting model, the run keeps the agent's model. preferLocal (on by default) favours local models such as Ollama's.

Without autoSelectModel, the run uses the agent's model, and the budget check estimates the run's cost from the input's complexity and that model's price (0 when the price is unknown, so the hourly and daily limits still hold against what was already spent).

getCostRouter() and getCostSummary() work before the first run; they return undefined only without costRouting.enabled.

cog.estimateCost() predicts a run's cost before you start it (see Cost Estimation):

const estimate = await cog.estimateCost({
  agent,
  input: 'Analyze this document and summarize key points',
  options: { assumeIterations: 3, assumeToolCalls: 5 },
});

if (estimate.expectedCost > 0.1) {
  console.log(`Expected $${estimate.expectedCost.toFixed(4)}`);
}

CostAwareRouterCopy Anchor Link

The CostAwareRouter orchestrates all cost-related decisions:

import { CostAwareRouter } from '@cogitator-ai/core';

const router = new CostAwareRouter({
  config: {
    enabled: true,
    autoSelectModel: true,
    preferLocal: true,
    minCapabilityMatch: 0.3,
    trackCosts: true,
    budget: {
      maxCostPerRun: 0.5,
      maxCostPerHour: 5.0,
      maxCostPerDay: 50.0,
      warningThreshold: 0.8,
      onBudgetWarning: (current, limit) => {
        console.log(`Budget warning: $${current.toFixed(2)} / $${limit}`);
      },
      onBudgetExceeded: (current, limit) => {
        console.log(`Budget exceeded: $${current.toFixed(2)} > $${limit}`);
      },
    },
  },
});

Task AnalysisCopy Anchor Link

The TaskAnalyzer inspects the input text to determine what capabilities the task requires:

const requirements = router.analyzeTask(
  'Look at this screenshot of our dashboard, search the logs and debug why the SQL query is slow'
);

console.log(requirements);
// {
//   needsVision: true,
//   needsToolCalling: true,
//   needsLongContext: false,
//   needsReasoning: 'basic',
//   needsSpeed: 'balanced',
//   costSensitivity: 'medium',
//   complexity: 'simple',
//   domains: ['code'],
// }

The analyzer works on keywords: vision needs (image/screenshot), tool requirements (search/execute), long context (documents, codebases, over 500 words), reasoning level (analyze/synthesize vs explain/list), speed preferences (quick/urgent vs thorough), cost sensitivity (cheap/budget vs best/premium), complexity (length, multiple steps, conditions) and domains (code, math, creative, legal, medical, finance and others).

Model SelectionCopy Anchor Link

The ModelSelector scores every available model against the task requirements and picks the best fit:

const recommendation = await router.recommendModel('Write a quick summary of this text');

recommendation.modelId; // e.g. 'gpt-6-luna'
recommendation.provider; // e.g. 'openai'
recommendation.score; // capability score, 100 = neutral
recommendation.reasons; // ['Fast response time', 'Efficient for simple tasks', ...]
recommendation.estimatedCost; // USD, estimated from the model's pricing
recommendation.fallbacks; // the next three model ids

Every model in the registry that is not deprecated and supports what the task needs (vision, tool calling, 32k+ context for long tasks) is scored from 100:

• Local models — +15, since they have no API cost
• Cost efficiency — cheap models gain and expensive ones lose when costSensitivity is high
• Speed — fast models (gpt-6-luna, claude-haiku, gemini-flash) score higher for urgent tasks
• Reasoning — advanced models (gpt-6.1-sol, claude-sonnet, o3) score higher for advanced reasoning and complex tasks
• Domain fit — code-specialized models score higher for programming tasks

Models below minCapabilityMatch × 100 (default 30) are dropped. With preferLocal, the best local model wins when it scores at least 80% of the best overall. With no candidate left, it recommends openai/gpt-6-luna. recommendModelForRequirements(requirements) skips the analysis step.

recommendAvailableModel(input, isProviderAvailable) ranks only the models of providers the callback accepts and has no fallback: it returns undefined when none of them fits. The runtime uses it for autoSelectModel:

const configured = new Set(['openai', 'ollama']);
const pick = await router.recommendAvailableModel(input, (provider) => configured.has(provider));

const model = pick ? `${pick.provider}/${pick.modelId}` : agent.model;

Budget EnforcementCopy Anchor Link

checkBudget() compares an estimated cost with the limits and the tracked spending:

const check = router.checkBudget(estimatedCost);

if (!check.allowed) {
  console.log('Budget blocked:', check.reason);
  // "Would exceed daily budget ($48.50 + $2.00 > $50.00)"
}

checkRunBudget(input, model) estimates the cost itself, from the complexity of input and the price of model (a registry model id), the way the runtime checks runs without autoSelectModel:

const runCheck = router.checkRunBudget('Summarize this report', 'gpt-6-luna');

Three budget levels are enforced independently:

When tracked spending reaches warningThreshold (default 0.8) of the hourly or daily limit, onBudgetWarning fires once (until clearCostHistory()). When a limit would be exceeded, onBudgetExceeded fires and the check returns allowed: false.

checkSpent(runCost) is the check the runtime makes before every model call: runCost is what the run has really spent so far, and the hourly and daily limits are checked against the recorded spending. It refuses once a limit is reached, where checkBudget refuses an estimate that would go over one.

const status = router.getBudgetStatus(); // undefined without a budget

if (status) {
  console.log(`Hourly: $${status.hourlyUsed.toFixed(2)} / $${status.hourlyLimit}`);
  console.log(`Daily: $${status.dailyUsed.toFixed(2)} / $${status.dailyLimit}`);
  console.log(`Remaining today: $${status.dailyRemaining?.toFixed(2)}`);
}

Token EstimationCopy Anchor Link

The TokenEstimator predicts token usage before making any API calls:

import { TokenEstimator } from '@cogitator-ai/core';

const estimator = new TokenEstimator();

const inputTokens = estimator.estimateInputTokens({
  systemPrompt: agent.instructions,
  userInput: 'Analyze this codebase...',
  toolSchemas: agent.tools.map((t) => t.toJSON()),
  iterations: 2,
  includeMemory: true,
});

const outputTokens = estimator.estimateOutputTokens({
  complexity: 'complex',
  hasTools: true,
  toolCallCount: 3,
  iterations: 2,
});

console.log(`Input: ${inputTokens.expected} tokens (${inputTokens.min}-${inputTokens.max})`);
console.log(`Output: ${outputTokens.expected} tokens (${outputTokens.min}-${outputTokens.max})`);

Each estimate returns min, max, and expected values accounting for system prompt size, tool schemas, memory context, and estimated iteration count. Text is counted at about 4 characters per token. estimateIterations(complexity, hasTools) and estimateToolCalls(complexity, toolCount) give the defaults the cost estimator uses.

Cost EstimationCopy Anchor Link

The CostEstimator combines token estimation with model pricing for a full cost prediction:

import { CostEstimator } from '@cogitator-ai/core';

const costEstimator = new CostEstimator();

const estimate = await costEstimator.estimate({
  agent,
  input: 'Design a microservices architecture for an e-commerce platform',
  options: { assumeIterations: 3, memoryTokenEstimate: 1500 },
});

console.log(`Expected: $${estimate.expectedCost}`);
console.log(`Range: $${estimate.minCost} - $${estimate.maxCost}`);
console.log(`Confidence: ${(estimate.confidence * 100).toFixed(0)}%`);
console.log(`Warnings:`, estimate.warnings);
console.log(`Breakdown:`, estimate.breakdown);

The estimate prices the agent's model (pass model for another one; cog.estimateCost() passes the model the run would use). Local models (Ollama) are reported as zero-cost. When model pricing data is unavailable, the estimator uses conservative defaults and flags a warning. options override the assumed iterations and tool calls and the memory size (includeMemory, memoryTokenEstimate).

Cost TrackingCopy Anchor Link

The CostTracker records actual spending and provides real-time aggregations:

router.recordCost({
  runId: 'run_abc',
  agentId: 'research-agent',
  model: 'gpt-6.1-sol',
  inputTokens: 2500,
  outputTokens: 800,
  cost: 0.035,
});

console.log(`This run: $${router.getRunCost('run_abc').toFixed(4)}`);
console.log(`Last hour: $${router.getHourlyCost().toFixed(2)}`);
console.log(`Today: $${router.getDailyCost().toFixed(2)}`);

const summary = router.getCostSummary();
console.log(`Total: $${summary.totalCost.toFixed(2)}`);
console.log(`By model:`, summary.byModel);
console.log(`By agent:`, summary.byAgent);
console.log(`Runs: ${summary.runCount}`);

The tracker maintains sliding windows for hourly and daily costs, automatically pruning old records. Use clearCostHistory() to reset all tracking data, and updateConfig() to change limits at runtime. CostTracker, BudgetEnforcer, TaskAnalyzer and ModelSelector are exported for use on their own; CostTracker.getSummary(filter) and getRecords(filter) filter by agent, run or time (since).