Session Budgets
Limits of money, tokens and tool calls for a session and for each agent in it, with a warning before the limit and a pause instead of a failure when it is reached.
Overview
A session (a thread, across all its runs) can have a budget: how much money, how many tokens and how many tool calls its runs may spend together, and how much each agent of it may spend. A run that reaches a limit does not fail. It pauses with an interrupt of kind budget, and a person raises the limit, lets the run go on past it, or stops it with what it has.
const cog = new Cogitator({
memory: { adapter: 'postgres', postgres: { connectionString } },
budget: {
session: { maxCostUsd: 2, maxTokens: 400_000, maxToolCalls: 200 },
agent: { maxCostUsd: 1 },
warnAt: 0.8,
onWarning: (warning) =>
notify(`${warning.scope} ${warning.metric} at ${warning.used}/${warning.limit}`),
},
});
const researcher = new Agent({
name: 'researcher',
instructions: 'You research.',
budget: { maxToolCalls: 40 }, // over budget.agent for this agent
});A session is a thread: a new threadId starts with nothing spent, so tie the thread ids your clients may use to your users if a budget must hold per user. An agent with a budget of its own is held to it within the thread, its spending kept under its id, else its name (so a new instance of the same agent, on a resume or in another process, counts on), which makes unique agent names a must for agent budgets, and what an agent run through agentAsTool() spends counts toward the caller's session and agent too. A paused run keeps its own budgets (RunOptions.budget) in its checkpoint for the resume.
Cost is the run's usage.cost: what the provider reported, else the tokens priced from the model registry or llm.pricing (see Cogitator). Tokens are input plus output tokens. Tool calls are the calls that ran. RunOptions.budget sets budgets for one run over the runtime's, field by field, and RunOptions.onBudgetWarning hears the warnings of one run.
Where the Spending Is Kept
What a session spent lives in the thread's metadata (budget, see SESSION_BUDGET_KEY): in all, by agent, and the limits answers raised. Without memory, or for a run with useMemory: false, it is kept in the process. Each run adds what it spent when it pauses, ends or fails, and RunResult.sessionSpend holds the total after it. Runs of one thread that go on at the same time in one process share one live count of the session, read again from storage when a run starts, so each sees what the others spend before every model call and tool call. Runs of one thread in different processes see each other's spending only once it is added, so give a session one run at a time (multitaskStrategy: 'enqueue') there when the limit must hold to the cent.
When a Limit Is Reached
Before each model call the run checks money and tokens, before each tool turn the tool calls the turn would add. A call counts once its tool starts, so a call refused or paused for an approval first does not use up the allowance, and right before it starts the allowance is checked again, with what delegates of the run spent meanwhile: a call past it answers the model that the budget is spent. What the memory upkeep after a run (extraction, compaction) spends on the model counts in the run's usage and budgets and in the session's spending. The first limit reached pauses the run:
const result = await cog.run(researcher, { input, threadId });
if (result.status === 'paused') {
const budget = result.interrupts?.find((entry) => entry.kind === 'budget');
console.log(budget?.message);
// budget.details: { scope: 'session', metric: 'cost', used: 2.04, limit: 2 }
await cog.resume(researcher, threadId, {
budget: { session: { maxCostUsd: 5 } }, // the host raises the limit
});
// or end it: responses: { [budget!.id]: { action: 'stop' } }
}Without budget.allowRaise: true a client can only answer { action: 'stop' } (the interrupt's responseSchema allows nothing else): raising a limit is the host's decision, made by resuming with a higher budget of its own (cog.resume(agent, threadId, { budget: { session: { maxCostUsd: 5 } } })). A budget the host passes to resume() wins over a limit an earlier answer raised or lifted, and changes only the limits it names: { session: { maxCostUsd: 5 } } keeps the run's maxToolCalls. With allowRaise: true:
| Answer | What happens |
|---|---|
{ action: 'continue', limit: 5 } | the limit is 5 from now on: for the session (kept in the thread) for a session or agent limit, for the run otherwise |
{ action: 'continue' } | the run goes on past this limit, the next run of the session stops at it again |
{ action: 'stop' } | the run ends with status: 'completed', budgetStop names the limit, calls of a paused tool turn get "not run" results |
An answer that does not match the interrupt's responseSchema fails the resume with INTERRUPT_ANSWER_INVALID. A resume without an answer pauses again at the same limit.
warnAt (default 0.8) is the share of a limit at which onWarning hears of it, once per run per limit.
Run Limits
limits.maxTokensPerRun and the cost routing budgets (maxCostPerRun, maxCostPerHour, maxCostPerDay) fail a run as before (RUN_TOKEN_LIMIT_EXCEEDED, BUDGET_EXCEEDED). With limits.onExceeded: 'interrupt' they pause it the same way instead, with scopes run, hour and day: a raised limit or a continue holds for the rest of that run.
const cog = new Cogitator({
limits: { maxTokensPerRun: 200_000, onExceeded: 'interrupt' },
});