diff --git a/reference/configuration/options.md b/reference/configuration/options.md index d12128410..c50531fad 100644 --- a/reference/configuration/options.md +++ b/reference/configuration/options.md @@ -426,10 +426,11 @@ agent: - `enabled` — Enable the agent component; _Default_: `false` - `provider` — Recorded on the session but **not yet used to route the model call** — only `model` reaches the provider. Set the provider through the [`models`](../models/overview.md#configuration) configuration instead - `model` — Model id override, passed through to the model call; _Default_: the [`models`](../models/overview.md#configuration) generative default -- `maxTurns` — Maximum tool-call iterations in a single run; _Default_: `50` +- `maxTurns` — Maximum tool-call iterations in a single run; a positive integer, validated from v5.4.0. A bare `maxTurns:` with no value fails the agent's start like any other invalid value; _Default_: `50` - `maxTokens` — Output-token cap sent with every model request the agent makes. A reply cut off at this cap ends the run with status `error`, not `completed`, and `lastError` says it was cut off; raise the cap if replies keep hitting it. A cut-off reply can also mean the session's transcript filled the model's context window, which a higher cap does not fix; start a new session instead. Keep it within the model's own output limit: most providers reject a larger value on every request, and the run then ends `error` with the provider's message, so lower it for a model with a small output limit (some older models allow 4096 or 8192). Before v5.4.0 the agent sent no cap: Anthropic models, called directly or through Bedrock, then used 4096, and the other backends sent none; _Default_: `16384` - `maxToolResultBytes` — Largest tool result, in bytes of JSON, the agent adds to its conversation. A larger result is cut to this size, ending with a note that gives the original size and tells the model how to ask for less. The file-reading tools return pages of at most half this size, measured as JSON, so a page arrives whole unless the file's path alone takes up most of the cap. Lower it for a model with a small context window. It must be an integer from `1024` to `1048576`; at startup an invalid value is logged and the default is used. See [When a conversation outgrows the model's context window](../operations-api/operations.md#when-a-conversation-outgrows-the-models-context-window); _Default_: `65536` -- `maxCostUsd` — Intended per-run cost ceiling. **Not enforced** — it is a stored setting only, and nothing checks spend against it; _Default_: `5.00` +- `maxSessionTokens` — Per-session cap on prompt plus completion tokens, summed over every model call in the session that returns, a reply the agent rejects as cut off or empty included; a positive integer. See [Spend caps](#agent-spend-caps); _Default_: none +- `maxCostUsd` — Per-session cap in USD, priced from the [`pricing`](../models/overview.md#pricing) on the `models.generative` entry that served each call; a number of `0` or more, and `0` stops every session before its first call. See [Spend caps](#agent-spend-caps). Before v5.4.0 this setting was stored but never enforced, and defaulted to `5.00`; _Default_: none - `autoApprove` — Run without per-action approval gates; _Default_: `false` - `allowDestructive` — Include the tools marked destructive in the agent's toolset: `write_file`, the inspector's code-evaluation tools, and any operations tool carrying MCP's [`destructiveHint`](../mcp/tool-metadata.md) (`drop_table`, `delete`, `restart`, `set_configuration`, ...). When `false` they are removed entirely rather than gated. That hint comes from a curated set in core which does not cover every damaging operation, so this is not a complete safety boundary on its own — see [Agent operations](../operations-api/operations.md#agent); _Default_: `false` - `user` — Harper user the agent's **operations** tools run as; the filesystem, HTTP, schedule, and inspector tools always run at process privilege regardless. If it cannot be resolved and it is not the default, the agent fails closed and runs with no operations tools; _Default_: `hdb_agent`, which falls back to a `super_user` bootstrap identity @@ -438,7 +439,7 @@ agent: - `httpFetch` — Whether the agent has its `http_fetch` tool, and which hosts it may reach: `true`, `false`, or `{ allow: [...] }`. Read at startup only. See [Restricting `http_fetch`](#restricting-http_fetch); _Default_: `true` - `systemPromptAppend` — Operator text appended to the agent's system prompt -`enabled`, `provider`, `model`, `maxTurns`, `maxTokens`, `maxToolResultBytes`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend` can also be changed at runtime with [`set_agent_config`](../operations-api/operations.md#set_agent_config), which applies in memory only. `enabled` is the exception worth knowing: it cannot switch the agent on, because with the agent disabled at startup no agent operation is registered at all. `httpFetch`, `componentsScope`, and `configScope` are read at startup only, and `set_agent_config` rejects them. +`enabled`, `provider`, `model`, `maxTurns`, `maxTokens`, `maxToolResultBytes`, `maxSessionTokens`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend` can also be changed at runtime with [`set_agent_config`](../operations-api/operations.md#set_agent_config), which applies in memory only. `enabled` is the exception worth knowing: it cannot switch the agent on, because with the agent disabled at startup no agent operation is registered at all. `httpFetch`, `componentsScope`, and `configScope` are read at startup only, and `set_agent_config` rejects them. ### Filesystem scopes @@ -472,6 +473,25 @@ A key with neither a key file name nor PEM armor, such as a `.p12` bundle or raw The config file can still hold secrets that are not PEM keys, such as a model `apiKey` or storage credentials, and the agent reads it. Setting `configScope` to `rootPath` also restores read access to the raw database files and backups. +### Agent spend caps + + + +`maxSessionTokens` and `maxCostUsd` bound a whole session, across every run in it. Approving a paused action or sending a follow-up prompt starts a new run, but does not reset a cap the way it resets `maxTurns`. Each assistant message records the usage of the model call that produced it, and [`get_agent_session`](../operations-api/operations.md#get_agent_session) reports the session's totals as `usage`. + +A cap is checked when a run starts, before it executes an approved action, and after every model call: + +- **At or over a cap**, the session makes no further model call and ends `error`, with `lastError` naming the cap, for example `agent.maxSessionTokens=200000 reached: this session used 201450 tokens`. Tool calls the model had just requested are answered with a `budget_exceeded` result rather than run, so the transcript stays valid if you continue the session. A final answer that crosses the cap is still delivered: the run ends `completed`, with `lastError` naming the cap. +- **A cap that cannot be measured stops the session too.** `maxCostUsd` needs every call priced, and `maxSessionTokens` needs every call's token counts. A model with no `pricing`, or a provider that reports no usage, stops the session after its first call rather than letting it run uncapped: it ends `error`, or `completed` if that call was the final answer, with `lastError` saying why either way. The unmeasured call stays on the session, so the session's total stays unmeasured and later runs stop before making any call, and raising the cap or adding `pricing` does not clear it: start a new session, or clear the cap. `set_agent_config` with `null` clears it until the next restart; to clear it for good, remove the key from the configuration file. A session created before v5.4.0, whose earlier calls recorded no usage, is in the same state. +- **Pending approvals are closed too.** A run stopped before it executes an approved action answers that action, and any approval still awaiting a decision, with `budget_exceeded`, so nothing is left pending. The action does not run later, even once the cap is raised; the model sees the refusal and can ask again. +- **A capped session stays capped.** Raise the cap with `set_agent_config` or start a new session. + +A cap is a threshold checked after each call, not a guarantee: the call that reaches it can overshoot it. Every call that returns is counted, including a reply the agent rejects as cut off or empty: that reply stays out of the transcript, but its usage is kept on the session. A call that fails, for example on a timeout or a provider error and whether or not another model then serves it, is not counted, even if the provider billed it. The caps bound one session only: there is no cross-session or per-user budget, so a caller allowed to run `agent_prompt` can keep spending by starting new sessions. + +A cap, or a `maxTurns`, that is not a usable number fails the agent's start: Harper logs the error and registers no agent operation. Through `set_agent_config` the same value is rejected with a 400. A bare `maxSessionTokens:` or `maxCostUsd:` with no value leaves that cap unset, as `null` does through `set_agent_config`; a bare `maxTurns:` fails, because `maxTurns` cannot be unset. + +If you upgrade with `maxCostUsd` already set in your configuration, it starts being enforced, and needs `pricing` on the generative entry the agent uses. + ### Restricting `http_fetch` diff --git a/reference/models/api.md b/reference/models/api.md index 29ffbf1d8..8f9e51a74 100644 --- a/reference/models/api.md +++ b/reference/models/api.md @@ -68,13 +68,14 @@ Additional options apply only when `toolMode: 'auto'`; they are documented in [T ### GenerateResult -| Field | Type | Description | -| -------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | -| `content` | `string` | The generated text | -| `finishReason` | `'stop'` \| `'length'` \| `'tool_calls'` \| `'content_filter'` | Why generation stopped, normalized across backends | -| `toolCalls` | `ToolCall[]` | Tool calls the model requested, when `finishReason` is `'tool_calls'` (each `{ id, name, arguments }`, with `arguments` parsed to an object) | -| `usage` | `TokenUsage` | Token usage reported by the backend (`promptTokens`, `completionTokens`, …), when available | -| `trace` | `ToolTraceEntry[]` | Per-tool-invocation trace; only populated by the `toolMode: 'auto'` loop — see [Tool Calling](./tool-calling) | +| Field | Type | Description | +| -------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `content` | `string` | The generated text | +| `finishReason` | `'stop'` \| `'length'` \| `'tool_calls'` \| `'content_filter'` | Why generation stopped, normalized across backends | +| `toolCalls` | `ToolCall[]` | Tool calls the model requested, when `finishReason` is `'tool_calls'` (each `{ id, name, arguments }`, with `arguments` parsed to an object) | +| `usage` | `TokenUsage` | Token usage reported by the backend (`promptTokens`, `completionTokens`, …), when available | +| `costUsd` | `number` | From v5.4.0, the call's cost from the serving entry's [pricing](./overview#pricing); absent when that entry is unpriced or usage is incomplete. With `toolMode: 'auto'`, the final round's only, like `usage` | +| `trace` | `ToolTraceEntry[]` | Per-tool-invocation trace; only populated by the `toolMode: 'auto'` loop — see [Tool Calling](./tool-calling) | ## generateStream() @@ -82,7 +83,7 @@ Additional options apply only when `toolMode: 'auto'`; they are documented in [T models.generateStream(input: GenerateInput, options?: GenerateOpts): AsyncIterable ``` -Identical to `generate()` but yields the completion incrementally: +Identical to `generate()` but yields the completion incrementally. Streamed calls are not [priced](./overview#pricing), so no chunk carries `costUsd`: ```javascript let text = ''; diff --git a/reference/models/backends.md b/reference/models/backends.md index 19b34e65b..be4b71886 100644 --- a/reference/models/backends.md +++ b/reference/models/backends.md @@ -26,6 +26,7 @@ All backends support these common fields: | `model` | Provider-side model identifier (e.g. `gpt-4o`) used when a call does not pass its own `model` option | | `requestTimeoutMs` | Per-request timeout in milliseconds; composed with any caller-provided `AbortSignal` | | `revision` | From v5.3.1, a non-secret label for the deployment behind the entry, for [calibration](./calibration#when-the-deployment-changes-but-the-configuration-does-not); change it when the same settings reach a different model | +| `pricing` | From v5.4.0, `generative` entries only: `{ inputPerMtok, outputPerMtok }`, what you pay in USD per million prompt and completion tokens. See [Pricing](./overview#pricing) | ## Ollama diff --git a/reference/models/overview.md b/reference/models/overview.md index 0e0ae29be..e389c3649 100644 --- a/reference/models/overview.md +++ b/reference/models/overview.md @@ -145,6 +145,30 @@ The logical name `default` is used when application code does not pass an explic See [Backends](./backends) for the full set of configuration fields supported by each backend. +### Pricing + + + +A `generative` entry can declare what you pay for its model, so Harper can put a dollar figure on each call. Harper ships no prices of its own: the rates are yours to keep current. + +```yaml +models: + generative: + default: + backend: openai + apiKey: ${OPENAI_API_KEY} + model: gpt-4o + pricing: + inputPerMtok: 2.5 # USD per million prompt tokens + outputPerMtok: 10 # USD per million completion tokens +``` + +Each `generate()` call is priced at the rates of the entry that served it, and the result carries `costUsd`. With `toolMode: 'auto'`, the result's `costUsd` covers the final round only, like its `usage`. A call that fell back to another entry in a [fallback group](./routing#fallback-groups) is priced at that entry's rates, not the requested one's. A call has no `costUsd` when the entry that served it declares no `pricing`, or when the provider did not report both a prompt and a completion token count. + +The two rates are the only cost inputs Harper uses. Discounts for cached prompt tokens, long-context surcharges and provisioned capacity are not, so set rates that match what an average call costs you, or rates that overestimate it if the figure backs a cap. Streaming calls (`generateStream()`) are not priced. + +The price is read by [`maxCostUsd`](./tool-calling#options) on `toolMode: 'auto'` calls and by the [built-in agent's `maxCostUsd`](../configuration/options#agent-spend-caps). `pricing` is accepted only on `generative` entries; on an `embedding` or `decision` entry it fails configuration validation. A price change applies on config reload, except on an entry whose `backend` is a module rather than a built-in, which takes it on restart like any other change to that entry. It does not affect [calibration](./calibration). + ### Credentials String values in model entries support environment-variable indirection with `${VAR_NAME}` syntax, resolved at startup. Use this for API keys rather than placing the literal key in the configuration file — Harper logs a warning at startup when a credential field contains a literal value. If the referenced environment variable is unset, the placeholder is left as-is; for credential fields the backend rejects the unresolved placeholder at startup, while other fields (such as `host` or `model`) carry the literal placeholder into requests — surfacing as per-request failures rather than a startup error. Indirection applies to string-typed fields only; numeric fields such as `requestTimeoutMs` must be literal values. diff --git a/reference/models/tool-calling.md b/reference/models/tool-calling.md index 0c0f1174e..553a6edb3 100644 --- a/reference/models/tool-calling.md +++ b/reference/models/tool-calling.md @@ -78,20 +78,21 @@ A handler receives the parsed arguments and a context object `{ signal, accounti All options below apply only with `toolMode: 'auto'`. -| Option | Type | Default | Description | -| -------------------- | ----------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `toolHandlers` | `Record` | — | Dispatch table keyed by tool name. A tool declared in `tools` with no handler here is a configuration error (400) | -| `maxToolIterations` | `number` | `10` | Hard cap on model → tools → model rounds | -| `maxToolTokens` | `number` | — | Cumulative prompt+completion token cap across rounds. Best-effort: requires the backend to report usage; if it does not, Harper warns once and `maxToolIterations` remains the bound. Not supported on `generateStream()` (throws) | -| `toolParallelism` | `'parallel'` \| `'serial'` | `'parallel'` | When one round requests multiple tool calls, run handlers concurrently or in order | -| `toolResultMaxBytes` | `number` | `65536` | Per-result byte cap (JSON-stringified). Larger results are truncated with a marker; the model sees the truncated form, the trace records the original size | -| `toolErrorMode` | `'recover'` \| `'abort'` | `'recover'` | `'recover'` feeds a handler error back to the model as the tool result so it can react; `'abort'` stops the loop and throws with the trace attached | -| `includeToolTrace` | `boolean` | `false` | Populate `result.trace` with one entry per tool invocation (iteration, name, arguments, result size, duration, error) | -| `conversation` | `ConversationAppender` | — | Optional persistence hook — see below | +| Option | Type | Default | Description | +| -------------------- | ----------------------------- | ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `toolHandlers` | `Record` | — | Dispatch table keyed by tool name. A tool declared in `tools` with no handler here is a configuration error (400) | +| `maxToolIterations` | `number` | `10` | Hard cap on model → tools → model rounds | +| `maxToolTokens` | `number` | — | Cumulative prompt+completion token cap across rounds. Requires the backend to report usage: from v5.4.0 a round without both token counts stops the loop with the budget error, where before Harper warned once and `maxToolIterations` remained the bound. Not supported on `generateStream()` (throws) | +| `maxCostUsd` | `number` | — | From v5.4.0, cumulative USD cap across rounds, from the [pricing](./overview#pricing) of the entry that served each round. A round that is unpriced stops the loop with the budget error, as the built-in agent's `maxCostUsd` stops rather than run unpriced. From v5.4.0 the caps are also checked before each round, so a cap of `0` makes no model call. Not supported on `generateStream()` (throws) | +| `toolParallelism` | `'parallel'` \| `'serial'` | `'parallel'` | When one round requests multiple tool calls, run handlers concurrently or in order | +| `toolResultMaxBytes` | `number` | `65536` | Per-result byte cap (JSON-stringified). Larger results are truncated with a marker; the model sees the truncated form, the trace records the original size | +| `toolErrorMode` | `'recover'` \| `'abort'` | `'recover'` | `'recover'` feeds a handler error back to the model as the tool result so it can react; `'abort'` stops the loop and throws with the trace attached | +| `includeToolTrace` | `boolean` | `false` | Populate `result.trace` with one entry per tool invocation (iteration, name, arguments, result size, duration, error) | +| `conversation` | `ConversationAppender` | — | Optional persistence hook — see below | ### Budgets and errors -When `maxToolIterations` or `maxToolTokens` is exhausted, the loop throws a budget-exceeded error (HTTP status 429) carrying a `partialTrace` of everything that ran — the trace is attached on error paths regardless of `includeToolTrace`. With `toolErrorMode: 'abort'`, a handler failure throws an error carrying the same trace. +When `maxToolIterations`, `maxToolTokens` or `maxCostUsd` is exhausted, or from v5.4.0 when a round leaves `maxToolTokens` or `maxCostUsd` unmeasurable, the loop throws a budget-exceeded error (HTTP status 429, its message saying which) carrying a `partialTrace` of everything that ran — the trace is attached on error paths regardless of `includeToolTrace`. With `toolErrorMode: 'abort'`, a handler failure throws an error carrying the same trace. If the model requests a tool name that was never declared in `tools` (a hallucinated tool), the call is treated as a tool error and routed by `toolErrorMode` — with `'recover'`, the model is told the tool is unknown and can correct itself. @@ -101,4 +102,4 @@ The `conversation` option accepts any object with an `append(turn)` method retur ### Reserved options -`toolArgValidation` (`'strict'` / `'lenient'` JSON Schema validation of tool arguments), `maxCostUsd`, and `conversationId` exist on the type surface but are not functional in 5.1 — the validation modes and streaming token budgets throw a `501` error, and the cost cap has no rate card behind it yet. Don't rely on them. +`toolArgValidation` (`'strict'` / `'lenient'` JSON Schema validation of tool arguments) and `conversationId` exist on the type surface but are not functional: the validation modes and streaming token and cost budgets throw a `501` error. Don't rely on them. Before v5.4.0, `maxCostUsd` had no prices behind it and never tripped. diff --git a/reference/operations-api/operations.md b/reference/operations-api/operations.md index 21815c98a..53d7b3de7 100644 --- a/reference/operations-api/operations.md +++ b/reference/operations-api/operations.md @@ -1855,9 +1855,9 @@ A session's `status` is one of: | `awaiting_approval` | Paused on one or more destructive tool calls; see `pendingApprovals` | | `completed` | The run ended with a final answer, or at the `maxTurns` ceiling; check `lastError` | | `aborted` | Cancelled by an operator via `cancel_agent_run` | -| `error` | The run failed; `lastError` carries the message | +| `error` | The run failed or a spend cap stopped it; `lastError` carries the message | -`completed` also covers hitting the `agent.maxTurns` ceiling — in that case `lastError` reads `Reached maxTurns= without a final answer.`, so check it before treating a completed session as finished. +`completed` also covers hitting the `agent.maxTurns` ceiling — in that case `lastError` reads `Reached maxTurns= without a final answer.`, so check it before treating a completed session as finished. From v5.4.0 a final answer that reached a [spend cap](../configuration/options.md#agent-spend-caps) also ends `completed` with `lastError` set, naming the cap. As of v5.4.0, a model reply that did not finish ends the run `error`, never `completed`, and `lastError` says why. That covers a reply cut off at [`agent.maxTokens`](../configuration/options.md#agent) or at the model's context window, one stopped by the provider's content filter or a model refusal, one that asks for tool calls of which none could be parsed, and one with neither text nor tool calls. Nothing from that reply is added to `messages`: its text is dropped, and its tool calls are neither run nor queued for approval. The session can be prompted again. Earlier versions ended such a run `completed`, often with an empty final message, and ran a cut-off tool call with its cut-off arguments. @@ -1894,12 +1894,34 @@ Response: { "session_id": "3f7c...", "status": "running" } ``` -A session that is `running` or `awaiting_approval` rejects a new prompt with a 409 — resolve the pending approval or cancel the run first. +A session that is `running` or `awaiting_approval` rejects a new prompt with a 409 — resolve the pending approval or cancel the run first. From v5.4.0 a session also answers 409 while a cancelled run is still finishing its current model call or tool. Nothing is recorded, and the session already reads `aborted`, so send the prompt again once [`get_agent_session`](#get_agent_session) on the node you prompted reports `runInFlight: false`, since each node reports only its own runs; the wait lasts as long as that call, at most the model's `requestTimeoutMs` or the tool's own run time. A prompt accepted while a [`cancel_agent_run`](#cancel_agent_run) for the same session arrives also answers 409. That prompt may already be in the transcript, where the next prompt's run sees it. + +From v5.4.0 each of these 409s carries a `code` and `retryable` beside `error`. `retryable: true` means nothing was recorded, so the same prompt can be sent again once the condition clears; `retryable: false` means the prompt was saved, and sending it again would add it twice: + +| `code` | `retryable` | When | +| ------------------------ | ----------- | ---------------------------------------------------------------------------- | +| `AGENT_SESSION_BUSY` | `true` | The session is `running` or `awaiting_approval` | +| `AGENT_RUN_FINISHING` | `true` | A cancelled run is still finishing; `runInFlight` turns `false` when it ends | +| `AGENT_PROMPT_CANCELLED` | `true` | A cancel arrived before the prompt was saved | +| `AGENT_PROMPT_NOT_RUN` | `false` | A cancel arrived after the prompt was saved, so no run started | ### `get_agent_session` Returns the full session record: `status`, `user` (the Operations API caller who created the session, falling back to `agent.user`), the `messages` transcript (user, assistant, and tool messages, including tool calls and their observations), `pendingApprovals`, `model`, `provider`, `createdAt`/`updatedAt`, and `lastError`. This is the polling endpoint for a run in flight. +From v5.4.0 each assistant message carries the `usage` its model call reported (`promptTokens`, `completionTokens`, and `costUsd` when the model is [priced](../models/overview.md#pricing)), and the response adds the session's totals: + +```json +{ + "session_id": "3f7c...", + "status": "completed", + "usage": { "calls": 3, "promptTokens": 41230, "completionTokens": 2210, "costUsd": 0.157 }, + "runInFlight": false +} +``` + +`calls` counts the model calls that returned, including a reply the agent rejected as cut off or empty. That reply is kept out of `messages`, and its usage is listed in the session's `rejectedCalls`. A failed call is not counted. A total is left out once any call in the session did not report it, such as an unpriced model for `costUsd`, so a missing total means unmeasured, not zero. The [spend caps](../configuration/options.md#agent-spend-caps) read the same totals. `runInFlight` is `true` while a run for the session is still executing on the node that answered, including a cancelled run finishing its current step. + ```json { "operation": "get_agent_session", "session_id": "3f7c..." } ``` @@ -1908,7 +1930,7 @@ Unknown `session_id` returns a 404. ### `list_agent_sessions` -Lists agent sessions. +Lists agent sessions, each with the same `usage` totals as `get_agent_session`. | Parameter | Type | Description | | --------- | ------- | ---------------------------------------- | @@ -1945,7 +1967,7 @@ When `agent.autoApprove` is off (the default), any tool call the agent makes to } ``` -Both decisions resume the loop: an approval executes the saved tool call, and a denial hands the refusal back to the model as an observation so it can adjust. Neither ends the run — use `cancel_agent_run` for that. If a single turn produced several gated calls, the session stays `awaiting_approval` until every one of them is resolved. Resolving an already-resolved approval is an error. +Both decisions resume the loop: an approval executes the saved tool call, and a denial hands the refusal back to the model as an observation so it can adjust. Neither ends the run — use `cancel_agent_run` for that. If a single turn produced several gated calls, the session stays `awaiting_approval` until every one of them is resolved. Resolving an already-resolved approval is an error. From v5.4.0 an approved action runs at most once: it is marked taken just before it runs, and if Harper stops after that and before recording its result, the next run answers that call `interrupted` with an unknown outcome instead of running it again. A tool call that needed no approval and was cut off the same way is answered `interrupted` too, so the session can continue. An approval still awaiting a decision stays pending. Whether a tool is treated as destructive at all is governed by `agent.allowDestructive`: when it is `false` (the default), destructive tools are removed from the agent's toolset entirely rather than gated. Which tools carry that mark is fixed in core — `write_file`, the inspector's code-evaluation tools, and the operations on MCP's [curated destructive set](../mcp/tool-metadata.md) — so an operation outside it (`drop_component` and `deploy_component` among them) is neither removed nor gated. @@ -1953,6 +1975,8 @@ Whether a tool is treated as destructive at all is governed by `agent.allowDestr Cancels a session's run, clears any followups it scheduled, and marks the session `aborted`. Works on a paused (`awaiting_approval`) or `idle` session as well as an actively running one. +From v5.4.0 the cancel also answers every pending approval, decided or not, with a `cancelled_by_operator` result, in the same write that marks the session `aborted`. An approved action that has not started by then never runs, neither in the cancelled run nor when you next prompt the session, and nothing is left awaiting a decision. An approved action that had already started is not answered by the cancel: if it finished, its result is recorded, and if Harper stopped before recording one, the next run answers it `interrupted`. A cancel between a turn's tool calls answers the rest of that turn `cancelled_by_operator`. The session stays `aborted`. A run caught mid-step finishes that step without marking the session `completed`, `error` or `awaiting_approval`. A tool call it then asks to have approved is answered `cancelled_by_operator` instead, and the run makes no further model call. + ```json { "operation": "cancel_agent_run", "session_id": "3f7c..." } ``` @@ -1971,7 +1995,7 @@ One gap is worth knowing: changing `allowDestructive` with [`set_agent_config`]( -Updates agent settings and returns the resulting configuration. Accepts any of `enabled`, `provider`, `model`, `maxTurns`, `maxTokens`, `maxToolResultBytes`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend`; keys not supplied are left unchanged. Each field is described under [`agent`](../configuration/options.md#agent). A request that includes `httpFetch`, `componentsScope`, or `configScope` is rejected with a 400 and nothing in it is applied: the [`http_fetch` policy](../configuration/options.md#restricting-http_fetch) and the [filesystem scopes](../configuration/options.md#filesystem-scopes) are read at startup only. A `maxTokens` that is not a positive integer is rejected the same way. A `maxToolResultBytes` that is not an integer from `1024` to `1048576` is rejected the same way. +Updates agent settings and returns the resulting configuration. Accepts any of `enabled`, `provider`, `model`, `maxTurns`, `maxTokens`, `maxToolResultBytes`, `maxSessionTokens`, `maxCostUsd`, `autoApprove`, `allowDestructive`, and `systemPromptAppend`; keys not supplied are left unchanged. Set `maxSessionTokens` or `maxCostUsd` to `null` to remove that cap. Each field is described under [`agent`](../configuration/options.md#agent). A request that includes `httpFetch`, `componentsScope`, or `configScope` is rejected with a 400 and nothing in it is applied: the [`http_fetch` policy](../configuration/options.md#restricting-http_fetch) and the [filesystem scopes](../configuration/options.md#filesystem-scopes) are read at startup only. A `maxTurns` or `maxTokens` that is not a positive integer, or a cap that is not a usable number, is rejected the same way. A `maxToolResultBytes` that is not an integer from `1024` to `1048576` is rejected the same way. ```json { "operation": "set_agent_config", "autoApprove": false, "maxTurns": 20 } @@ -1980,7 +2004,7 @@ Updates agent settings and returns the resulting configuration. Accepts any of ` Three limits are worth knowing: - **The change is in-memory and not persisted.** It applies for the life of the process and is lost on restart; edit `harper-config.yaml` for a durable change. -- **A run already in flight keeps the settings it started with** — its toolset, `autoApprove`, `model`, `maxTokens`, `maxToolResultBytes`, and `systemPromptAppend` are all captured at start. Changes take effect on the next run, and a paused run that `approve_agent_action` resumes counts as a new one: it picks up the current settings. To stop a run immediately, use `cancel_agent_run`. +- **A run already in flight keeps the settings it started with** — its toolset, `autoApprove`, `model`, `maxTurns`, `maxTokens`, `maxToolResultBytes`, `systemPromptAppend` and spend caps are all captured at start. Changes take effect on the next run, and a paused run that `approve_agent_action` resumes counts as a new one: it picks up the current settings. To stop a run immediately, use `cancel_agent_run`. - **`enabled` is not a kill switch.** It cannot turn the agent on — if it was off at startup, this operation does not exist. Setting it to `false` only makes subsequent `agent_prompt` calls return 409; a run already in flight continues, and `approve_agent_action` still resumes a paused one. Use `cancel_agent_run` to stop a run. ### MCP access diff --git a/release-notes/v5-lincoln/5.4.md b/release-notes/v5-lincoln/5.4.md index e1f638196..aff7a881f 100644 --- a/release-notes/v5-lincoln/5.4.md +++ b/release-notes/v5-lincoln/5.4.md @@ -32,7 +32,7 @@ A durable MQTT session now resumes through the same checked replay. When a recon ### Cut-Off Agent Replies End the Run `error` -A model reply that did not finish now ends the agent session `error`, with the reason in `lastError`, instead of `completed`. That covers a reply cut off at the output-token cap or at the model's context window, one stopped by a content filter or a refusal, one that asks for tool calls of which none could be parsed, and an empty reply. Nothing from that reply is recorded, and its tool calls are neither run nor queued for approval. The new `agent.maxTokens` (default `16384`) is sent with every model request the agent makes; lower it for a model whose own output limit is smaller, or the provider rejects each request. See [`agent`](/reference/v5/configuration/options#agent) and [Agent](/reference/v5/operations-api/operations#agent). +A model reply that did not finish now ends the agent session `error`, with the reason in `lastError`, instead of `completed`. That covers a reply cut off at the output-token cap or at the model's context window, one stopped by a content filter or a refusal, one that asks for tool calls of which none could be parsed, and an empty reply. Nothing from that reply reaches the transcript, and its tool calls are neither run nor queued for approval. Its usage is still kept on the session, and counts toward the agent's spend caps. The new `agent.maxTokens` (default `16384`) is sent with every model request the agent makes; lower it for a model whose own output limit is smaller, or the provider rejects each request. See [`agent`](/reference/v5/configuration/options#agent) and [Agent](/reference/v5/operations-api/operations#agent). ### The `config` Scope Is the Config File @@ -42,6 +42,20 @@ The built-in agent's read-only `config` filesystem scope used to be the director The built-in agent added every tool result to its conversation in full, so one large result, such as a log read whole, could fill the model's context window and leave the session unusable. A tool result is now cut to `agent.maxToolResultBytes` (default `65536`) before it is stored, and `read_file` pages through a file by line, with no size limit, returning a cursor (`nextLine`, `nextOffset`) to continue from. When a provider still rejects a request as too long for the model, the agent cuts the most recent tool results to 2 KiB and retries once, and prompting a session left unusable by an earlier version usually recovers it the same way. The agent also stops sending its system prompt twice per request. See [When a conversation outgrows the model's context window](/reference/v5/operations-api/operations#when-a-conversation-outgrows-the-models-context-window). +### Spend Caps for the Built-in Agent + +The agent records the usage of each model call on its session, and `get_agent_session` and `list_agent_sessions` report a session's totals as `usage`. Two caps bound a whole session, across every run in it: the new `agent.maxSessionTokens`, and `agent.maxCostUsd`, which is now enforced from model pricing and no longer defaults to `5.00`. A reply the agent rejects, such as one cut off at `agent.maxTokens`, still counts toward a cap, though it stays out of the transcript. A session at a cap makes no further model call: it ends `error`, or `completed` with `lastError` naming the cap when the call that reached it was the final answer. A cap that cannot be measured stops the session rather than letting it run uncapped. Caps count calls that return: a call that fails, even one the provider billed, is not counted. If your configuration already sets `agent.maxCostUsd`, declare `pricing` on the generative entry the agent uses, and start new sessions: a session created before 5.4.0 recorded no usage, so it cannot be continued under a cap. See [Agent spend caps](/reference/v5/configuration/options#agent-spend-caps). + +### Agent Cancellation and Limits + +`cancel_agent_run` now answers every pending approval with a `cancelled_by_operator` result in the same write that marks the session `aborted`, so an approved action that has not started never runs, then or on the session's next prompt, and the session stays `aborted` even when a run was mid-step. The 409s `agent_prompt` returns for a busy session or a cancel now carry a `code` and `retryable`, so a client can tell a prompt it may resend from one already saved, and `get_agent_session` and `list_agent_sessions` report `runInFlight`. `agent.maxTurns` is now validated as a positive integer at startup and by `set_agent_config`, where a value such as `null` used to make every later run end without a model call; a bare `maxTurns:` in the configuration file now fails the agent's start. See [`cancel_agent_run`](/reference/v5/operations-api/operations#cancel_agent_run). + +## Models + +### Model Pricing and Per-Call Cost + +A `models.generative` entry can now declare `pricing: { inputPerMtok, outputPerMtok }`, what you pay in USD per million prompt and completion tokens. Harper ships no prices of its own. Each `generate()` call is priced at the rates of the entry that served it, a fallback included, and the result carries `costUsd` when that entry is priced and the provider reported both token counts. With `toolMode: 'auto'` it covers the final round, and streaming calls are not priced. This makes `maxCostUsd` work on `toolMode: 'auto'` calls; before, it had no prices behind it and never tripped. A `toolMode: 'auto'` round that leaves `maxCostUsd` or `maxToolTokens` unmeasurable, because the entry is unpriced or the provider reported no token counts, now stops the call with the budget error (429) instead of warning once and continuing, so set `pricing` on entries you cap by cost. See [Pricing](/reference/v5/models/overview#pricing). + ## Component Deploys ### Certifying a Release in a Canary Worker