Repository navigation
fix(llm): carry usage and finish_reason on structured output failures - #256
Merged
Merged
Conversation
added 3 commits
November 10, 2026 15:55
A structured output call that failed after the provider answered (a reply cut by the output limit, a refusal, a reply that is not JSON, a missing tool call) dropped the reply's usage and finish reason. A truncated reply surfaced as invalid JSON, and its tokens were not recorded. The error builder takes the received response's usage and finish reason, and every error built afterwards carries them in details.tokens and details.finish_reason. Each structured output driver reports a reply cut by the output limit as model_error with finish_reason length before decoding it. A failure without a provider reply carries no tokens. The rejected usage record holds the failed reply's tokens and finish reason.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Structured-output failures after a provider reply lose the reply's usage and finish reason. This change adds
details.tokensand normalizeddetails.finish_reasonto those errors and records the reported tokens and finish reason through the existing rejected-usage tracker. Failures without a provider reply add neither field. Error metadata extraction validates provider field types and omits unusable fields. Google retains its existing protected response-mapping boundary.All five providers preserve their existing success/failure outcomes, error kinds, retryability, exact messages and existing details. Parseable JSON or tool input with a length stop reason retains its existing behavior, including Claude native structured mode's existing rejection. Existing error details and caller metadata retain precedence when a key already exists. The README explains detecting truncation through
details.finish_reason == "length".Validation uses runtime v0.3.59a and refreshed dependencies:
origin/masterfor outcomes, exact messages, kinds, retryability and existing details. Additive metadata checks cover all five providers, malformed usage and finish-reason fields, nested usage details, malformed Google envelopes and transport failures. Success cases also compare tokens, finish reason and metadata with the base branch.