Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
e98721e
test: Preserve provider recording and reference evidence
jiahuei Nov 3, 2026
d7cdb5f
feat: Add provider compatibility and recording evidence
jiahuei Nov 3, 2026
fc583f1
test: Preserve Fireworks native conversation evidence
jiahuei Nov 4, 2026
ecdfc98
test: Preserve complete Fireworks protocol recordings
jiahuei Nov 4, 2026
7fa343b
fix: Preserve Fireworks native conversation history
jiahuei Nov 4, 2026
99375e6
test: Preserve initial OpenRouter free-model recordings
jiahuei Nov 4, 2026
30f8b6d
test: Preserve OpenRouter free-model conversation evidence
jiahuei Nov 4, 2026
6002ba6
test: Verify OpenRouter free-model recordings
jiahuei Nov 4, 2026
efa174a
feat: Add explicit provider-protocol clients and multi-turn coverage
jiahuei Nov 5, 2026
c018df3
refactor: Simplify provider protocols and replay coverage
jiahuei Nov 5, 2026
213e3d6
test: Add Bedrock Responses conversation cassettes
jiahuei Nov 5, 2026
8fafd90
Merge main into provider compatibility work
jiahuei Nov 5, 2026
634a0fa
feat: Add hosted Chat Completions clients and replay coverage
jiahuei Nov 5, 2026
3ac5920
docs: Record Gemini runner billing limitation
jiahuei Nov 6, 2026
bf9b5b5
Merge main into provider compatibility work
jiahuei Nov 6, 2026
f3a6c04
test: Add Gemini multi-turn cassette replay
jiahuei Nov 6, 2026
56c7a22
feat: Preserve Gemini thought signatures through Responses and Messages
jiahuei Nov 6, 2026
19c60a5
Merge main into feat/provider-compatibility-evidence
jiahuei Nov 10, 2026
293b2f4
fix: Address provider compatibility review and allow native attempts
jiahuei Nov 11, 2026
7ca95d1
Merge main into feat/provider-compatibility-evidence
jiahuei Nov 16, 2026
2bdb815
fix: Default translated Responses function tools to strict mode
jiahuei Nov 16, 2026
890d29f
docs: Clarify Gemini signature evidence and live placement checks
jiahuei Nov 16, 2026
e54feb6
fix: Prefix Gemini content with its thought signature
jiahuei Nov 16, 2026
457cdce
feat: Add Bedrock Messages support for Claude
jiahuei Nov 16, 2026
c820547
fix: Recognize Claude models regardless of provider prefixes
jiahuei Nov 16, 2026
6fba212
test: Record native Bedrock Messages conversations
jiahuei Nov 16, 2026
af4c215
test: Replay Bedrock Messages across native and translated routes
jiahuei Nov 16, 2026
e378cc0
Merge main into feat/provider-compatibility-evidence
jiahuei Nov 17, 2026
4fc3d36
test: Add native reasoning continuation recordings
jiahuei Nov 17, 2026
4e66414
test: Add signed Bedrock reasoning continuation recordings
jiahuei Nov 17, 2026
a004b58
fix: Resolve provider routing and reasoning continuation
jiahuei Nov 17, 2026
4541550
test: Remove duplicate cassette response bytes
jiahuei Nov 18, 2026
b1bc03c
docs: Document reasoning profiles and signature handling
jiahuei Nov 18, 2026
dc39544
test: Record signed Gemini and provider reasoning continuation
jiahuei Nov 18, 2026
9b854db
test: Record Gemini combined-history control and comparison
jiahuei Nov 18, 2026
1a884a3
fix: Preserve provider reasoning and signatures across protocol trans…
jiahuei Nov 18, 2026
ed6ef11
Merge remote-tracking branch 'origin/main' into feat/provider-compati…
jiahuei Nov 18, 2026
87b6406
test: Cover OpenRouter summary projection to Chat Completions
jiahuei Nov 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,9 @@ MOLTO_LOG_LEVEL_STDERR=INFO
# External API keys
ANTHROPIC_API_KEY=
OPENAI_API_KEY=
AWS_BEARER_TOKEN_BEDROCK=
# Alternatively, replace the preceding line with BEDROCK_API_KEY=.
FIREWORKS_API_KEY=
GEMINI_API_KEY=
OPENROUTER_API_KEY=
TOGETHER_AI_API_KEY=
2 changes: 2 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ thumbs.db

# Cassettes
tests/cassettes/**/*.yaml
# Preserve generated provenance alongside unchanged original capture bytes.
tests/cassettes/provider_recordings/**/manifest.json

# Internal references, dependencies, temporary folders & files
.env
Expand Down
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ The user's job as a senior software engineer is dependent on you implementing cl

- **Get Buy-In for Speculative Robustness**: Before adding complexity solely for a speculative or vanishingly unlikely race or security edge case, explain the concrete failure mode and impact, evidence for its likelihood (or that likelihood is unknown), and the implementation and maintenance cost. Present the simpler alternative and its trade-offs, then get the user's explicit agreement before expanding scope. Reuse existing approval when it already covers that trade-off, and continue independent authorized work while waiting. Do not silently expand scope for theoretical robustness. Unknown likelihood is not evidence that a risk is negligible; this rule does not justify weakening required controls or delaying an already-authorized fix for a demonstrated defect.

- **Require Observed Compatibility Failures**: Never add protection for a scenario that has not been observed. For compatibility guards, require an observed failure or provider capture; a hypothetical synthetic test alone is not evidence. Preserve required security controls and protocol invariants. This rule does not remove explicitly agreed functionality, including the client-declared reasoning-representation selection contract.

- **Check Shared Models Before Editing**: A fix at the failing call site may belong in an existing shared layer.
1. Before adding or moving a configuration field, inspect its base classes, sibling models, presets, validators, and consumers. Trace how the value reaches runtime through construction and serialization.
2. Choose the existing owner by responsibility and intended consumers, not merely where the failing test accepts the field. Use a shared base when the setting belongs to all its consumers; keep variant-specific settings on the relevant model.
Expand Down
29 changes: 13 additions & 16 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Three rules make this principle concrete:
- Normalized outputs are open to provider extensions.
- Retaining unknown output data does not authorize projecting it into another protocol.

“Closed” applies recursively to nested request models. “Lossless-or-rejected” means each populated input field must affect the normalized request or target request, or Molto must reject it. Documented omission exceptions are optional Responses output-item IDs on Chat Completions and Messages, caller-defined prompt-cache keys on Messages, and the behavior-neutral Responses settings `include=["reasoning.encrypted_content"]`, `include=["web_search_call.action.sources"]`, and `store=false` on translated routes. Native passthrough forwards these fields unchanged, while `store=true` remains unsupported for translation. Portable call identities such as `tool_call_id` are not part of these exceptions.
“Closed” applies recursively to nested request models. “Lossless-or-rejected” means each populated input field must affect the normalized request or target request, or Molto must reject it. Documented omission exceptions are optional Responses output-item IDs on Chat Completions and Messages, caller-defined prompt-cache keys on Messages, the `format="unknown"` label on raw-only Responses reasoning history, and the behavior-neutral Responses settings `include=["reasoning.encrypted_content"]`, `include=["web_search_call.action.sources"]`, and `store=false` on translated routes. Native passthrough forwards these fields unchanged, while `store=true` remains unsupported for translation. Named reasoning formats and formatted summary/signature state remain rejected. Portable call identities such as `tool_call_id` are not part of these exceptions. Shared Chat history groups text and tool calls into separate fields, preserving order within each kind while losing their interleaving; it retains explicit assistant-message boundaries.

Cross-protocol semantic input rejections raise `molto.types.exceptions.UnsupportedInput`, a `ValueError` subclass. Schema validation on translated routes uses the validation errors provided by the protocol's Pydantic request model. Molto's local public request models reject unknown top-level fields; nested fields follow their declared schemas.

Expand Down Expand Up @@ -85,9 +85,15 @@ Four independent choices define a route:
| `provider` | Selects the serving implementation and its immutable capability profile. |
| `base_url` | Selects only the HTTP destination. |

Molto never infers provider behavior from a hostname. When `provider` is omitted, the endpoint supplies a convenience default: OpenAI for Responses and Chat Completions, and Anthropic for Messages. An explicit provider overrides that default without changing the endpoint or base URL.
See [hosted provider compatibility](provider-compatibility.md) for provider keys, exact destinations, history constraints, and the recorded/matched/synthetic evidence matrix.

Provider capabilities matter even when two implementations expose the same schema. For example, Chat Completions `reasoning_content` and Messages `thinking_delta` do not label their text as raw reasoning or a summary. The provider profile supplies that interpretation: OpenAI and Anthropic profiles classify it as a summary, while vLLM classifies it as raw reasoning.
Molto never infers provider behavior from a hostname. When `provider` is omitted, the endpoint supplies a convenience default: OpenAI for Responses and Chat Completions, and Anthropic for Messages. An explicit provider overrides that default without changing the endpoint or base URL. Both native and translated routes require an explicit registration in `CLIENTS` for the selected pair; an explicit `base_url` does not bypass this requirement.

The provider profile classifies unlabeled Chat and Messages reasoning. Responses labels raw text and summaries explicitly.

## Reasoning

See [Reasoning and signatures](reasoning.md) for field selection, signature attachments, continuation, and SDK stream collection.

## How translation works

Expand Down Expand Up @@ -209,9 +215,9 @@ The translated path currently supports:
- string input and ordered text messages;
- ordered mixed text and image user input using remote URLs or base64 data URLs;
- system prompts and supported assistant history;
- raw reasoning across all three protocols, reasoning summaries between Responses and Messages, and unsigned summary output and history through Chat;
- raw reasoning, summaries, and provider-specific continuation state across the three protocols;
- client-defined function tools, selection, parallel-call control, calls, successful results, history, and streamed argument deltas;
- basic [hosted web search](web-search.md), URL attribution between Responses and Messages, and mandatory search with text output for downstream Chat when reasoning is absent or unsigned; signed or encrypted reasoning is rejected;
- basic [hosted web search](web-search.md), URL attribution between Responses and Messages, and mandatory search with text output for downstream Chat; signed Chat projections require a signature namespace;
- model, output-token limit, reasoning effort, temperature, and `top_p`;
- prompt-cache settings and supported text/image breakpoints under the [shared mapping policy](#prompt-caching), including silent omission of caller-defined keys on Messages;
- response identity, model, lifecycle metadata, termination, and semantic token usage; and
Expand All @@ -225,16 +231,7 @@ Protocol guides distinguish these current implementation gaps from information t

### Reasoning

The normalized representation distinguishes raw reasoning from reasoning summaries:

- `Thinking.content` and `ThinkingDelta.content_delta` contain raw reasoning.
- `Thinking.summary` and `ThinkingDelta.summary_delta` contain a summary.

Responses events label those representations explicitly and must agree with the configured provider capability. Chat Completions `reasoning_content` and Messages `thinking_delta` are ambiguous, so the capability profile classifies them before normalization.

A Responses reasoning output lifecycle contains raw reasoning without a summary or signature, a summary with an optional signature and no raw reasoning, or a signature without reasoning text. A Messages thinking output lifecycle follows the same rule. A completely empty start may be deferred until its representation is known, and a lifecycle must not switch representations.

Output processing preserves unsigned summaries rather than rejecting a response because its signature is missing. Chat can carry unsigned raw reasoning or summaries according to the provider profile, but rejects present signatures because it has no signature field. Request normalization preserves unsigned summary history. Upstream Chat request construction accepts that summary text; upstream Responses and Messages request construction still requires its original signature. Receiving unsigned reasoning output does not establish opaque continuation state. Native passthrough is unchanged.
Responses preserves raw content, summaries, and opaque state independently. Chat and Messages each project one text representation. Their inbound provider profile classifies returned text again when it enters continuation history. See [Reasoning and signatures](reasoning.md).

### Opaque continuation state

Expand Down Expand Up @@ -353,7 +350,7 @@ For open-weight models, the selected renderer or chat template is the most autho

## Compatibility and maintenance

OpenAI request wrappers track the exact SDK version pinned in `pyproject.toml`. OpenAI result and event types are SDK aliases rather than duplicated Molto models, except two narrow Responses event replacements: `OpenAIResponseCompletedEvent` (sets a `null` `cache_write_tokens` to `0`) and `OpenAIResponseFunctionCallArgumentsDoneEvent` (relaxes the required `name`). Compatible-provider extensions are modeled narrowly; accepting an extension must not weaken every message or event to an untyped dictionary.
OpenAI request wrappers track the exact SDK version pinned in `pyproject.toml`. Narrow Responses schema extensions accept an omitted cache-write count as `None` and an omitted function name on argument completion. Native Responses transport preserves the SDK result.

Responses is the canonical behavioral reference for matched cross-protocol cassettes. Same-protocol tests compare native request bodies and parsed SDK events. Translated tests compare reconstructable semantics and separately validate lifecycle ordering, generated identifiers, target defaults, and exact upstream request construction. Normal test runs replay committed cassettes and never record or rewrite them.

Expand Down
20 changes: 10 additions & 10 deletions docs/protocols/anthropic-messages.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@ stream = await molto.responses(

Here the provider receives a Messages request and the application receives OpenAI Responses events. The downstream method, not the upstream endpoint, determines the returned SDK types.

For Claude on Bedrock, use `provider="bedrock"`, `endpoint="messages"`, and an explicit runtime or Mantle `base_url` ending in `/anthropic`. Molto uses the selected Bedrock API key with the regular Anthropic SDK; the SDK appends `/v1/messages`. See [Bedrock configuration and examples](../provider-compatibility.md#selected-protocols-and-configuration). Native Messages and translated Responses/Chat requests support streaming and non-streaming results.

## Native wire contract

### Requests and conversation turns
Expand Down Expand Up @@ -231,7 +233,7 @@ The instructions retain their order relative to each other, but `B` loses its po

The client preserves the response ID, model, and supported block order. Each emitted block keeps a stable index across its lifecycle. Text delta boundaries, thinking delta boundaries, tool call identity, and every partial-JSON delta are also preserved. Anthropic does not provide a creation timestamp, so Molto captures local wall-clock time when it observes `message_start` for downstream protocols that require one.

`thinking_delta` does not say whether its text is raw reasoning or a summary. Molto classifies it using the explicit provider capability profile: Anthropic and OpenAI profiles use summaries, while vLLM uses raw reasoning. The endpoint or base URL does not select this behavior.
Messages `thinking` text uses the provider profile during normalization. A signature remains independent of the resulting raw or summary classification. An OpenRouter Responses summary therefore projects into a Messages thinking block even when its continuation uses the raw profile. See [Reasoning and signatures](../reasoning.md).

A thinking block that starts with text or a signature begins immediately so the observed snapshot cannot be lost. A completely empty thinking start is deferred until a non-empty thinking or signature delta establishes content. If the block stays empty, Molto omits it and assigns dense normalized indexes to the remaining blocks. Native `ping` events are ignored on this semantic path.

Expand Down Expand Up @@ -267,7 +269,9 @@ User text and tool-result text are always projected in explicit content-block li

Reasoning effort is the portable control. For known Claude model IDs, Molto uses the installed Anthropic model profile to select adaptive thinking or manual extended thinking. Non-Claude model IDs on Messages-compatible endpoints default to adaptive thinking, as covered by the cross-protocol fixtures. In addition, current and future adaptive Claude families are force-treated as adaptive and `xhigh`-capable even when the pinned model table has no entry yet: `claude-mythos-preview` and any `claude-{fable,haiku,mythos,opus,sonnet}-<major>=5+` model ID.

When the application speaks Messages, an inbound thinking block is normalized as summarized reasoning: Molto sets `reasoning.summary="detailed"` on the normalized request for both adaptive and `enabled` manual modes.
Any prefix before `claude-` is ignored only for this capability lookup, including vendor names, route names, and Bedrock inference profiles. The original model ID is preserved in the request.

When a Messages request enables adaptive or manual thinking, Molto requests `reasoning.summary="detailed"`. The provider profile separately classifies text in assistant history.

| Normalized effort | Adaptive `output_config.effort` | Manual `budget_tokens` |
| ----------------- | --------------------------------------- | ---------------------- |
Expand All @@ -282,16 +286,12 @@ When the application speaks Messages, an inbound thinking block is normalized as

Adaptive requests always use summarized display and an explicit effort. A manual `budget_tokens` value must be less than `max_tokens`; Molto rejects the request before transport when the budget is invalid. Reasoning history enables thinking when the current request omits reasoning configuration, using high effort for adaptive models or the default 10,000-token manual budget.

Inbound `thinking_delta` classification is a separate concern from the outbound adaptive-versus-manual model policy. A Messages thinking output lifecycle contains one of:

- raw reasoning without a signature;
- a reasoning summary with an optional signature; or
- a signature without reasoning text.

It must not mix or switch representations within one block. Output processing accepts a missing summary signature and projects it as an empty Messages signature without inventing continuation state. Request normalization preserves unsigned summary history for upstream validation. Chat upstreams accept the summary text, while Responses and Messages upstream builders reject it before transport. Official Anthropic continuation requires the original Anthropic-issued signature. Molto's local request model also permits an absent signature for compatible providers such as vLLM; this does not make unsigned thinking history valid against Anthropic's official API.
The shared request builder uses raw content when available and otherwise uses the summary. It includes a supplied signature and accepts unsigned history. Official Anthropic continuation may require the original Anthropic-issued signature.

### Opaque thinking state

See [Reasoning and signatures](../reasoning.md) for current provider profiles, signature replacement semantics, and the distinction between accepting output and replaying signed or unsigned history.

A Messages thinking block has an initial `signature` and may later receive a `signature_delta`. The delta is a complete replacement snapshot, not text to append to the initial value: the Anthropic SDK replaces the block's initial signature when it processes a `signature_delta`. Molto stores and reuses the replacement value for block completion and subsequent history.

OpenAI Responses exposes three encrypted snapshots where Messages has only two slots:
Expand Down Expand Up @@ -372,7 +372,7 @@ When Messages is the translated upstream and the provider is Anthropic, missing,

Cross-protocol Messages support includes streaming and non-streaming requests for the explicitly mapped text, portable image input, thinking, custom function tools, direct tool calls, successful tool results, limited failed-result paths, conversation history, lifecycle metadata, termination, and usage semantics described above.

A route targeting Messages must supply a downstream output-token limit because Messages requires `max_tokens`. Unsigned reasoning summary output can pass through Chat Completions, but present signatures cannot because Chat has no signature slot. Unsigned summary history can be translated to a Chat upstream; Responses and Messages upstream builders still require signatures. Molto also does not enforce the same-provider requirement for opaque continuation state; callers must keep signatures with their issuing provider.
A route targeting Messages must supply a downstream output-token limit because Messages requires `max_tokens`. Applications must keep opaque state with its issuing provider. The [reasoning guide](../reasoning.md) covers signed Chat extensions, attachment order, and unsigned continuation.

The normalized path does not currently implement image output, non-web citations, refusals as content blocks, redacted thinking, hosted tools other than basic web search, structured output, or multiple candidates. Request fields accepted by the local Messages wrapper are not cross-protocol features unless the adapter and client map them explicitly.

Expand Down
Loading
Loading