diff --git a/docs/README.md b/docs/README.md index a37b78b..78a9ef3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,7 +25,7 @@ Cross-protocol semantic input rejections raise `molto.types.exceptions.Unsupport Native same-protocol routes preserve request dictionaries, including unknown properties, nested extensions, and explicit nulls, without validating them against Molto's protocol models. The SDK and upstream provider own request validation. Explicit `extra_body` overrides take precedence over fields supplied in `body`. Molto still enforces the destination restriction on its own [Anthropic search metadata extension](web-search.md#preserving-anthropic-search-history). -Native coordinators select streaming only when `body["stream"]` is the boolean `True`. Other values are forwarded unchanged for SDK and provider validation; values such as `1` or `"false"` do not select Molto's streaming path. +Native coordinators select streaming when `body["stream"]` is truthy, the same rule the OpenAI and Anthropic SDKs use (`stream=stream or False`), so Molto always expects the response type the SDK returns. The value is forwarded unchanged for provider validation; `1` or `"false"` therefore streams, and a provider that rejects the value raises its own error. Each coordinator accepts a dictionary or its corresponding Molto request model. Serialize caller-defined Pydantic models to dictionaries before passing them to Molto. On native routes, Molto request models use their normal `model_dump()` behavior, which omits unset and `None`-valued fields. Model serialization can discard unknown fields, including extras accepted through `extra="allow"`; use a dictionary to preserve unknown fields and explicit nulls. SDK models nested in native request dictionaries retain their SDK serialization behavior. @@ -74,6 +74,10 @@ Use the same protocol for the method and `endpoint` to select native passthrough `Molto` owns its shared `httpx.AsyncClient`. A returned stream releases its response when its context exits, including after early termination. Closing `Molto`, either with `async with` or `await molto.aclose()`, closes the shared HTTP client. +## Observability + +Molto emits one OpenTelemetry span per coordinator call, using only `opentelemetry-api`; the application configures export. The [observability contract](observability.md) defines the span, its attributes and events, and its content policy. + ## Routing Four independent choices define a route: diff --git a/docs/observability.md b/docs/observability.md new file mode 100644 index 0000000..f2ad606 --- /dev/null +++ b/docs/observability.md @@ -0,0 +1,187 @@ +# Observability + +Molto emits one OpenTelemetry span for each coordinator call: `Molto.responses()`, `Molto.chat_completions()`, or `Molto.messages()`. The span shows what Molto did with the call: the route it chose, the downstream and upstream protocols it translated between, the provider, token usage, finish reasons, and why a call failed. + +Molto depends only on `opentelemetry-api`. It obtains its tracer with `trace.get_tracer("molto", __version__)` and never configures a tracer provider, exporter, sampler, or resource. With no SDK configured, the API returns no-op spans and calls behave identically. + +Spans for the application's own work, such as request handling, authentication, guardrails, or billing, belong to the application, not to Molto. Molto does not emit metrics or log events, and does not wrap the OpenAI or Anthropic SDKs with third-party instrumentors. + +## Enabling export + +The application owns the OpenTelemetry SDK and its configuration: + +1. Install `opentelemetry-sdk` and an exporter, such as `opentelemetry-exporter-otlp-proto-http`. +2. Configure a tracer provider once, at application startup. +3. Configure the exporter and resource with the standard `OTEL_*` environment variables, such as `OTEL_SERVICE_NAME` and `OTEL_EXPORTER_OTLP_ENDPOINT`. + +```python +from opentelemetry import trace +from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import BatchSpanProcessor + +provider = TracerProvider() +provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter())) +trace.set_tracer_provider(provider) +``` + +### HTTP child spans + +Molto calls providers through the OpenAI and Anthropic SDKs over `httpx`, and does not create HTTP spans itself. To trace the upstream HTTP requests, install `opentelemetry-instrumentation-httpx` and instrument `httpx`: + +```python +from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor + +HTTPXClientInstrumentor().instrument() +``` + +Global instrumentation wraps `httpx`'s network transport, so it covers Molto's default client and any injected `httpx.AsyncClient` that uses that transport, including clients created before `instrument()` is called. It does not see a client built on a custom transport, such as `httpx.MockTransport`. + +To instrument only the client Molto uses, or a client with a custom transport, pass an instrumented `httpx.AsyncClient` instead: + +```python +from httpx import AsyncClient +from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor + +from molto import Molto + +http_client = AsyncClient() +HTTPXClientInstrumentor.instrument_client(http_client) +molto = Molto(http_client=http_client) +``` + +The SDK call runs while the Molto span is current, so the HTTP spans nest under it, on native and translated routes, streaming and non-streaming. + +An `httpx` client span ends when the response headers arrive, not when a stream finishes. For a streaming call it measures only the time to first byte. The Molto span covers the whole generation, and its `molto.first_token` event records the time to first token. + +## Span contract + +The span follows the [GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/client-inference.md) for inference spans. Those conventions have Development status, so attribute keys may still change upstream. + +| Property | Value | +| -------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| Kind | `CLIENT` | +| Name | `{gen_ai.operation.name} {gen_ai.request.model}`, for example `chat gpt-5`; `{gen_ai.operation.name}` when the request has no model | +| Status | `UNSET` on success and on cancellation; `ERROR` only on a genuine failure | +| Parent | The span active when the coordinator is awaited | + +Molto creates no separate spans for adapters, normalizers, upstream clients, or translation. Translation runs interleaved with streaming, event by event, rather than as a separate phase, so a translation span would not isolate its cost. + +### Operation name + +`gen_ai.operation.name` is `chat` for all three coordinators, so a Messages call is also named `chat {model}`. The attribute holds the cross-vendor operation type, multi-turn generation, not Molto's protocol name. Observability backends recognize LLM spans by its well-known values, so a custom value such as `responses` or `messages` would leave Molto spans unrecognized. The protocols are carried by `molto.downstream.protocol` and `molto.upstream.endpoint`. + +### Streaming lifetime + +A streaming coordinator returns a `MoltoStream` after prefetching the first event. The span outlives that `await`: it ends exactly once, when the stream is exhausted, fails, is closed with `aclose()`, or is garbage-collected. A failure during prefetch ends the span in the coordinator. + +A stream that is neither consumed nor closed holds its upstream HTTP response until it is garbage-collected. `MoltoStream` cannot await `aclose()` then, but the event loop's async-generator finalizer closes the dropped source on its next turns, which releases the response. Use `async with` or `aclose()` to release it as soon as the caller is done. A dropped stream ends its span when it is garbage-collected, with `molto.stream.outcome=abandoned`, at the time of the last event the caller received rather than the collection time. CPython collects an object as soon as its last reference goes, later when it is part of a reference cycle, and possibly never at interpreter shutdown, so an abandoned span, and the release of its response, can come late or not at all. + +Every streaming span records how the stream ended in `molto.stream.outcome`: + +| Value | When | +| --- | --- | +| `completed` | The caller read the stream to its end. Closing it afterwards keeps `completed`. | +| `closed` | The caller called `aclose()` before the end, for example by leaving `async with` early. | +| `cancelled` | The caller's task was cancelled, while reading the stream or before the first event arrived. This is how a client hanging up appears. The status stays `UNSET` and the cancellation is re-raised. Any other exit that is not an `Exception`, such as `KeyboardInterrupt`, also reports `cancelled`, because it too is the caller stopping the call rather than the call failing. | +| `error` | The call failed, while the caller read the stream or before the first event. `molto.error.phase` says which, and the [error attributes](#errors) describe the failure. | +| `abandoned` | The caller dropped the stream without consuming or closing it. The span ends at the last event received; its upstream response stayed open until garbage collection. | + +Non-streaming spans do not carry the attribute. + +The span starts in the caller's context, so iterating the stream from another task does not reparent it. The Molto span is current only while the upstream request is sent: for the whole call when non-streaming and for the prefetch when streaming. It is not current while the caller iterates the stream. + +### Attributes known at start + +Molto resolves these values before transport. + +| Key | Value | +| --- | --- | +| `gen_ai.operation.name` | `chat` | +| `gen_ai.provider.name` | Resolved `provider`: `openai`, `anthropic`, or `vllm` | +| `gen_ai.request.model` | Request `model` | +| `gen_ai.request.max_tokens`, `gen_ai.request.temperature`, `gen_ai.request.top_p` | Request fields, only when set with the expected type | +| `gen_ai.request.top_k` | Request `top_k`, only when set as an integer | +| `gen_ai.request.frequency_penalty`, `gen_ai.request.presence_penalty` | Request fields, only when set as numbers | +| `gen_ai.request.seed` | Request `seed`, only when set as an integer | +| `gen_ai.request.choice.count` | Request `n`, only when set as an integer other than `1` | +| `gen_ai.request.stream` | Set only when the coordinator streams the call | +| `server.address`, `server.port` | Host and port of the effective `base_url`; the scheme's default port when the URL has none | +| `molto.downstream.protocol` | `responses`, `chat/completions`, or `messages`, from the coordinator | +| `molto.upstream.endpoint` | Effective `endpoint` | +| `molto.route` | `native` or `translated` | + +Native request bodies reach the SDK unvalidated, so a request field becomes an attribute only when it has the convention's type: a non-empty string model, an integer output-token limit, `top_k`, `seed`, and `n`, and numeric `temperature`, `top_p`, and penalty values, reported as doubles. Booleans do not count as numbers. `stream` follows the coordinator's own rule: native passthrough streams for any truthy value, as the OpenAI and Anthropic SDKs do, while translated routes validate it, streaming for `true`, `1`, or `1.0` and rejecting strings. Chat Completions reports `max_completion_tokens`, falling back to the deprecated `max_tokens`. A `base_url` without a host, or one `httpx` cannot parse, omits the server attributes and leaves the error to the SDK. + +Molto's request types define `top_k` only on Messages, and `seed`, `n`, and the penalties only on Chat Completions. Native passthrough forwards fields its request type does not define, so these attributes are read from any native body as sent: a native vLLM Chat request with `top_k`, for example, reports it. Translation cannot express these fields, so a translated request that sets one is rejected with `UnsupportedInput`. Its span still carries the attribute, because the span starts before translation, so the rejected call shows the value that caused it. + +The conventions also define `gen_ai.request.stop_sequences`. Molto does not record it: stop sequences are caller-written text, and spans carry no request content (see [Content and privacy](#content-and-privacy)). + +`vllm` is not a well-known `gen_ai.provider.name` value; the conventions allow a custom value for other providers. Molto has no separate `molto.provider` attribute, because it would duplicate this value and `server.address` already identifies the host. + +### Attributes known at end + +| Key | Translated route | Native route | +| --- | --- | --- | +| `gen_ai.response.id` | `ResponseStartEvent` | SDK response or terminal event | +| `gen_ai.response.model` | `ResponseStartEvent` | SDK response or terminal event | +| `gen_ai.response.finish_reasons` | `ResponseEndEvent.termination_reason` | SDK finish or stop reason; for Responses, the incomplete reason or terminal status, including `failed` | +| `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | `ResponseEndEvent` normalized usage | SDK usage | +| `gen_ai.usage.cache_read.input_tokens`, `gen_ai.usage.cache_write.input_tokens` | Normalized usage details, when non-zero | SDK usage details, when non-zero | +| `gen_ai.usage.reasoning.output_tokens` | Normalized usage details, when non-zero | SDK usage details, when non-zero | +| `molto.usage.web_search_requests` | Normalized usage `web_search_requests`, when non-zero | Responses `tool_usage.web_search.num_requests` or Messages `usage.server_tool_use.web_search_requests`, when non-zero; Chat Completions reports none | +| `error.type`, `http.response.status_code`, `molto.error.phase` | See [Errors](#errors) | See [Errors](#errors) | +| `molto.stream.outcome` | Streaming only; see [Streaming lifetime](#streaming-lifetime) | Same | + +Translated spans report Molto's normalized termination reason, such as `stop`, `max_output_tokens`, or `tool_call`. Native spans report the provider's own value, such as Chat Completions `stop` or `length` and Messages `end_turn`. The same call can therefore report different finish reasons on its two routes: an Anthropic call that ends normally reports `["end_turn"]` natively and `["stop"]` through a translated route. Group by `molto.route` to compare them. Responses has no finish-reason field, so a native Responses span reports `incomplete_details.reason` for an incomplete response, such as `max_output_tokens`, and the terminal `status`, such as `completed` or `failed`, otherwise. A native Responses stream records its terminal response from `response.completed`, `response.incomplete`, or `response.failed`. The SDK raises only on a separate `error` event, so a stream that ends in `response.failed` without one returns normally: its span reports `["failed"]` with `molto.stream.outcome=completed` and `UNSET` status, as the same non-streaming response would. + +Normalized usage stores a missing detail count as zero, so the cache and reasoning attributes appear only when non-zero, on every route, as the conventions recommend. Native reasoning counts come from Responses `output_tokens_details.reasoning_tokens`, Chat Completions `completion_tokens_details.reasoning_tokens`, and Messages `output_tokens_details.thinking_tokens`. The OpenAI and Anthropic APIs already include reasoning tokens in their output-token counts, as the conventions require for `gen_ai.usage.output_tokens`, so Molto reports both counts unchanged. Native cache counts come from Responses `input_tokens_details.cached_tokens` and `cache_write_tokens`, Chat Completions `prompt_tokens_details.cached_tokens` and `cache_write_tokens`, and Messages `cache_read_input_tokens` and `cache_creation_input_tokens`. + +The conventions define no web-search or server-tool usage count, so `molto.usage.web_search_requests` is Molto's own key. It counts the search requests the provider reports. A translated span reports it only when the upstream reports it: a Chat Completions upstream never does. + +Molto reports the provider's token counts. The one adjustment is native Messages input: the Anthropic SDK `input_tokens` excludes cached tokens, so a native Messages span reports `input_tokens + cache_creation_input_tokens + cache_read_input_tokens`. This matches the normalized usage a translated route reports for the same call. + +`gen_ai.usage.cache_write.input_tokens` is the current spec key. Molto defines attribute keys as its own string constants rather than importing `opentelemetry-semantic-conventions`, which as of 0.66b2 defines only the older `gen_ai.usage.cache_creation.input_tokens`. A backend built on that package may not recognize the new key yet. + +A native Chat Completions stream reports usage only when the caller sets `stream_options.include_usage`. Molto never injects that option, because native passthrough forwards the request unchanged, so these spans omit the usage attributes when the stream reports none. + +### Errors + +On failure, the span status is `ERROR`, the exception is recorded with `record_exception`, and these attributes are set: + +| Key | Value | +| --- | --- | +| `error.type` | The normalized `APIException` subclass name when the exception or its `__cause__` is one, for example `ContextLengthExceededError`; otherwise the raw exception class name, for example `RateLimitError` or `UnsupportedInput` | +| `http.response.status_code` | Set only from a real upstream error response with status 400 or higher: the normalized cause's `response` when a Molto cause exists, otherwise the status of an SDK `APIStatusError`. Omitted when no such response exists | +| `molto.error.phase` | `setup` when the failure is raised before the coordinator returns, such as a local rejection, credential error, or HTTP error; `stream` when it is raised after the coordinator returned a stream, while the caller iterates or when `aclose()` fails | + +For a recognized upstream failure, the caller receives a downstream SDK exception whose `__cause__` is the normalized Molto exception, so `error.type` checks `__cause__` rather than reporting the caller-specific SDK class. Native and translated routes therefore report the same `error.type` for the same upstream failure. Unknown errors pass through unchanged and have no Molto cause, and `UnsupportedInput` is raised directly, so their own class is the `error.type`. A raw class name such as `RateLimitError` exists in both SDKs; `molto.upstream.endpoint` identifies which SDK raised it. See [Errors](errors.md) for the cause chain and recognition rules. + +`http.response.status_code` is never synthesized. A missing-credentials error carries a synthetic `401` on the outward exception, but no HTTP call occurs and its normalized cause has no response, so the attribute is omitted. A Responses context-length failure during streaming arrives on an HTTP `200` response; the attribute is omitted there too, and `error.type` and `molto.error.phase` describe the failure. + +The exception is recorded as one `exception` event without its message, because exception messages can contain request content: a rejected request's validation error quotes its input, and an upstream error can echo a request field. The event's `exception.message` is the exception's class name, and its `exception.stacktrace` lists the class and stack frames of the exception and each `__cause__` in its chain, without any messages. Frame lines show code locations, never runtime values. The type, frames, and error attributes identify the failure without the message, which never appears on an attribute, so `error.type` stays low-cardinality. + +### Events + +| Event | When | +| --- | --- | +| `molto.first_token` | Streaming only, at the first delta with non-empty content: text, a refusal, reasoning, or tool-call arguments. Translated routes use the first such `PartDeltaEvent`; native routes use the first such delta event of their protocol. | + +The event marks the first content delta rather than the first protocol event. Responses `response.created` arrives before any generation, so the first-event time would only repeat the `httpx` span's time to first byte. + +Content deltas are, per protocol: + +| Route | Content deltas | +| --- | --- | +| Translated | `TextDelta`; `ThinkingDelta` with reasoning or a summary; `ToolCallDelta` with arguments | +| Native Responses | `response.output_text.delta`, `response.refusal.delta`, `response.reasoning_text.delta`, `response.reasoning_summary_text.delta`, `response.function_call_arguments.delta`, `response.custom_tool_call_input.delta`, `response.mcp_call_arguments.delta` | +| Native Chat Completions | A choice delta with `content`, `refusal`, the compatible-provider `reasoning_content` extension, or tool-call `function.arguments` | +| Native Messages | `content_block_delta` with `text_delta`, `thinking_delta`, or `input_json_delta` | + +Empty deltas do not count, and neither do deltas that carry no generated content: a reasoning signature, which replaces opaque state, a tool call's name or ID alone, or a role. + +## Content and privacy + +Spans never carry prompts, completions, tool arguments, headers, or credentials. Content capture is not available; an opt-in would be a separate, explicitly approved change. + +Telemetry is data, not behavior. Span attributes are derived from values the coordinator already resolved or from events the pipeline already emitted. Instrumentation never changes routing, validation, request construction, projection, or errors, and it never reads the `extra_body` or `extra_headers` request extensions. Native passthrough does not validate upstream output, so a field a provider omits is left off the span rather than raising. If recording still fails on output no extractor expects, Molto drops those attributes, logs the exception class, without its message or traceback, at `DEBUG` level through Molto's `loguru` logger, and the call continues unchanged. diff --git a/pyproject.toml b/pyproject.toml index 29ecd25..84358d8 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -79,11 +79,17 @@ classifiers = [ # The openai floor is a hard import requirement, not a preference; # the ceilings are releases where tests start failing. # Re-derive from the suite before widening either bound; see AGENTS.md, "Dependency Versions". +# opentelemetry-api: the full suite passes on 1.36.0 with the matching SDK and httpx +# instrumentor; 1.28.0 (pydantic-ai-slim's floor) fails only tests of the test-only +# instrumentor's API. The <1.46 ceiling stops just above 1.45.2, the newest +# release the suite passes on; each SDK release requires the matching API version, so +# re-derive it with the suite when a new release appears. # The test extra pins the exact versions the suite is developed against. dependencies = [ "anthropic>=0.119.0,<0.124", "loguru~=0.7", "openai>=2.45.0,<3.2", + "opentelemetry-api>=1.36.0,<1.46", "orjson~=3.11", "pydantic-ai-slim[retries]~=1.104.0", "pydantic-settings~=2.13.1", @@ -100,6 +106,9 @@ test = [ "anthropic==0.121.0", "fastapi[standard]==0.135.4", "openai==2.53.0", + "opentelemetry-api==1.45.2", + "opentelemetry-instrumentation-httpx==0.66b2", + "opentelemetry-sdk==1.45.2", "pytest==8.4.2", "pytest-asyncio==1.4.0", "pytest-cov==7.1.0", diff --git a/python/ARCHITECTURE.md b/python/ARCHITECTURE.md index 3699b28..73fde1e 100644 --- a/python/ARCHITECTURE.md +++ b/python/ARCHITECTURE.md @@ -123,6 +123,10 @@ Adapters and upstream clients do not depend on each other. Protocol-specific coo Upstream clients normalize recognized SDK failures as `APIException`; downstream adapters project them into their SDK namespace. Both native and translated routes use this pipeline for streaming and non-streaming requests. See [Errors](../docs/errors.md) for recognition rules, status mappings, and special cases. +## Instrumentation + +The coordinator owns spans: only `core.py` creates and ends them, while adapters, normalizers, and upstream clients stay telemetry-free. Telemetry never influences routing or translation. `_traced` in `core.py` starts one span per coordinator call, and `MoltoStream` ends it for streams. The coordinators pass upstream output through `observe` and a `SpanRecorder` (`utils/telemetry.py`), which only read it to add end attributes and the `molto.first_token` event. `record_safely` logs and drops a recording failure, so telemetry never fails a call. The [observability contract](../docs/observability.md) defines the span and its rules. + ## Orchestration facade and coordinators `Molto` is expected to expose one coordinator method per downstream protocol: diff --git a/python/molto/core.py b/python/molto/core.py index 05efc9d..dfe8156 100644 --- a/python/molto/core.py +++ b/python/molto/core.py @@ -1,9 +1,14 @@ -from collections.abc import AsyncIterator, Mapping +from collections.abc import AsyncIterator, Awaitable, Callable, Mapping +from time import time_ns from typing import Any, Self from httpx import AsyncClient, Headers +from opentelemetry import trace +from opentelemetry.trace import Span, SpanKind, Status, StatusCode +from pydantic import BaseModel from molto.adapters.anthropic import AnthropicMessagesAdapter, CitationPolicy +from molto.adapters.base import BaseAdapter from molto.adapters.openai import OpenAIChatCompletionsAdapter, OpenAIResponsesAdapter from molto.configs import ENV_CONFIG from molto.providers import ( @@ -30,11 +35,37 @@ has_anthropic_search_metadata, ) from molto.upstream.anthropic import AnthropicMessagesClient +from molto.upstream.base import BaseClient from molto.upstream.openai import OpenAIChatCompletionsClient, OpenAIResponsesClient from molto.utils.io import request_to_dict +from molto.utils.telemetry import ( + MOLTO_STREAM_OUTCOME, + ErrorPhase, + ResolvedRoute, + SpanRecorder, + StreamOutcome, + chat_completions_response_attributes, + error_attributes, + exception_event_attributes, + get_tracer, + messages_response_attributes, + observe, + record_safely, + responses_response_attributes, + span_name, + start_attributes, +) VLLM_DUMMY_API_KEY = "EMPTY" +# Each endpoint's client, used to resolve the effective destination for span attributes +# exactly as the client will before transport. +_UPSTREAM_CLIENTS: dict[Endpoint, type[BaseClient]] = { + "responses": OpenAIResponsesClient, + "chat/completions": OpenAIChatCompletionsClient, + "messages": AnthropicMessagesClient, +} + _CREDENTIAL_HEADERS = frozenset( { "anthropic-api-key", @@ -98,8 +129,38 @@ def _upstream_headers(extra_headers: Mapping[str, str] | None) -> dict[str, str] return {name: value for name, value in headers.items() if name.lower() not in excluded} +def _failure_outcome(error: BaseException) -> StreamOutcome: + # Cancellation, and any other exit that is not an `Exception` (such as + # `KeyboardInterrupt`), is the caller stopping the call, not the call failing: it + # leaves the status UNSET and reports `cancelled` (docs/observability.md). + return "error" if isinstance(error, Exception) else "cancelled" + + +def _finish_span( + span: Span, + error: BaseException | None = None, + phase: ErrorPhase = "setup", + outcome: StreamOutcome | None = None, + end_time: int | None = None, +) -> None: + # Without an SDK, or when sampled out, the span is never exported, so skip + # building attributes and formatting stack traces for it. + if span.is_recording(): + if outcome is not None: + span.set_attribute(MOLTO_STREAM_OUTCOME, outcome) + if isinstance(error, Exception): + span.set_status(Status(StatusCode.ERROR)) + span.set_attributes(error_attributes(error, phase)) + span.record_exception(error, attributes=exception_event_attributes(error)) + span.end(end_time=end_time) + + class MoltoStream[EventT](AsyncIterator[EventT]): - """Replay prefetched events and manage the source stream lifecycle.""" + """Replay prefetched events, manage the source stream lifecycle, and end its span. + + A span attached with `attach_span` ends exactly once, recording how the stream + ended: on exhaustion, an error, `aclose()`, or garbage collection of a dropped stream. + """ def __init__( self, @@ -110,6 +171,10 @@ def __init__( self._events = events self._stream = stream self._closed = False + self._span: Span | None = None + # When the caller last received an event; an abandoned stream's span ends here. + # The prefetch has just received the first event. + self._last_event_ns = time_ns() self.response_headers = Headers(response_headers) def __aiter__(self) -> AsyncIterator[EventT]: @@ -119,8 +184,18 @@ async def __anext__(self) -> EventT: if self._closed: raise StopAsyncIteration if self._events: + self._last_event_ns = time_ns() return self._events.pop() - return await anext(self._stream) + try: + event = await anext(self._stream) + self._last_event_ns = time_ns() + return event + except StopAsyncIteration: + self._end_span() + raise + except BaseException as error: + self._end_span(error) + raise async def __aenter__(self) -> Self: return self @@ -134,8 +209,81 @@ async def aclose(self) -> None: self._closed = True self._events.clear() close = getattr(self._stream, "aclose", None) - if close is not None: - await close() + try: + if close is not None: + await close() + except BaseException as error: + self._end_span(error) + raise + # A stream already exhausted or failed has ended its span, so this records + # `closed` only for a stream the caller stopped before its end. + self._end_span(outcome="closed") + + def attach_span(self, span: Span) -> None: + """Make this stream end `span` when it is exhausted, fails, closes, or is dropped.""" + self._span = span + + def __del__(self) -> None: + # A dropped, unclosed stream must still be visible (docs/observability.md, + # "Streaming lifetime"). `aclose()` cannot be awaited here; the event loop's + # async-generator finalizer closes the dropped source and releases its response. + # The span ends at the last event so its duration is not inflated by when + # garbage collection happens to run. + if getattr(self, "_span", None) is None: + return + try: + self._end_span(outcome="abandoned", end_time=self._last_event_ns) + except Exception: + # Exceptions in `__del__` are only printed, and at interpreter shutdown the + # SDK or exporter may already be gone; telemetry must not add noise here. + pass + + def _end_span( + self, + error: BaseException | None = None, + outcome: StreamOutcome = "completed", + *, + end_time: int | None = None, + ) -> None: + # Exhaustion, an error, `aclose()`, and garbage collection can all reach here; + # end the span once. A failure decides the outcome over the given one. A failing + # `aclose()` also reports phase `stream`: it is raised after the stream returned. + if self._span is not None: + span, self._span = self._span, None + final = _failure_outcome(error) if error is not None else outcome + _finish_span(span, error, "stream", final, end_time) + + +async def _run_native[EventT, ResponseT: BaseModel]( + adapter: BaseAdapter[Any, EventT, ResponseT], + upstream: BaseClient[Any, EventT, ResponseT], + body: dict[str, Any], + *, + record_event: Callable[[EventT], None], + record_response: Callable[[ResponseT], None], +) -> MoltoStream[EventT] | ResponseT: + """Pass a same-protocol request through, recording its native output on the span.""" + if _streams(body, native=True): + return await _prefetch_stream( + adapter.project_native_events(observe(upstream.native_events(body), record_event)), + upstream.response_headers, + ) + response = await adapter.project_native_response(upstream.native_response(body)) + record_safely(record_response, response) + return response + + +def _streams(body: Mapping[str, Any] | BaseModel, *, native: bool) -> bool: + """Return whether a coordinator streams `body`, by its route's rule. + + Native passthrough follows the SDKs, which stream for any truthy `stream` + (`stream=stream or False`), so Molto expects the response type the SDK returns. + Translated routes validate `stream` as `Literal[True, False]`, which compares by + equality: `1` and `1.0` stream, strings are rejected. The span starts before that + validation, so it applies the same rule to the raw value. + """ + stream = body.get("stream") if isinstance(body, Mapping) else getattr(body, "stream", None) + return bool(stream) if native else stream == True # noqa: E712 async def _prefetch_stream[EventT]( @@ -149,6 +297,48 @@ async def _prefetch_stream[EventT]( return MoltoStream([first_event], stream, response_headers) +async def _traced[ResultT]( + protocol: Endpoint, + endpoint: Endpoint, + provider: Provider, + base_url: str | None, + body: Mapping[str, Any] | BaseModel, + call: Callable[[SpanRecorder], Awaitable[ResultT]], +) -> ResultT: + """Run one coordinator call inside its span. + + The span starts in the caller's context, so it parents to the caller's active + span. It is current only while `call` sends the upstream request (the whole call + when non-streaming, the prefetch when streaming), so instrumented `httpx` spans + nest under it. `call` records end attributes through its `SpanRecorder`; a + returned stream ends the span itself. + """ + route = ResolvedRoute( + protocol, endpoint, provider, _UPSTREAM_CLIENTS[endpoint].resolve_base_url(base_url) + ) + streaming = _streams(body, native=protocol == endpoint) + attributes = start_attributes(route, body, streaming=streaming) + span = get_tracer().start_span( + span_name(attributes), kind=SpanKind.CLIENT, attributes=attributes + ) + recorder = SpanRecorder(span, streaming=streaming) + try: + # `_finish_span` sets the status instead of `use_span`, whose ERROR status + # description would carry the exception message. + with trace.use_span(span, record_exception=False, set_status_on_exception=False): + result = await call(recorder) + except BaseException as error: + # A streaming call that ends before its first event still reports how it ended. + _finish_span(span, error, outcome=_failure_outcome(error) if streaming else None) + raise + if isinstance(result, MoltoStream): + # The stream outlives this `await`, so it ends the span itself. + result.attach_span(span) + else: + _finish_span(span) + return result + + class Molto: def __init__( self, @@ -212,6 +402,36 @@ async def responses( resolved_provider = self._resolve_provider(endpoint, provider) if not isinstance(body, (dict, OpenAIResponsesCreateRequest)): raise TypeError("body must be a dictionary or OpenAIResponsesCreateRequest.") + return await _traced( + "responses", + endpoint, + resolved_provider, + base_url, + body, + lambda recorder: self._run_responses( + body, + endpoint=endpoint, + resolved_provider=resolved_provider, + api_key=api_key, + base_url=base_url, + extra_body=extra_body, + extra_headers=extra_headers, + recorder=recorder, + ), + ) + + async def _run_responses( + self, + body: OpenAIResponsesCreateRequest | dict[str, Any], + *, + endpoint: Endpoint, + resolved_provider: Provider, + api_key: str | None, + base_url: str | None, + extra_body: Mapping[str, Any] | None, + extra_headers: Mapping[str, str] | None, + recorder: SpanRecorder, + ) -> MoltoStream[OpenAIResponsesStreamEvent] | OpenAIResponsesResponse: resolved_api_key = self._resolve_api_key(resolved_provider, api_key) capabilities = PROVIDER_CAPABILITIES[resolved_provider] upstream_headers = _upstream_headers(extra_headers) @@ -232,12 +452,15 @@ async def responses( extra_headers=upstream_headers, ) adapter = OpenAIResponsesAdapter(capabilities) - if body.get("stream") is True: - return await _prefetch_stream( - adapter.project_native_events(upstream.native_events(body)), - upstream.response_headers, - ) - return await adapter.project_native_response(upstream.native_response(body)) + return await _run_native( + adapter, + upstream, + body, + record_event=recorder.record_responses_event, + record_response=lambda response: recorder.set_known( + responses_response_attributes(response) + ), + ) request = OpenAIResponsesCreateRequest.model_validate(body) preserve_search_metadata = endpoint == "messages" and resolved_provider == "anthropic" @@ -271,7 +494,10 @@ async def responses( else: raise ValueError(f"Unsupported endpoint: {endpoint!r}.") projected_events = adapter.project_events( - upstream.events(normalized_request.model_copy(update={"stream": True})), + observe( + upstream.events(normalized_request.model_copy(update={"stream": True})), + recorder.record_normalized, + ), normalized_request, ) if request.stream is True: @@ -297,6 +523,36 @@ async def chat_completions( resolved_provider = self._resolve_provider(endpoint, provider) if not isinstance(body, (dict, OpenAIChatCompletionsCreateRequest)): raise TypeError("body must be a dictionary or OpenAIChatCompletionsCreateRequest.") + return await _traced( + "chat/completions", + endpoint, + resolved_provider, + base_url, + body, + lambda recorder: self._run_chat_completions( + body, + endpoint=endpoint, + resolved_provider=resolved_provider, + api_key=api_key, + base_url=base_url, + extra_body=extra_body, + extra_headers=extra_headers, + recorder=recorder, + ), + ) + + async def _run_chat_completions( + self, + body: OpenAIChatCompletionsCreateRequest | dict[str, Any], + *, + endpoint: Endpoint, + resolved_provider: Provider, + api_key: str | None, + base_url: str | None, + extra_body: Mapping[str, Any] | None, + extra_headers: Mapping[str, str] | None, + recorder: SpanRecorder, + ) -> MoltoStream[OpenAIChatCompletionsStreamChunk] | OpenAIChatCompletionsResponse: resolved_api_key = self._resolve_api_key(resolved_provider, api_key) capabilities = PROVIDER_CAPABILITIES[resolved_provider] upstream_headers = _upstream_headers(extra_headers) @@ -311,12 +567,15 @@ async def chat_completions( extra_headers=upstream_headers, ) adapter = OpenAIChatCompletionsAdapter(capabilities) - if body.get("stream") is True: - return await _prefetch_stream( - adapter.project_native_events(upstream.native_events(body)), - upstream.response_headers, - ) - return await adapter.project_native_response(upstream.native_response(body)) + return await _run_native( + adapter, + upstream, + body, + record_event=recorder.record_chat_completions_chunk, + record_response=lambda response: recorder.set_known( + chat_completions_response_attributes(response) + ), + ) request = OpenAIChatCompletionsCreateRequest.model_validate(body) adapter = OpenAIChatCompletionsAdapter(capabilities) @@ -342,7 +601,10 @@ async def chat_completions( else: raise ValueError(f"Unsupported endpoint: {endpoint!r}.") projected_events = adapter.project_events( - upstream.events(normalized_request.model_copy(update={"stream": True})), + observe( + upstream.events(normalized_request.model_copy(update={"stream": True})), + recorder.record_normalized, + ), normalized_request, ) if request.stream is True: @@ -374,6 +636,38 @@ async def messages( resolved_provider = self._resolve_provider(endpoint, provider) if not isinstance(body, (dict, AnthropicMessagesCreateRequest)): raise TypeError("body must be a dictionary or AnthropicMessagesCreateRequest.") + return await _traced( + "messages", + endpoint, + resolved_provider, + base_url, + body, + lambda recorder: self._run_messages( + body, + endpoint=endpoint, + resolved_provider=resolved_provider, + api_key=api_key, + base_url=base_url, + extra_body=extra_body, + extra_headers=extra_headers, + citation_policy=citation_policy, + recorder=recorder, + ), + ) + + async def _run_messages( + self, + body: AnthropicMessagesCreateRequest | dict[str, Any], + *, + endpoint: Endpoint, + resolved_provider: Provider, + api_key: str | None, + base_url: str | None, + extra_body: Mapping[str, Any] | None, + extra_headers: Mapping[str, str] | None, + citation_policy: CitationPolicy, + recorder: SpanRecorder, + ) -> MoltoStream[AnthropicMessagesStreamEvent] | AnthropicMessagesResponse: resolved_api_key = self._resolve_api_key(resolved_provider, api_key) capabilities = PROVIDER_CAPABILITIES[resolved_provider] upstream_headers = _upstream_headers(extra_headers) @@ -388,12 +682,15 @@ async def messages( extra_headers=upstream_headers, ) adapter = AnthropicMessagesAdapter(capabilities, citation_policy=citation_policy) - if body.get("stream") is True: - return await _prefetch_stream( - adapter.project_native_events(upstream.native_events(body)), - upstream.response_headers, - ) - return await adapter.project_native_response(upstream.native_response(body)) + return await _run_native( + adapter, + upstream, + body, + record_event=recorder.record_messages_event, + record_response=lambda response: recorder.set_known( + messages_response_attributes(response) + ), + ) request = AnthropicMessagesCreateRequest.model_validate(body) # Chat normalization drops annotations, so buffering cannot recover citations. @@ -423,7 +720,10 @@ async def messages( else: raise ValueError(f"Unsupported endpoint: {endpoint!r}.") projected_events = adapter.project_events( - upstream.events(normalized_request.model_copy(update={"stream": True})), + observe( + upstream.events(normalized_request.model_copy(update={"stream": True})), + recorder.record_normalized, + ), normalized_request, ) if request.stream is True: diff --git a/python/molto/upstream/anthropic.py b/python/molto/upstream/anthropic.py index 8f1cf03..0305049 100644 --- a/python/molto/upstream/anthropic.py +++ b/python/molto/upstream/anthropic.py @@ -329,7 +329,7 @@ class AnthropicMessagesClient( def _make_anthropic_client(self) -> AsyncAnthropic: return AsyncAnthropic( api_key=self._api_key, - base_url=self._base_url or self.DEFAULT_BASE_URL, + base_url=self.resolve_base_url(self._base_url), http_client=self._http_client, ) diff --git a/python/molto/upstream/base.py b/python/molto/upstream/base.py index dc1ffea..b2998d9 100644 --- a/python/molto/upstream/base.py +++ b/python/molto/upstream/base.py @@ -42,6 +42,11 @@ def __init__( self._extra_headers = extra_headers self.response_headers = Headers() + @classmethod + def resolve_base_url(cls, base_url: str | None) -> str: + """Return the HTTP destination for `base_url`, defaulting to the client's native URL.""" + return base_url or cls.DEFAULT_BASE_URL + def _capture_response_headers(self, headers: Headers) -> None: self.response_headers.clear() self.response_headers.update(headers) @@ -63,7 +68,7 @@ def _request_kwargs(self, request: NativeRequestT | dict[str, Any]) -> dict[str, def _make_sdk_client(self) -> AsyncOpenAI: return AsyncOpenAI( api_key=self._api_key, - base_url=self._base_url or self.DEFAULT_BASE_URL, + base_url=self.resolve_base_url(self._base_url), default_headers={ # Preserve Molto's explicit credential boundary instead of merging # OpenAI account settings from the SDK's environment variables. @@ -80,7 +85,7 @@ def _make_authentication_request(self, path: str) -> Request: # this unsent request supplies its intended URL for our synthetic 401 response. # HTTP and stream errors retain their real requests instead. Match SDK path # appending here, including custom base prefixes and trailing slashes. - base_url = URL(self._base_url or self.DEFAULT_BASE_URL) + base_url = URL(self.resolve_base_url(self._base_url)) raw_path = base_url.raw_path if not raw_path.endswith(b"/"): raw_path += b"/" diff --git a/python/molto/upstream/openai.py b/python/molto/upstream/openai.py index bcf1735..72f9215 100644 --- a/python/molto/upstream/openai.py +++ b/python/molto/upstream/openai.py @@ -325,6 +325,14 @@ def _merge_search_result(items: list[dict[str, Any]], result: SearchResult) -> N ] +def responses_web_search_requests(response: OpenAIResponsesResponse) -> int | None: + """Return the web search requests a Responses response reports, or None when absent.""" + # Dump the response so an SDK-typed `tool_usage` and today's untyped extra field + # read the same way. + tool_usage = response.model_dump().get("tool_usage") or {} + return (tool_usage.get("web_search") or {}).get("num_requests") + + def _responses_usage(raw_usage: Any) -> Usage | None: """Project Responses usage into normalized usage, preserving absent usage.""" if raw_usage is None: @@ -892,10 +900,7 @@ def part_index(source_index: int) -> int: termination_reason = incomplete_details.reason usage = _responses_usage(metadata.usage) if usage is not None: - # Dump metadata so both SDK-typed fields and extras have the same shape. - tool_usage = metadata.model_dump().get("tool_usage") or {} - search_usage = tool_usage.get("web_search") or {} - usage.web_search_requests = search_usage.get("num_requests") + usage.web_search_requests = responses_web_search_requests(metadata) yield ResponseEndEvent( id=metadata.id, created_at=int(metadata.created_at), diff --git a/python/molto/utils/telemetry.py b/python/molto/utils/telemetry.py new file mode 100644 index 0000000..eadfca7 --- /dev/null +++ b/python/molto/utils/telemetry.py @@ -0,0 +1,659 @@ +"""OpenTelemetry attribute keys and span attributes for coordinator calls. + +Only the `Molto` coordinators use this module; adapters and upstream clients stay +telemetry-free. See `docs/observability.md` for the span contract. +""" + +import traceback +from collections.abc import AsyncIterator, Callable, Mapping +from contextlib import aclosing +from dataclasses import dataclass, fields +from typing import Any, Literal, TypeAlias + +import anthropic +import openai +from anthropic.types import MessageDeltaUsage, OutputTokensDetails, ServerToolUsage +from anthropic.types import Usage as AnthropicUsage +from httpx import URL, InvalidURL +from loguru import logger +from openai.types import CompletionUsage +from opentelemetry import trace +from opentelemetry.trace import Span +from opentelemetry.util.types import AttributeValue +from pydantic import BaseModel + +from molto.providers import Endpoint, Provider +from molto.types.anthropic_messages import AnthropicMessagesResponse, AnthropicMessagesStreamEvent +from molto.types.exceptions import APIException +from molto.types.normalized import ( + ModelResponseStreamEvent, + PartDeltaEvent, + ResponseEndEvent, + ResponseStartEvent, + TextDelta, + ThinkingDelta, + ToolCallDelta, +) +from molto.types.openai_chat_completions import ( + OpenAIChatCompletionsResponse, + OpenAIChatCompletionsStreamChunk, +) +from molto.types.openai_responses import OpenAIResponsesResponse, OpenAIResponsesStreamEvent +from molto.upstream.openai import responses_web_search_requests +from molto.version import __version__ + +# Keys follow the GenAI semantic conventions. They are plain strings rather than +# imports from `opentelemetry-semantic-conventions`, which exposes GenAI keys only +# from a private, Development-status module that lags the current spec. +GEN_AI_OPERATION_NAME = "gen_ai.operation.name" +GEN_AI_PROVIDER_NAME = "gen_ai.provider.name" +GEN_AI_REQUEST_CHOICE_COUNT = "gen_ai.request.choice.count" +GEN_AI_REQUEST_FREQUENCY_PENALTY = "gen_ai.request.frequency_penalty" +GEN_AI_REQUEST_MAX_TOKENS = "gen_ai.request.max_tokens" +GEN_AI_REQUEST_MODEL = "gen_ai.request.model" +GEN_AI_REQUEST_PRESENCE_PENALTY = "gen_ai.request.presence_penalty" +GEN_AI_REQUEST_SEED = "gen_ai.request.seed" +GEN_AI_REQUEST_STREAM = "gen_ai.request.stream" +GEN_AI_REQUEST_TEMPERATURE = "gen_ai.request.temperature" +GEN_AI_REQUEST_TOP_K = "gen_ai.request.top_k" +GEN_AI_REQUEST_TOP_P = "gen_ai.request.top_p" +SERVER_ADDRESS = "server.address" +SERVER_PORT = "server.port" +MOLTO_DOWNSTREAM_PROTOCOL = "molto.downstream.protocol" +MOLTO_ROUTE = "molto.route" +MOLTO_UPSTREAM_ENDPOINT = "molto.upstream.endpoint" +GEN_AI_RESPONSE_FINISH_REASONS = "gen_ai.response.finish_reasons" +GEN_AI_RESPONSE_ID = "gen_ai.response.id" +GEN_AI_RESPONSE_MODEL = "gen_ai.response.model" +GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS = "gen_ai.usage.cache_read.input_tokens" +GEN_AI_USAGE_CACHE_WRITE_INPUT_TOKENS = "gen_ai.usage.cache_write.input_tokens" +GEN_AI_USAGE_INPUT_TOKENS = "gen_ai.usage.input_tokens" +GEN_AI_USAGE_OUTPUT_TOKENS = "gen_ai.usage.output_tokens" +GEN_AI_USAGE_REASONING_OUTPUT_TOKENS = "gen_ai.usage.reasoning.output_tokens" +# The conventions define no web-search or server-tool usage count, so this one is Molto's. +MOLTO_USAGE_WEB_SEARCH_REQUESTS = "molto.usage.web_search_requests" +ERROR_TYPE = "error.type" +HTTP_RESPONSE_STATUS_CODE = "http.response.status_code" +MOLTO_ERROR_PHASE = "molto.error.phase" +MOLTO_STREAM_OUTCOME = "molto.stream.outcome" +FIRST_TOKEN_EVENT = "molto.first_token" + +# Every span attribute the contract defines. Spans carry no other keys, so no +# attribute can hold request or completion content (docs/observability.md). +ATTRIBUTE_KEYS = frozenset( + { + GEN_AI_OPERATION_NAME, + GEN_AI_PROVIDER_NAME, + GEN_AI_REQUEST_CHOICE_COUNT, + GEN_AI_REQUEST_FREQUENCY_PENALTY, + GEN_AI_REQUEST_MAX_TOKENS, + GEN_AI_REQUEST_MODEL, + GEN_AI_REQUEST_PRESENCE_PENALTY, + GEN_AI_REQUEST_SEED, + GEN_AI_REQUEST_STREAM, + GEN_AI_REQUEST_TEMPERATURE, + GEN_AI_REQUEST_TOP_K, + GEN_AI_REQUEST_TOP_P, + SERVER_ADDRESS, + SERVER_PORT, + MOLTO_DOWNSTREAM_PROTOCOL, + MOLTO_ROUTE, + MOLTO_UPSTREAM_ENDPOINT, + GEN_AI_RESPONSE_FINISH_REASONS, + GEN_AI_RESPONSE_ID, + GEN_AI_RESPONSE_MODEL, + GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS, + GEN_AI_USAGE_CACHE_WRITE_INPUT_TOKENS, + GEN_AI_USAGE_INPUT_TOKENS, + GEN_AI_USAGE_OUTPUT_TOKENS, + GEN_AI_USAGE_REASONING_OUTPUT_TOKENS, + MOLTO_USAGE_WEB_SEARCH_REQUESTS, + ERROR_TYPE, + HTTP_RESPONSE_STATUS_CODE, + MOLTO_ERROR_PHASE, + MOLTO_STREAM_OUTCOME, + } +) + +ErrorPhase = Literal["setup", "stream"] +# How a streaming call ended: the caller consumed it, closed it early, cancelled it, or +# dropped it without closing it, or it failed. Set on every streaming span, including +# failures before the first event. +StreamOutcome = Literal["completed", "closed", "cancelled", "error", "abandoned"] +# Native extractors can meet a provider that omits a field; `SpanRecorder.set_known` drops +# the `None` values. +MaybeAttributes: TypeAlias = dict[str, AttributeValue | None] + +# The cross-vendor operation type for every coordinator, not Molto's protocol name: +# backends recognize LLM spans by well-known operation names. +OPERATION_CHAT = "chat" + +# Downstream request fields holding the output-token limit, in precedence order. +# Chat Completions still accepts the deprecated `max_tokens`. The first field with an +# integer value wins, so a mistyped preferred field does not hide a valid fallback. +_MAX_TOKENS_FIELDS: dict[Endpoint, tuple[str, ...]] = { + "responses": ("max_output_tokens",), + "chat/completions": ("max_completion_tokens", "max_tokens"), + "messages": ("max_tokens",), +} +_DEFAULT_PORTS = {"http": 80, "https": 443} + + +def get_tracer() -> trace.Tracer: + """Return Molto's tracer from the application's provider, a no-op without an SDK.""" + return trace.get_tracer("molto", __version__) + + +def span_name(attributes: Mapping[str, AttributeValue]) -> str: + """Name a span `{operation} {model}`, or `{operation}` when the request has no model.""" + model = attributes.get(GEN_AI_REQUEST_MODEL) + return f"{OPERATION_CHAT} {model}" if model else OPERATION_CHAT + + +@dataclass(frozen=True) +class ResolvedRoute: + """Route values a coordinator resolves before transport. + + Attributes: + protocol: Downstream protocol, from the coordinator. + endpoint: Effective upstream endpoint. + provider: Resolved provider, reported unchanged as `gen_ai.provider.name`. + base_url: Effective upstream base URL: the caller's, or the client's default. + """ + + protocol: Endpoint + endpoint: Endpoint + provider: Provider + base_url: str + + +def start_attributes( + route: ResolvedRoute, body: Mapping[str, Any] | BaseModel, *, streaming: bool +) -> dict[str, AttributeValue]: + """Return the span attributes known before transport. + + Args: + route: The coordinator's resolved route. + body: Downstream request: a native dictionary or a validated request model. + streaming: Whether the coordinator streams the call, decided by its own rule. + """ + attributes: dict[str, AttributeValue] = { + GEN_AI_OPERATION_NAME: OPERATION_CHAT, + GEN_AI_PROVIDER_NAME: route.provider, + MOLTO_DOWNSTREAM_PROTOCOL: route.protocol, + MOLTO_UPSTREAM_ENDPOINT: route.endpoint, + MOLTO_ROUTE: "native" if route.protocol == route.endpoint else "translated", + } + # Native bodies are unvalidated dictionaries, so record a field only when it + # has the expected type; telemetry must never reject or alter a request. + model = _field(body, "model") + if isinstance(model, str) and model: + attributes[GEN_AI_REQUEST_MODEL] = model + for name in _MAX_TOKENS_FIELDS[route.protocol]: + value = _field(body, name) + if _is_int(value): + attributes[GEN_AI_REQUEST_MAX_TOKENS] = value + break + for key, name in ( + (GEN_AI_REQUEST_TEMPERATURE, "temperature"), + (GEN_AI_REQUEST_TOP_P, "top_p"), + (GEN_AI_REQUEST_FREQUENCY_PENALTY, "frequency_penalty"), + (GEN_AI_REQUEST_PRESENCE_PENALTY, "presence_penalty"), + ): + value = _field(body, name) + if _is_int(value) or isinstance(value, float): + attributes[key] = float(value) + # Molto's request types define these fields on one protocol each (Messages `top_k`, + # Chat `seed` and `n`, and the penalties above), but native passthrough forwards + # unknown fields, so they are read from any native body as sent. Translation rejects + # them, and the span of a rejected call still shows the value that caused it. + for key, name in ( + (GEN_AI_REQUEST_TOP_K, "top_k"), + (GEN_AI_REQUEST_SEED, "seed"), + ): + value = _field(body, name) + if _is_int(value): + attributes[key] = value + # The convention reports a choice count only when it differs from the default of one. + choice_count = _field(body, "n") + if _is_int(choice_count) and choice_count != 1: + attributes[GEN_AI_REQUEST_CHOICE_COUNT] = choice_count + if streaming: + attributes[GEN_AI_REQUEST_STREAM] = True + attributes.update(_server_attributes(route.base_url)) + return attributes + + +def _field(body: Mapping[str, Any] | BaseModel, name: str) -> Any: + if isinstance(body, Mapping): + return body.get(name) + return getattr(body, name, None) + + +def _is_int(value: Any) -> bool: + return isinstance(value, int) and not isinstance(value, bool) + + +def _server_attributes(base_url: str) -> dict[str, AttributeValue]: + # An unparsable URL must surface from the SDK at transport, not from telemetry. + try: + url = URL(base_url) + except InvalidURL: + return {} + if not url.host: + return {} + port = url.port or _DEFAULT_PORTS.get(url.scheme) + return {SERVER_ADDRESS: url.host} | ({SERVER_PORT: port} if port else {}) + + +def usage_attributes( + input_tokens: int | None, + output_tokens: int | None, + *, + cache_read_tokens: int | None = None, + cache_write_tokens: int | None = None, + reasoning_tokens: int | None = None, + web_search_requests: int | None = None, +) -> dict[str, AttributeValue]: + """Return token-usage attributes. + + `input_tokens` must already include cached tokens and `output_tokens` reasoning + tokens, as every provider Molto supports reports them. + """ + attributes: dict[str, AttributeValue] = {} + if _is_int(input_tokens): + attributes[GEN_AI_USAGE_INPUT_TOKENS] = input_tokens + if _is_int(output_tokens): + attributes[GEN_AI_USAGE_OUTPUT_TOKENS] = output_tokens + # Normalized usage stores a missing detail count as zero, so detail counts are + # reported only when non-zero on every route, as the convention recommends. + for key, value in ( + (GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS, cache_read_tokens), + (GEN_AI_USAGE_CACHE_WRITE_INPUT_TOKENS, cache_write_tokens), + (GEN_AI_USAGE_REASONING_OUTPUT_TOKENS, reasoning_tokens), + (MOLTO_USAGE_WEB_SEARCH_REQUESTS, web_search_requests), + ): + if _is_int(value) and value: + attributes[key] = value + return attributes + + +# Native passthrough does not validate upstream output, so the native extractors below +# read every field that a provider may omit with a default; telemetry must never turn +# a successful call into a failure. + + +def normalized_event_attributes(event: ModelResponseStreamEvent) -> dict[str, AttributeValue]: + """Return end attributes carried by a translated route's normalized lifecycle event. + + Translated spans report Molto's normalized termination reason, not the + upstream provider's own value. + """ + if isinstance(event, ResponseStartEvent): + return {GEN_AI_RESPONSE_ID: event.id, GEN_AI_RESPONSE_MODEL: event.model} + if not isinstance(event, ResponseEndEvent): + return {} + attributes: dict[str, AttributeValue] = { + GEN_AI_RESPONSE_ID: event.id, + GEN_AI_RESPONSE_MODEL: event.model, + GEN_AI_RESPONSE_FINISH_REASONS: [event.termination_reason], + } + if event.usage is not None: + details = event.usage.input_tokens_details + attributes |= usage_attributes( + event.usage.input_tokens, + event.usage.output_tokens, + cache_read_tokens=details.cached_tokens, + cache_write_tokens=details.cache_write_tokens, + reasoning_tokens=event.usage.output_tokens_details.reasoning_tokens, + web_search_requests=event.usage.web_search_requests, + ) + return attributes + + +def is_normalized_content_delta(event: ModelResponseStreamEvent) -> bool: + """Return whether a normalized event carries text, reasoning, or tool-call arguments.""" + if not isinstance(event, PartDeltaEvent): + return False + delta = event.delta + if isinstance(delta, TextDelta): + return bool(delta.content_delta) + if isinstance(delta, ThinkingDelta): + # A signature-only delta replaces opaque state and carries no content. + return bool(delta.content_delta or delta.summary_delta) + if isinstance(delta, ToolCallDelta): + return bool(delta.args_delta) + return False + + +def responses_response_attributes(response: OpenAIResponsesResponse) -> MaybeAttributes: + """Return end attributes for a native Responses `Response`. + + Responses has no finish-reason field, so the span reports the incomplete reason + when the response is incomplete and its terminal status otherwise. + """ + status = getattr(response, "status", None) + details = getattr(response, "incomplete_details", None) + reason = getattr(details, "reason", None) if status == "incomplete" else None + attributes: MaybeAttributes = { + GEN_AI_RESPONSE_ID: getattr(response, "id", None), + GEN_AI_RESPONSE_MODEL: getattr(response, "model", None), + GEN_AI_RESPONSE_FINISH_REASONS: [reason or status] if reason or status else None, + } + usage = getattr(response, "usage", None) + if usage is not None: + input_details = getattr(usage, "input_tokens_details", None) + output_details = getattr(usage, "output_tokens_details", None) + attributes |= usage_attributes( + getattr(usage, "input_tokens", None), + getattr(usage, "output_tokens", None), + cache_read_tokens=getattr(input_details, "cached_tokens", None), + cache_write_tokens=getattr(input_details, "cache_write_tokens", None), + reasoning_tokens=getattr(output_details, "reasoning_tokens", None), + web_search_requests=responses_web_search_requests(response), + ) + return attributes + + +def is_responses_content_delta(event: OpenAIResponsesStreamEvent) -> bool: + """Return whether a native Responses event carries text, reasoning, or tool-call arguments.""" + return getattr(event, "type", None) in _RESPONSES_CONTENT_DELTAS and bool( + getattr(event, "delta", None) + ) + + +def chat_completions_usage_attributes(usage: CompletionUsage) -> dict[str, AttributeValue]: + """Return token-usage attributes for Chat Completions usage.""" + details = getattr(usage, "prompt_tokens_details", None) + completion_details = getattr(usage, "completion_tokens_details", None) + return usage_attributes( + getattr(usage, "prompt_tokens", None), + getattr(usage, "completion_tokens", None), + cache_read_tokens=getattr(details, "cached_tokens", None), + cache_write_tokens=getattr(details, "cache_write_tokens", None), + reasoning_tokens=getattr(completion_details, "reasoning_tokens", None), + ) + + +def chat_completions_finish_reasons(choices: Any) -> dict[int, str]: + """Return each choice's finish reason by choice index, skipping unfinished choices.""" + reasons: dict[int, str] = {} + for choice in choices or []: + reason = getattr(choice, "finish_reason", None) + if reason: + index = getattr(choice, "index", None) + reasons[index if _is_int(index) else 0] = reason + return reasons + + +def chat_completions_response_attributes( + completion: OpenAIChatCompletionsResponse, +) -> MaybeAttributes: + """Return end attributes for a native non-streaming `ChatCompletion`.""" + reasons = chat_completions_finish_reasons(getattr(completion, "choices", None)) + attributes: MaybeAttributes = { + GEN_AI_RESPONSE_ID: getattr(completion, "id", None), + GEN_AI_RESPONSE_MODEL: getattr(completion, "model", None), + GEN_AI_RESPONSE_FINISH_REASONS: [reasons[index] for index in sorted(reasons)] or None, + } + usage = getattr(completion, "usage", None) + if usage is not None: + attributes |= chat_completions_usage_attributes(usage) + return attributes + + +def is_chat_completions_content_delta(chunk: OpenAIChatCompletionsStreamChunk) -> bool: + """Return whether a native Chat chunk carries text, reasoning, or tool-call arguments.""" + for choice in getattr(chunk, "choices", None) or []: + delta = getattr(choice, "delta", None) + if delta is None: + continue + # `reasoning_content` is a compatible-provider extension, such as vLLM's. + if any( + getattr(delta, field, None) for field in ("content", "refusal", "reasoning_content") + ): + return True + for call in getattr(delta, "tool_calls", None) or []: + if getattr(getattr(call, "function", None), "arguments", None): + return True + return False + + +@dataclass +class MessagesUsage: + """Anthropic Messages token counts; `message_delta` updates the `message_start` snapshot.""" + + input_tokens: int | None = None + output_tokens: int | None = None + cache_read_input_tokens: int | None = None + cache_creation_input_tokens: int | None = None + output_tokens_details: OutputTokensDetails | None = None + server_tool_use: ServerToolUsage | None = None + + def update(self, usage: AnthropicUsage | MessageDeltaUsage | None) -> None: + """Take every count `usage` reports, keeping earlier counts it omits.""" + for field in fields(self): + value = getattr(usage, field.name, None) + if value is not None: + setattr(self, field.name, value) + + def attributes(self) -> dict[str, AttributeValue]: + # Anthropic `input_tokens` excludes cached tokens; the convention and normalized + # usage include them, so native and translated spans agree for the same call. + cache_read = self.cache_read_input_tokens + cache_write = self.cache_creation_input_tokens + total_input = ( + self.input_tokens + (cache_read or 0) + (cache_write or 0) + if _is_int(self.input_tokens) + else None + ) + return usage_attributes( + total_input, + self.output_tokens, + cache_read_tokens=cache_read, + cache_write_tokens=cache_write, + # Anthropic `output_tokens` already includes thinking tokens. + reasoning_tokens=getattr(self.output_tokens_details, "thinking_tokens", None), + web_search_requests=getattr(self.server_tool_use, "web_search_requests", None), + ) + + +def messages_response_attributes(message: AnthropicMessagesResponse) -> MaybeAttributes: + """Return end attributes for a native non-streaming Messages `Message`.""" + stop_reason = getattr(message, "stop_reason", None) + usage = MessagesUsage() + usage.update(getattr(message, "usage", None)) + return { + GEN_AI_RESPONSE_ID: getattr(message, "id", None), + GEN_AI_RESPONSE_MODEL: getattr(message, "model", None), + GEN_AI_RESPONSE_FINISH_REASONS: [stop_reason] if stop_reason else None, + **usage.attributes(), + } + + +def is_messages_content_delta(event: AnthropicMessagesStreamEvent) -> bool: + """Return whether a native Messages event carries text, reasoning, or tool-call arguments.""" + if getattr(event, "type", None) != "content_block_delta": + return False + delta = getattr(event, "delta", None) + field = _MESSAGES_CONTENT_DELTAS.get(getattr(delta, "type", None)) + return field is not None and bool(getattr(delta, field, None)) + + +def error_attributes(error: BaseException, phase: ErrorPhase) -> dict[str, AttributeValue]: + """Return low-cardinality error attributes; the message is never an attribute.""" + normalized = next( + ( + candidate + for candidate in (error, error.__cause__) + if isinstance(candidate, APIException) + ), + None, + ) + # A recognized failure reports its normalized class, so native and translated + # routes agree; the outward exception is the caller's SDK-specific class. + attributes: dict[str, AttributeValue] = { + ERROR_TYPE: type(normalized or error).__name__, + MOLTO_ERROR_PHASE: phase, + } + if normalized is not None: + status = normalized.response.status_code if normalized.response is not None else None + elif isinstance(error, (openai.APIStatusError, anthropic.APIStatusError)): + status = error.status_code + else: + status = None + # Only a real upstream error response: missing credentials carry a synthetic 401 + # on the outward exception alone, and a mid-stream failure can arrive on HTTP 200. + if _is_int(status) and status >= 400: + attributes[HTTP_RESPONSE_STATUS_CODE] = status + return attributes + + +def exception_event_attributes(error: BaseException) -> dict[str, AttributeValue]: + """Return content-free overrides for the attributes `Span.record_exception` builds. + + Exception messages can quote request content (a validation error's input, or an + upstream error echoing a field), and the default stack trace repeats the message + of every chained cause. Report the class name as the message and a stack trace of + frames only: frame lines show code locations, never runtime values. + """ + lines: list[str] = [] + seen: set[int] = set() + current: BaseException | None = error + while current is not None and id(current) not in seen: + seen.add(id(current)) + lines.append(f"{type(current).__module__}.{type(current).__qualname__}\n") + lines.extend(traceback.format_tb(current.__traceback__)) + current = current.__cause__ + return {"exception.message": type(error).__name__, "exception.stacktrace": "".join(lines)} + + +_RESPONSES_CONTENT_DELTAS = frozenset( + { + "response.output_text.delta", + "response.reasoning_text.delta", + "response.reasoning_summary_text.delta", + "response.function_call_arguments.delta", + "response.custom_tool_call_input.delta", + "response.mcp_call_arguments.delta", + "response.refusal.delta", + } +) +# The SDK raises only on a separate `error` event, so a `response.failed` that arrives +# without one passes through native streaming and must still be recorded. +_RESPONSES_TERMINAL_EVENTS = frozenset( + {"response.completed", "response.incomplete", "response.failed"} +) +# Messages content-block delta types that carry content, mapped to their text field. +_MESSAGES_CONTENT_DELTAS = { + "text_delta": "text", + "thinking_delta": "thinking", + "input_json_delta": "partial_json", +} + + +async def observe[EventT]( + events: AsyncIterator[EventT], record: Callable[[EventT], None] +) -> AsyncIterator[EventT]: + """Yield `events` unchanged, passing each to `record` first.""" + # Closing this wrapper must close the source and release its HTTP response. + async with aclosing(events) as source: + async for event in source: + record_safely(record, event) + yield event + + +def record_safely[ValueT](record: Callable[[ValueT], None], value: ValueT) -> None: + """Pass `value` to `record`; a recording failure loses attributes, never the call. + + Native output is unvalidated, so a provider can send a value no extractor expects. + """ + try: + record(value) + except Exception as error: + # Only the class: loguru's traceback diagnosis prints local values, which can + # hold request or completion content. + logger.debug("Molto could not record span attributes: {}", type(error).__name__) + + +class SpanRecorder: + """Record end attributes and the first-token event on a coordinator's span. + + The coordinator starts and ends the span; the recorder only adds what the + upstream output reveals, so the span carries it before it ends. It holds the + state that spans several stream events. + """ + + def __init__(self, span: Span, *, streaming: bool) -> None: + self._span = span + self._streaming = streaming + self._first_token_recorded = False + self._chat_identity: tuple[Any, Any] = (None, None) + self._chat_finish_reasons: dict[int, str] = {} + self._messages_usage = MessagesUsage() + + def set_known(self, attributes: MaybeAttributes) -> None: + """Set the attributes whose value is known, skipping `None` values.""" + self._span.set_attributes( + {key: value for key, value in attributes.items() if value is not None} + ) + + def record_normalized(self, event: ModelResponseStreamEvent) -> None: + self.set_known(normalized_event_attributes(event)) + if is_normalized_content_delta(event): + self._record_first_token() + + def record_responses_event(self, event: OpenAIResponsesStreamEvent) -> None: + if getattr(event, "type", None) in _RESPONSES_TERMINAL_EVENTS: + self.set_known(responses_response_attributes(getattr(event, "response", None))) + elif is_responses_content_delta(event): + self._record_first_token() + + def record_chat_completions_chunk(self, chunk: OpenAIChatCompletionsStreamChunk) -> None: + # Every chunk repeats the response identity, so set it only when it changes. + identity = (getattr(chunk, "id", None), getattr(chunk, "model", None)) + if identity != self._chat_identity: + self._chat_identity = identity + self.set_known({GEN_AI_RESPONSE_ID: identity[0], GEN_AI_RESPONSE_MODEL: identity[1]}) + if is_chat_completions_content_delta(chunk): + self._record_first_token() + # Each choice reports its finish reason in its own chunk. + reasons = self._chat_finish_reasons | chat_completions_finish_reasons( + getattr(chunk, "choices", None) + ) + if reasons != self._chat_finish_reasons: + self._chat_finish_reasons = reasons + self.set_known({GEN_AI_RESPONSE_FINISH_REASONS: [reasons[i] for i in sorted(reasons)]}) + # Present only when the caller set `stream_options.include_usage`. + usage = getattr(chunk, "usage", None) + if usage is not None: + self.set_known(chat_completions_usage_attributes(usage)) + + def record_messages_event(self, event: AnthropicMessagesStreamEvent) -> None: + event_type = getattr(event, "type", None) + if event_type == "message_start": + message = getattr(event, "message", None) + self.set_known( + { + GEN_AI_RESPONSE_ID: getattr(message, "id", None), + GEN_AI_RESPONSE_MODEL: getattr(message, "model", None), + } + ) + self._record_messages_usage(getattr(message, "usage", None)) + elif event_type == "message_delta": + stop_reason = getattr(getattr(event, "delta", None), "stop_reason", None) + if stop_reason: + self.set_known({GEN_AI_RESPONSE_FINISH_REASONS: [stop_reason]}) + self._record_messages_usage(getattr(event, "usage", None)) + elif is_messages_content_delta(event): + self._record_first_token() + + def _record_messages_usage(self, usage: AnthropicUsage | MessageDeltaUsage | None) -> None: + self._messages_usage.update(usage) + self.set_known(self._messages_usage.attributes()) + + def _record_first_token(self) -> None: + # Time to first token matters only to a streaming caller. + if self._streaming and not self._first_token_recorded: + self._first_token_recorded = True + self._span.add_event(FIRST_TOKEN_EVENT) diff --git a/python/molto/utils/test.py b/python/molto/utils/test.py index 0fb1b8c..6cf3bdb 100644 --- a/python/molto/utils/test.py +++ b/python/molto/utils/test.py @@ -4,9 +4,11 @@ from typing import Any from anthropic.lib.streaming._messages import accumulate_event +from httpx import AsyncClient, MockTransport, Request, Response from openai.lib.streaming.chat import ChatCompletionStreamState from pydantic import BaseModel +from molto.core import Molto, MoltoStream from molto.providers import Endpoint from molto.types.anthropic_messages import AnthropicMessagesStreamEventAdapter from molto.types.openai_chat_completions import ( @@ -17,7 +19,7 @@ OpenAIResponsesCreateRequest, OpenAIResponsesStreamEventAdapter, ) -from molto.utils.io import dumps_sse, load_sse +from molto.utils.io import dumps_sse, load_sse, read_yaml _RECONSTRUCTIBLE_RESPONSE_FIELDS = { "id", @@ -314,3 +316,64 @@ def reconstructable_anthropic_event(sse: str) -> dict[str, Any]: elif event["type"] == "message_delta": event["delta"].pop("stop_details", None) return event + + +def replay_transport(cassette: dict[str, Any]) -> MockTransport: + """Serve a cassette's recorded response for every request, without checking the request.""" + recorded = cassette["response"] + + def handler(request: Request) -> Response: + if "sse" in recorded: + return Response( + recorded["status_code"], + content="".join(recorded["sse"]).encode(), + headers={"content-type": "text/event-stream"}, + ) + return Response(recorded["status_code"], json=recorded["body"]) + + return MockTransport(handler) + + +async def coordinator_outcome( + method: str, + body: dict[str, Any], + http_client: AsyncClient, + **arguments: Any, +) -> dict[str, Any]: + """Call one `Molto` coordinator and return what the caller observes, as JSON values. + + A stream is consumed in full. A failure is reported by exception class and + message, so outcomes compare equal only when the caller saw the same thing. + """ + async with Molto(http_client=http_client) as molto: + try: + result = await getattr(molto, method)(body=body, **arguments) + if isinstance(result, MoltoStream): + async with result: + return {"events": [event.model_dump(mode="json") async for event in result]} + return {"response": result.model_dump(mode="json")} + except Exception as error: + return {"error": type(error).__name__, "message": str(error)} + + +async def replayed_outcomes(cases: list[dict[str, Any]]) -> list[dict[str, Any]]: + """Return each case's coordinator outcome, replaying its recorded upstream response. + + Each case names a `method`, an `endpoint`, the `request` cassette whose body is + sent, and an optional `response` cassette (the request cassette otherwise). Paths + are cassette files; no request reaches the network. + """ + outcomes = [] + for case in cases: + request = read_yaml(case["request"]) + response = read_yaml(case.get("response") or case["request"]) + outcomes.append( + await coordinator_outcome( + case["method"], + request["request"]["body"], + AsyncClient(transport=replay_transport(response)), + endpoint=case["endpoint"], + api_key="test-key", + ) + ) + return outcomes diff --git a/tests/core/test_request_validation.py b/tests/core/test_request_validation.py index 9cb05de..f464679 100644 --- a/tests/core/test_request_validation.py +++ b/tests/core/test_request_validation.py @@ -144,7 +144,12 @@ async def test_native_requests_preserve_provider_fields_and_sdk_parity( async def test_native_non_boolean_stream_preserves_provider_error( protocol: Protocol, stream: int | str, monkeypatch: pytest.MonkeyPatch ) -> None: - """Ensures truthy non-booleans use the response path and retain SDK error fidelity.""" + """Ensures truthy non-booleans take the stream path, as the SDKs do, with SDK error fidelity. + + The SDKs stream for any truthy `stream` (`stream=stream or False`); choosing the + response path instead fails with a `TypeError` when a provider accepts the value + and returns a stream. + """ path = _matched_cassette_path("error-invalid_request_error", protocol, "stream-00.yaml") body = {**deepcopy(_read_cassette(path)["request"]["body"]), "stream": stream} error_type = AnthropicBadRequestError if protocol == "messages" else OpenAIBadRequestError @@ -154,10 +159,10 @@ async def test_native_non_boolean_stream_preserves_provider_error( "messages": AnthropicMessagesClient, }[protocol] - def unexpected_stream(*args: Any) -> None: - raise AssertionError("Only the boolean True should select native_events") + async def unexpected_response(*args: Any) -> None: + raise AssertionError("A truthy stream should select native_events, as the SDK streams") - monkeypatch.setattr(client_type, "native_events", unexpected_stream) + monkeypatch.setattr(client_type, "native_response", unexpected_response) # Replay an existing provider error to verify forwarding and error handling, # without claiming live acceptance of the modified stream value. async with _get_http_client([path], protocol, expected_body=body) as http_client: diff --git a/tests/core/test_tracing.py b/tests/core/test_tracing.py new file mode 100644 index 0000000..7a7d1da --- /dev/null +++ b/tests/core/test_tracing.py @@ -0,0 +1,1805 @@ +import asyncio +import gc +import subprocess +import sys +from collections.abc import AsyncIterator, Awaitable, Callable, Iterator +from contextlib import contextmanager +from pathlib import Path +from time import time_ns +from typing import Any, NamedTuple + +import openai +import pytest +from httpx import ( + AsyncByteStream, + AsyncClient, + AsyncHTTPTransport, + Headers, + MockTransport, + Request, + Response, +) +from opentelemetry import trace +from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor +from opentelemetry.metrics import NoOpMeterProvider +from opentelemetry.sdk.trace import ReadableSpan, TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter +from opentelemetry.trace import SpanKind, StatusCode +from pydantic import SecretStr, ValidationError + +from molto.configs import ENV_CONFIG +from molto.core import Molto, MoltoStream, _prefetch_stream, _streams, _traced +from molto.types.exceptions import ContextLengthExceededError, UnsupportedInput +from molto.utils.io import json_dumps, json_loads, read_yaml +from molto.utils.telemetry import ATTRIBUTE_KEYS +from molto.utils.test import replay_transport, replayed_outcomes +from molto.version import __version__ + +BODY = {"model": "gpt-5", "input": "Hello!"} +# Placed in request content, extensions, headers, and credentials; it must never +# reach a span. +CONTENT_MARKER = "secret-content-marker" +# Event attribute keys Molto may record; anything else could carry content. +EVENT_ATTRIBUTE_KEYS = { + "molto.first_token": set(), + "exception": { + "exception.type", + "exception.message", + "exception.stacktrace", + "exception.escaped", + }, +} +CASSETTES = Path(__file__).parents[1] / "cassettes" + + +@pytest.fixture +def tracer_provider(monkeypatch: pytest.MonkeyPatch) -> TracerProvider: + """Route Molto's tracer to a local SDK provider, leaving the global provider untouched.""" + provider = TracerProvider() + monkeypatch.setattr("molto.core.get_tracer", lambda: provider.get_tracer("molto")) + return provider + + +@pytest.fixture(autouse=True) +def exporter( + tracer_provider: TracerProvider, caplog: pytest.LogCaptureFixture +) -> Iterator[InMemorySpanExporter]: + """Collect every span the local provider ends, and fail if one ends twice. + + The SDK ignores a second `end()` and only logs a warning, so the warning is the + evidence that a span was ended more than once. + """ + exporter = InMemorySpanExporter() + tracer_provider.add_span_processor(SimpleSpanProcessor(exporter)) + yield exporter + # `caplog.records` holds only the current (teardown) phase; check the test body. + records = caplog.get_records("call") + assert not [record for record in records if "ended span" in record.getMessage()] + + +def _molto_span(exporter: InMemorySpanExporter) -> ReadableSpan: + (span,) = exporter.get_finished_spans() + return span + + +def _assert_cancelled(span: ReadableSpan) -> None: + """Cancellation ends the span as UNSET with no error (docs/observability.md).""" + assert span.status.status_code is StatusCode.UNSET + assert "error.type" not in span.attributes + assert "molto.error.phase" not in span.attributes + assert span.events == () + + +async def _events(*events: str, error: BaseException | None = None) -> AsyncIterator[str]: + for event in events: + yield event + if error is not None: + raise error + + +async def _trace[ResultT](call: Callable[[], Awaitable[ResultT]]) -> ResultT: + return await _traced("responses", "responses", "openai", None, BODY, lambda _: call()) + + +async def _traced_stream(events: AsyncIterator[str]) -> MoltoStream[str]: + return await _trace(lambda: _prefetch_stream(events, Headers())) + + +def _cassette(name: str) -> dict[str, Any]: + """Read the cassette at `name`, a path under `tests/cassettes` without `.yaml`.""" + return read_yaml(str(CASSETTES / f"{name}.yaml")) + + +def _replay(cassette_name: str) -> AsyncClient: + """Serve one recorded response for any request, without checking the request.""" + return AsyncClient(transport=replay_transport(_cassette(cassette_name))) + + +def _recorded_response_id(cassette_name: str) -> str: + """Return the response ID a recording carries, from its body or first identified event.""" + response = _cassette(cassette_name)["response"] + if "body" in response: + return response["body"]["id"] + for line in "".join(response["sse"]).splitlines(): + if line.startswith("data:") and line != "data: [DONE]": + data = json_loads(line.removeprefix("data:")) + identified = data.get("response") or data.get("message") or data + if "id" in identified: + return identified["id"] + raise AssertionError(f"{cassette_name} has no response ID") + + +def _request_body(cassette_name: str) -> dict[str, Any]: + return _cassette(cassette_name)["request"]["body"] + + +async def test_non_streaming_call_ends_span_on_return(exporter: InMemorySpanExporter): + """Ensures a non-streaming call ends one CLIENT span with UNSET status when it returns.""" + + async def call() -> str: + return "response" + + assert await _trace(call) == "response" + + span = _molto_span(exporter) + assert span.name == "chat gpt-5" + assert span.kind is SpanKind.CLIENT + assert span.status.status_code is StatusCode.UNSET + assert span.attributes["molto.route"] == "native" + + +async def test_non_streaming_failure_marks_span_error(exporter: InMemorySpanExporter): + """Asserts that a failed call ends its span with ERROR and re-raises the original error.""" + + async def call() -> str: + raise RuntimeError("upstream failed") + + with pytest.raises(RuntimeError, match="upstream failed"): + await _trace(call) + + span = _molto_span(exporter) + assert span.status.status_code is StatusCode.ERROR + # The message must not reach the status description (docs/observability.md). + assert not span.status.description + + +async def test_cancelled_call_ends_span_without_error(exporter: InMemorySpanExporter): + """Ensures cancellation ends the span with UNSET status, since the call did not fail.""" + + async def call() -> str: + raise asyncio.CancelledError + + with pytest.raises(asyncio.CancelledError): + await _trace(call) + + _assert_cancelled(_molto_span(exporter)) + + +async def test_span_parents_to_caller_and_is_current_only_during_the_call( + exporter: InMemorySpanExporter, +): + """Ensures the span nests under the caller's span and is current only while the call runs. + + HTTP client spans parent to the current span, so the Molto span must be current + while the upstream request is sent and must not leak into the caller's context. + """ + current_during_call = [] + + async def call() -> str: + current_during_call.append(trace.get_current_span()) + return "response" + + with TracerProvider().get_tracer("caller").start_as_current_span("caller") as caller: + await _trace(call) + assert trace.get_current_span() is caller + + span = _molto_span(exporter) + assert span.parent.span_id == caller.get_span_context().span_id + assert current_during_call[0].get_span_context().span_id == span.context.span_id + + +async def test_stream_span_outlives_coordinator_and_ends_on_exhaustion( + exporter: InMemorySpanExporter, +): + """Ensures a streaming span stays open after the coordinator returns and ends at exhaustion.""" + stream = await _traced_stream(_events("a", "b")) + + assert exporter.get_finished_spans() == () + assert [event async for event in stream] == ["a", "b"] + + assert _molto_span(exporter).status.status_code is StatusCode.UNSET + await stream.aclose() + + +async def test_empty_stream_ends_span_on_first_iteration(exporter: InMemorySpanExporter): + """Ensures a stream with no events, already exhausted by the prefetch, still ends its span.""" + stream = await _traced_stream(_events()) + + assert [event async for event in stream] == [] + + assert _molto_span(exporter).status.status_code is StatusCode.UNSET + + +async def test_stream_span_ends_once_on_early_aclose(exporter: InMemorySpanExporter): + """Ensures closing a partly consumed stream ends its span once, without ERROR.""" + stream = await _traced_stream(_events("a", "b", "c")) + + assert await anext(stream) == "a" + await stream.aclose() + await stream.aclose() + + assert _molto_span(exporter).status.status_code is StatusCode.UNSET + + +async def test_stream_failure_marks_span_error(exporter: InMemorySpanExporter): + """Asserts that an upstream error while the caller iterates ends the span with ERROR once.""" + stream = await _traced_stream(_events("a", error=RuntimeError("stream failed"))) + + with pytest.raises(RuntimeError, match="stream failed"): + async with stream: + async for _ in stream: + pass + + assert _molto_span(exporter).status.status_code is StatusCode.ERROR + + +async def test_failing_close_marks_span_error(exporter: InMemorySpanExporter): + """Asserts that an error while closing the source stream ends the span with ERROR.""" + + class FailingClose: + def __aiter__(self) -> "FailingClose": + return self + + async def __anext__(self) -> str: + return "a" + + async def aclose(self) -> None: + raise RuntimeError("close failed") + + stream = await _traced_stream(FailingClose()) + + with pytest.raises(RuntimeError, match="close failed"): + await stream.aclose() + + assert _molto_span(exporter).status.status_code is StatusCode.ERROR + + +async def test_stream_cancellation_ends_span_without_error(exporter: InMemorySpanExporter): + """Ensures a stream cancelled mid-iteration ends its span with UNSET status.""" + stream = await _traced_stream(_events("a", error=asyncio.CancelledError())) + + with pytest.raises(asyncio.CancelledError): + async for _ in stream: + pass + + _assert_cancelled(_molto_span(exporter)) + + +async def test_prefetch_failure_ends_span_in_coordinator(exporter: InMemorySpanExporter): + """Asserts that a failure before the first event ends the span with ERROR, with no stream.""" + with pytest.raises(RuntimeError, match="setup failed"): + await _traced_stream(_events(error=RuntimeError("setup failed"))) + + assert _molto_span(exporter).status.status_code is StatusCode.ERROR + + +async def test_prefetch_cancellation_ends_span_without_error(exporter: InMemorySpanExporter): + """Ensures a call cancelled before its first event ends the span with UNSET status.""" + with pytest.raises(asyncio.CancelledError): + await _traced_stream(_events(error=asyncio.CancelledError())) + + _assert_cancelled(_molto_span(exporter)) + + +async def test_stream_span_is_not_current_while_the_caller_iterates( + exporter: InMemorySpanExporter, +): + """Ensures iterating a stream runs in the caller's context, not under the Molto span. + + The span is current for the prefetch only; later iteration, possibly from another + task, must not make it current or reparent spans the caller creates. + """ + current = [] + + async def events() -> AsyncIterator[str]: + current.append(trace.get_current_span()) + yield "a" + current.append(trace.get_current_span()) + yield "b" + + stream = await _traced_stream(events()) + async with stream: + assert [event async for event in stream] == ["a", "b"] + + span = _molto_span(exporter) + prefetch, iteration = current + assert prefetch.get_span_context().span_id == span.context.span_id + assert iteration.get_span_context().span_id != span.context.span_id + + +COORDINATOR_CASES = [ + # (method, protocol, endpoint, request cassette, response cassette, finish reason) + ( + "responses", + "responses", + "responses", + "matched/simple_text/openai-responses-stream-00", + None, + "completed", + ), + ( + "responses", + "responses", + "responses", + "matched/simple_text/openai-responses-non_stream-00", + None, + "completed", + ), + ( + "chat_completions", + "chat/completions", + "chat/completions", + "matched/simple_text/openai-chat_completions-stream-00", + None, + "stop", + ), + ( + "chat_completions", + "chat/completions", + "chat/completions", + "matched/simple_text/openai-chat_completions-non_stream-00", + None, + "stop", + ), + ( + "messages", + "messages", + "messages", + "matched/simple_text/anthropic-messages-stream-00", + None, + "end_turn", + ), + ( + "messages", + "messages", + "messages", + "matched/simple_text/anthropic-messages-non_stream-00", + None, + "end_turn", + ), + # Translated spans report Molto's normalized termination reason. + ( + "responses", + "responses", + "chat/completions", + "matched/simple_text/openai-responses-stream-00", + "matched/simple_text/openai-chat_completions-stream-00", + "stop", + ), + ( + "messages", + "messages", + "responses", + "matched/simple_text/anthropic-messages-non_stream-00", + # Translated calls always stream from the upstream, even when non-streaming. + "matched/simple_text/openai-responses-stream-00", + "stop", + ), + ( + "responses", + "responses", + "messages", + "matched/simple_text/openai-responses-non_stream-00", + "matched/simple_text/anthropic-messages-stream-00", + "stop", + ), + ( + "chat_completions", + "chat/completions", + "messages", + "matched/simple_text/openai-chat_completions-stream-00", + "matched/simple_text/anthropic-messages-stream-00", + "stop", + ), + ( + "chat_completions", + "chat/completions", + "responses", + "matched/simple_text/openai-chat_completions-non_stream-00", + "matched/simple_text/openai-responses-stream-00", + "stop", + ), + ( + "messages", + "messages", + "chat/completions", + "matched/simple_text/anthropic-messages-stream-00", + "matched/simple_text/openai-chat_completions-stream-00", + "stop", + ), +] + + +async def _run_coordinator( + method: str, + endpoint: str, + request_cassette: str, + response_cassette: str | None, + **body_fields: Any, +) -> bool: + """Call one coordinator against a replayed response, consuming any stream.""" + body = {**_request_body(request_cassette), **body_fields} + client = _replay(response_cassette or request_cassette) + async with Molto(http_client=client) as molto: + result = await getattr(molto, method)(body=body, endpoint=endpoint, api_key="test-key") + if not isinstance(result, MoltoStream): + return False + async with result: + [event async for event in result] + return True + + +@pytest.mark.parametrize( + ("method", "protocol", "endpoint", "request_cassette", "response_cassette", "finish_reason"), + COORDINATOR_CASES, +) +async def test_coordinator_emits_one_span_per_call( + exporter: InMemorySpanExporter, + method: str, + protocol: str, + endpoint: str, + request_cassette: str, + response_cassette: str | None, + finish_reason: str, +): + """Ensures each coordinator emits one complete span per call on native and translated routes. + + The span is a CLIENT child of the caller's span and carries the start attributes, + the response identity, finish reason, and usage, and, for a streaming caller only, + one `molto.first_token` event. + """ + with TracerProvider().get_tracer("caller").start_as_current_span("caller") as caller: + # The replay ignores the request, so sampling values only reach the span. + streamed = await _run_coordinator( + method, endpoint, request_cassette, response_cassette, temperature=0.5, top_p=0.9 + ) + + span = _molto_span(exporter) + attributes = span.attributes + assert span.name == "chat gpt-4.1-nano" + assert span.kind is SpanKind.CLIENT + assert span.parent.span_id == caller.get_span_context().span_id + assert span.status.status_code is StatusCode.UNSET + assert attributes["gen_ai.operation.name"] == "chat" + assert attributes["gen_ai.provider.name"] == ( + "anthropic" if endpoint == "messages" else "openai" + ) + assert attributes["server.address"] == ( + "api.anthropic.com" if endpoint == "messages" else "api.openai.com" + ) + assert attributes["server.port"] == 443 + assert attributes["gen_ai.request.model"] == "gpt-4.1-nano" + assert attributes["gen_ai.request.max_tokens"] == 1024 + assert attributes["gen_ai.request.temperature"] == 0.5 + assert attributes["gen_ai.request.top_p"] == 0.9 + assert attributes["molto.downstream.protocol"] == protocol + assert attributes["molto.upstream.endpoint"] == endpoint + assert attributes["molto.route"] == ("native" if protocol == endpoint else "translated") + assert attributes.get("gen_ai.request.stream") is (True if streamed else None) + assert attributes["gen_ai.response.id"] == _recorded_response_id( + response_cassette or request_cassette + ) + assert attributes["gen_ai.response.model"] == "gpt-4.1-nano-2025-04-14" + assert attributes["gen_ai.response.finish_reasons"] == (finish_reason,) + assert attributes["gen_ai.usage.input_tokens"] == 15 + assert isinstance(attributes["gen_ai.usage.output_tokens"], int) + # The recorded calls used no prompt cache, so no zero-valued cache counts appear. + assert "gen_ai.usage.cache_read.input_tokens" not in attributes + assert "gen_ai.usage.cache_write.input_tokens" not in attributes + assert [event.name for event in span.events] == (["molto.first_token"] if streamed else []) + + +async def test_native_and_translated_messages_spans_report_the_same_usage( + exporter: InMemorySpanExporter, +): + """Asserts that the same Messages upstream call reports identical usage on both routes. + + Native Messages sums Anthropic's uncached and cached input tokens, matching the + normalized usage that the translated route reports. + """ + await _run_coordinator( + "messages", "messages", "matched/simple_text/anthropic-messages-stream-00", None + ) + await _run_coordinator( + "responses", + "messages", + "matched/simple_text/openai-responses-stream-00", + "matched/simple_text/anthropic-messages-stream-00", + ) + + native, translated = exporter.get_finished_spans() + usage_keys = ("gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens") + assert [native.attributes[key] for key in usage_keys] == [ + translated.attributes[key] for key in usage_keys + ] + assert native.attributes["gen_ai.response.finish_reasons"] == ("end_turn",) + assert translated.attributes["gen_ai.response.finish_reasons"] == ("stop",) + + +def _not_found(request: Request) -> Response: + return Response( + 404, + json={"error": {"message": "secret-request-marker not found", "type": "not_found"}}, + ) + + +@pytest.mark.parametrize( + ("method", "body", "endpoint"), + [ + ("responses", {"model": "gpt-5", "input": "Hello!"}, "responses"), + ( + "chat_completions", + {"model": "gpt-5", "messages": [{"role": "user", "content": "Hello!"}]}, + "responses", + ), + ], +) +async def test_upstream_error_reports_the_same_error_type_on_both_routes( + exporter: InMemorySpanExporter, method: str, body: dict, endpoint: str +): + """Asserts that an upstream HTTP error records the normalized error on native and translated routes. + + The caller receives an SDK-specific exception, but `error.type` reports the + normalized class so both routes agree, and the recorded exception event carries + no message text from the upstream error. + """ + async with Molto(http_client=AsyncClient(transport=MockTransport(_not_found))) as molto: + with pytest.raises(Exception, match="secret-request-marker"): + await getattr(molto, method)(body=body, endpoint=endpoint, api_key="test-key") + + span = _molto_span(exporter) + assert span.status.status_code is StatusCode.ERROR + assert span.attributes["error.type"] == "ResourceNotFoundError" + assert span.attributes["http.response.status_code"] == 404 + assert span.attributes["molto.error.phase"] == "setup" + # `use_span` must not record its own exception event with the full message. + (event,) = span.events + assert event.name == "exception" + assert event.attributes["exception.message"] == "NotFoundError" + assert "secret-request-marker" not in event.attributes["exception.stacktrace"] + assert "ResourceNotFoundError" in event.attributes["exception.stacktrace"] + + +async def test_validation_error_records_no_request_content(exporter: InMemorySpanExporter): + """Asserts that a rejected request's values never reach the span, though the error message holds them.""" + async with Molto() as molto: + with pytest.raises(ValidationError, match="secret-request-marker"): + await molto.responses( + body={"model": "gpt-5", "input": "Hello!", "extra": "secret-request-marker"}, + endpoint="messages", + api_key="test-key", + ) + + span = _molto_span(exporter) + assert span.attributes["error.type"] == "ValidationError" + assert "http.response.status_code" not in span.attributes + (event,) = span.events + assert event.attributes["exception.message"] == "ValidationError" + recorded = [*span.attributes.values(), *event.attributes.values()] + assert not any("secret-request-marker" in str(value) for value in recorded) + + +async def test_mid_stream_error_reports_stream_phase(exporter: InMemorySpanExporter): + """Ensures an error raised while the caller iterates is attributed to the stream phase.""" + stream = await _traced_stream(_events("a", error=RuntimeError("secret-request-marker"))) + + with pytest.raises(RuntimeError): + async for _ in stream: + pass + + span = _molto_span(exporter) + assert span.attributes["error.type"] == "RuntimeError" + assert span.attributes["molto.error.phase"] == "stream" + (event,) = span.events + assert "secret-request-marker" not in str(dict(event.attributes)) + + +async def test_coordinator_reports_caller_base_url(exporter: InMemorySpanExporter): + """Asserts that a caller-supplied `base_url`, not the client default, sets the server attributes.""" + body = _request_body("matched/simple_text/openai-responses-non_stream-00") + async with Molto( + http_client=_replay("matched/simple_text/openai-responses-non_stream-00") + ) as molto: + await molto.responses(body=body, base_url="http://localhost:8000/v1", api_key="test-key") + + span = _molto_span(exporter) + assert span.attributes["server.address"] == "localhost" + assert span.attributes["server.port"] == 8000 + + +async def test_coordinator_rejection_is_recorded_on_its_span(exporter: InMemorySpanExporter): + """Asserts that a coordinator's local rejection ends its span with ERROR. + + The span starts before request validation, so failures such as an invalid + translated body are visible in traces. The effective base URL is the upstream + client's default when the caller passes none. + """ + async with Molto() as molto: + with pytest.raises(ValidationError): + await molto.responses( + body={"model": "claude-sonnet-5", "input": "Hello!", "unknown_field": 1}, + endpoint="messages", + api_key="test-key", + ) + + span = _molto_span(exporter) + assert span.name == "chat claude-sonnet-5" + assert span.status.status_code is StatusCode.ERROR + assert span.attributes["molto.route"] == "translated" + assert span.attributes["gen_ai.provider.name"] == "anthropic" + assert span.attributes["server.address"] == "api.anthropic.com" + + +async def test_cached_input_is_reported_alike_on_every_route(exporter: InMemorySpanExporter): + """Asserts that one cached call reports the same input and cache-read counts on all routes. + + The recorded Anthropic call reports 34 uncached and 7440 cache-read input tokens; + the matched Responses call reports 7474 input tokens with 7440 cached. Native + Messages must add the cached tokens to match Responses and the translated route. + """ + scenario = "matched/prompt_cache_explicit" + await _run_coordinator( + "responses", "responses", f"{scenario}/openai-responses-stream-00", None + ) + await _run_coordinator( + "messages", "messages", f"{scenario}/anthropic-messages-stream-00", None + ) + await _run_coordinator( + "responses", + "messages", + f"{scenario}/openai-responses-stream-00", + f"{scenario}/anthropic-messages-stream-00", + ) + + for span in exporter.get_finished_spans(): + assert span.attributes["gen_ai.usage.input_tokens"] == 7474 + assert span.attributes["gen_ai.usage.cache_read.input_tokens"] == 7440 + assert "gen_ai.usage.cache_write.input_tokens" not in span.attributes + + +REASONING = "matched/reasoning_encrypted" + + +@pytest.mark.parametrize( + ("method", "endpoint", "request_cassette", "response_cassette", "expected"), + [ + ("responses", "responses", f"{REASONING}/openai-responses-stream-00", None, 36), + ("responses", "responses", f"{REASONING}/openai-responses-non_stream-00", None, 110), + ("messages", "messages", f"{REASONING}/anthropic-messages-stream-00", None, 36), + ("messages", "messages", f"{REASONING}/anthropic-messages-non_stream-00", None, 110), + ( + "responses", + "messages", + f"{REASONING}/openai-responses-stream-00", + f"{REASONING}/anthropic-messages-stream-00", + 36, + ), + ], +) +async def test_reasoning_tokens_are_reported_alike_on_every_route( + exporter: InMemorySpanExporter, + method: str, + endpoint: str, + request_cassette: str, + response_cassette: str | None, + expected: int, +): + """Asserts that a reasoning call reports the provider's reasoning-token count on every route. + + The matched recordings report the same count as Responses `reasoning_tokens` and + Anthropic `thinking_tokens`; both are already part of the output-token count. + """ + await _run_coordinator(method, endpoint, request_cassette, response_cassette) + + span = _molto_span(exporter) + reasoning = span.attributes["gen_ai.usage.reasoning.output_tokens"] + assert reasoning == expected + assert span.attributes["gen_ai.usage.output_tokens"] >= reasoning + + +async def test_native_chat_stream_reports_reasoning_tokens(exporter: InMemorySpanExporter): + """Ensures a native Chat Completions stream reports `completion_tokens_details.reasoning_tokens`.""" + await _run_coordinator( + "chat_completions", + "chat/completions", + "matched/reasoning_raw/vllm-chat_completions-stream-00", + None, + ) + + span = _molto_span(exporter) + assert span.attributes["gen_ai.usage.reasoning.output_tokens"] == 15 + + +async def test_top_k_is_reported_on_native_messages_and_on_its_translated_rejection( + exporter: InMemorySpanExporter, +): + """Asserts that `top_k` reaches the span both where it is sent and where it is rejected. + + Native Messages forwards `top_k`; a translated route cannot express it and rejects + the call, and the span must show the requested value next to the error. + """ + await _run_coordinator( + "messages", + "messages", + "matched/simple_text/anthropic-messages-stream-00", + None, + top_k=40, + ) + async with Molto() as molto: + with pytest.raises(UnsupportedInput): + await molto.messages( + body={ + "model": "gpt-5", + "max_tokens": 8, + "messages": [{"role": "user", "content": "Hello!"}], + "top_k": 40, + }, + endpoint="responses", + api_key="test-key", + ) + + native, rejected = exporter.get_finished_spans() + assert native.attributes["gen_ai.request.top_k"] == 40 + assert native.status.status_code is StatusCode.UNSET + assert rejected.attributes["gen_ai.request.top_k"] == 40 + assert rejected.status.status_code is StatusCode.ERROR + assert rejected.attributes["error.type"] == "UnsupportedInput" + + +WEB_SEARCH = "matched/web_search" + + +@pytest.mark.parametrize( + ("method", "endpoint", "request_cassette", "response_cassette"), + [ + ("responses", "responses", f"{WEB_SEARCH}/openai-responses-stream-00", None), + ("responses", "responses", f"{WEB_SEARCH}/openai-responses-non_stream-00", None), + ("messages", "messages", f"{WEB_SEARCH}/anthropic-messages-stream-00", None), + ("messages", "messages", f"{WEB_SEARCH}/anthropic-messages-non_stream-00", None), + # The recorded Responses request sets no output-token limit, which a Messages + # upstream requires, so the translated case runs from Messages to Responses. + ( + "messages", + "responses", + f"{WEB_SEARCH}/anthropic-messages-stream-00", + f"{WEB_SEARCH}/openai-responses-stream-00", + ), + ], +) +async def test_web_search_requests_are_reported_on_every_route_that_counts_them( + exporter: InMemorySpanExporter, + method: str, + endpoint: str, + request_cassette: str, + response_cassette: str | None, +): + """Asserts that one recorded web search is reported as one request on native and translated routes. + + Responses reports the count in `tool_usage.web_search.num_requests` and Messages in + `usage.server_tool_use.web_search_requests`; translated routes read normalized usage. + """ + await _run_coordinator(method, endpoint, request_cassette, response_cassette) + + span = _molto_span(exporter) + assert span.attributes["molto.usage.web_search_requests"] == 1 + + +async def test_native_chat_web_search_reports_no_request_count(exporter: InMemorySpanExporter): + """Ensures a native Chat search call omits the count, because Chat Completions reports none.""" + await _run_coordinator( + "chat_completions", + "chat/completions", + f"{WEB_SEARCH}/openai-chat_completions-stream-00", + None, + ) + + assert "molto.usage.web_search_requests" not in _molto_span(exporter).attributes + + +async def test_chat_sampling_controls_reach_the_span_where_sent_and_where_rejected( + exporter: InMemorySpanExporter, +): + """Asserts that Chat seed, penalties, and choice count are recorded on native and rejected calls. + + Native Chat forwards them; a translated route cannot express them and rejects the + call, and the span must show the requested values next to the error. + """ + controls = {"seed": 7, "frequency_penalty": 0.1, "presence_penalty": 0.2, "n": 2} + expected = { + "gen_ai.request.seed": 7, + "gen_ai.request.frequency_penalty": 0.1, + "gen_ai.request.presence_penalty": 0.2, + "gen_ai.request.choice.count": 2, + } + await _run_coordinator( + "chat_completions", + "chat/completions", + "matched/simple_text/openai-chat_completions-stream-00", + None, + **controls, + ) + async with Molto() as molto: + with pytest.raises(UnsupportedInput): + await molto.chat_completions( + body={ + "model": "gpt-5", + "messages": [{"role": "user", "content": "Hello!"}], + **controls, + }, + endpoint="messages", + api_key="test-key", + ) + + native, rejected = exporter.get_finished_spans() + for span in (native, rejected): + assert {key: span.attributes[key] for key in expected} == expected + assert native.status.status_code is StatusCode.UNSET + assert rejected.status.status_code is StatusCode.ERROR + assert rejected.attributes["error.type"] == "UnsupportedInput" + + +async def test_incomplete_native_responses_stream_reports_its_reason( + exporter: InMemorySpanExporter, +): + """Ensures a native Responses stream ending in `response.incomplete` reports the incomplete reason.""" + await _run_coordinator( + "responses", "responses", "errors/incomplete_response/openai-responses-stream-00", None + ) + + span = _molto_span(exporter) + assert span.status.status_code is StatusCode.UNSET + assert span.attributes["gen_ai.response.finish_reasons"] == ("max_output_tokens",) + + +async def test_missing_credentials_report_no_http_status( + exporter: InMemorySpanExporter, monkeypatch: pytest.MonkeyPatch +): + """Asserts that a call failing for missing credentials reports no HTTP status. + + No request is sent; the outward SDK error carries a synthetic 401 that must not + be reported as an upstream response. + """ + monkeypatch.delenv("OPENAI_API_KEY", raising=False) + monkeypatch.setattr(ENV_CONFIG, "openai_api_key", SecretStr("")) + async with Molto(http_client=AsyncClient(transport=MockTransport(_not_found))) as molto: + with pytest.raises(Exception) as raised: + await molto.responses(body={"model": "gpt-5", "input": "Hello!"}) + + assert raised.value.response.status_code == 401 + span = _molto_span(exporter) + assert span.attributes["error.type"] == "AuthenticationError" + assert span.attributes["molto.error.phase"] == "setup" + assert "http.response.status_code" not in span.attributes + + +async def test_failure_after_a_successful_stream_response_reports_the_stream_phase( + exporter: InMemorySpanExporter, +): + """Asserts that a Responses error arriving on an HTTP 200 stream is a stream-phase failure. + + The first event arrives before the error, so the coordinator returns a stream + and the failure surfaces while the caller iterates. Its HTTP response succeeded, + so no status code is reported. + """ + body = _request_body("errors/context_length_exceeded/openai-responses-stream-00") + client = _replay("errors/context_length_exceeded/openai-responses-stream-00") + async with Molto(http_client=client) as molto: + stream = await molto.responses(body=body, api_key="test-key") + with pytest.raises(Exception) as raised: + async with stream: + [event async for event in stream] + + assert isinstance(raised.value.__cause__, ContextLengthExceededError) + span = _molto_span(exporter) + assert span.attributes["error.type"] == "ContextLengthExceededError" + assert span.attributes["molto.error.phase"] == "stream" + assert "http.response.status_code" not in span.attributes + + +async def test_coordinator_stream_closed_early_ends_its_span_once(exporter: InMemorySpanExporter): + """Ensures a coordinator stream closed after its first event ends its span once, without ERROR.""" + name = "matched/simple_text/openai-responses-stream-00" + async with Molto(http_client=_replay(name)) as molto: + stream = await molto.responses(body=_request_body(name), api_key="test-key") + await anext(stream) + await stream.aclose() + await stream.aclose() + + span = _molto_span(exporter) + assert span.status.status_code is StatusCode.UNSET + assert "error.type" not in span.attributes + + +async def test_streaming_coordinator_failure_before_the_first_event_ends_its_span( + exporter: InMemorySpanExporter, +): + """Asserts that a streaming call failing before its first event ends the span in the coordinator.""" + async with Molto(http_client=AsyncClient(transport=MockTransport(_not_found))) as molto: + with pytest.raises(Exception, match="secret-request-marker"): + await molto.responses( + body={"model": "gpt-5", "input": "Hello!", "stream": True}, api_key="test-key" + ) + + span = _molto_span(exporter) + assert span.status.status_code is StatusCode.ERROR + assert span.attributes["error.type"] == "ResourceNotFoundError" + assert span.attributes["molto.error.phase"] == "setup" + + +_CONTENT_FIELDS = { + "responses": {"input": CONTENT_MARKER}, + "chat/completions": {"messages": [{"role": "user", "content": CONTENT_MARKER}]}, + "messages": {"messages": [{"role": "user", "content": CONTENT_MARKER}]}, +} +_MARKED_EXTENSIONS = { + "api_key": f"{CONTENT_MARKER}-key", + "extra_body": {"metadata": {"note": CONTENT_MARKER}}, + "extra_headers": { + "x-trace-note": CONTENT_MARKER, + "authorization": f"Bearer {CONTENT_MARKER}", + }, +} + + +def _assert_carries_no_content(span: ReadableSpan) -> None: + """Spans hold only contract keys, and no recorded value contains the marker.""" + assert set(span.attributes) <= ATTRIBUTE_KEYS + recorded = [*span.attributes.values()] + for event in span.events: + assert set(event.attributes) <= EVENT_ATTRIBUTE_KEYS[event.name] + recorded += [event.name, *event.attributes.values()] + assert not any(CONTENT_MARKER in str(value) for value in recorded) + + +async def _consume(result: Any) -> None: + if isinstance(result, MoltoStream): + async with result: + [event async for event in result] + + +@pytest.mark.parametrize( + ("method", "protocol", "endpoint", "request_cassette", "response_cassette", "finish_reason"), + COORDINATOR_CASES, +) +async def test_spans_carry_no_content_headers_or_credentials( + exporter: InMemorySpanExporter, + method: str, + protocol: str, + endpoint: str, + request_cassette: str, + response_cassette: str | None, + finish_reason: str, +): + """Asserts that request content, extensions, headers, and credentials never reach a span. + + Covers every coordinator route. Spans hold only the contract's attribute keys, so + completion text cannot appear either. + """ + body = {**_request_body(request_cassette), **_CONTENT_FIELDS[protocol]} + async with Molto(http_client=_replay(response_cassette or request_cassette)) as molto: + await _consume( + await getattr(molto, method)(body=body, endpoint=endpoint, **_MARKED_EXTENSIONS) + ) + + _assert_carries_no_content(_molto_span(exporter)) + + +@pytest.mark.parametrize( + ("method", "protocol", "endpoint", "request_cassette", "response_cassette"), + [ + ( + "responses", + "responses", + "responses", + "matched/simple_text/openai-responses-non_stream-00", + "errors/not_found/openai-responses-non_stream-00", + ), + ( + "responses", + "responses", + "responses", + "matched/simple_text/openai-responses-stream-00", + "errors/not_found/openai-responses-stream-00", + ), + ( + "chat_completions", + "chat/completions", + "responses", + "matched/simple_text/openai-chat_completions-stream-00", + "errors/not_found/openai-responses-stream-00", + ), + ], +) +async def test_failed_spans_carry_no_content_headers_or_credentials( + exporter: InMemorySpanExporter, + method: str, + protocol: str, + endpoint: str, + request_cassette: str, + response_cassette: str, +): + """Asserts that a failed call's error attributes and exception event carry no request content.""" + body = {**_request_body(request_cassette), **_CONTENT_FIELDS[protocol]} + async with Molto(http_client=_replay(response_cassette)) as molto: + with pytest.raises(openai.NotFoundError): + await _consume( + await getattr(molto, method)(body=body, endpoint=endpoint, **_MARKED_EXTENSIONS) + ) + + span = _molto_span(exporter) + assert span.status.status_code is StatusCode.ERROR + _assert_carries_no_content(span) + + +@pytest.mark.parametrize( + ("endpoint", "response_cassette"), + [ + ("responses", None), + ("messages", "matched/simple_text/anthropic-messages-stream-00"), + ], +) +async def test_request_extensions_never_change_span_attributes( + exporter: InMemorySpanExporter, endpoint: str, response_cassette: str | None +): + """Asserts that `extra_body`, `extra_headers`, and the API key never influence the span. + + The same call with and without request extensions produces identical attributes, + so telemetry cannot be reading them. + """ + request_cassette = "matched/simple_text/openai-responses-stream-00" + body = _request_body(request_cassette) + for extensions in ({"api_key": "test-key"}, _MARKED_EXTENSIONS): + async with Molto(http_client=_replay(response_cassette or request_cassette)) as molto: + await _consume(await molto.responses(body=body, endpoint=endpoint, **extensions)) + + plain, extended = exporter.get_finished_spans() + assert dict(plain.attributes) == dict(extended.attributes) + + +class _StalledStream(AsyncByteStream): + """Send `head`, then wait forever, as an upstream that stops responding.""" + + def __init__(self, head: bytes, sent: asyncio.Event) -> None: + self._head = head + self._sent = sent + + async def __aiter__(self) -> AsyncIterator[bytes]: + if self._head: + yield self._head + self._sent.set() + await asyncio.Event().wait() + + +SIMPLE_STREAM = "matched/simple_text/openai-responses-stream-00" + + +def _first_event(name: str) -> bytes: + """The first recorded SSE event, after which a stalled upstream stops responding.""" + return ("".join(_cassette(name)["response"]["sse"]).split("\n\n")[0] + "\n\n").encode() + + +def _outcome(exporter: InMemorySpanExporter) -> str | None: + return _molto_span(exporter).attributes.get("molto.stream.outcome") + + +def _stalled_client(head: bytes, sent: asyncio.Event) -> AsyncClient: + def handler(request: Request) -> Response: + return Response( + 200, + headers={"content-type": "text/event-stream"}, + stream=_StalledStream(head, sent), + ) + + return AsyncClient(transport=MockTransport(handler)) + + +async def test_cancelling_a_coordinator_call_ends_its_span_without_error( + exporter: InMemorySpanExporter, +): + """Ensures cancelling a real coordinator call before its first event ends its span as UNSET. + + It reports `molto.stream.outcome=cancelled`, so a client hanging up before the + first token looks the same as one hanging up mid-generation. + """ + sent = asyncio.Event() + async with Molto(http_client=_stalled_client(b"", sent)) as molto: + call = asyncio.create_task( + molto.responses(body={**BODY, "stream": True}, api_key="test-key") + ) + await sent.wait() + call.cancel() + with pytest.raises(asyncio.CancelledError): + await call + + assert _outcome(exporter) == "cancelled" + _assert_cancelled(_molto_span(exporter)) + + +async def test_cancelling_a_coordinator_stream_ends_its_span_without_error( + exporter: InMemorySpanExporter, +): + """Ensures cancelling the task iterating a real coordinator stream ends its span once, as UNSET. + + It reports `molto.stream.outcome=cancelled`: a client hanging up mid-generation, + not an upstream failure. + """ + sent = asyncio.Event() + async with Molto(http_client=_stalled_client(_first_event(SIMPLE_STREAM), sent)) as molto: + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + consumer = asyncio.create_task(_consume(stream)) + await sent.wait() + consumer.cancel() + with pytest.raises(asyncio.CancelledError): + await consumer + + assert _outcome(exporter) == "cancelled" + _assert_cancelled(_molto_span(exporter)) + + +# Recorded calls compared with and without an SDK: native and translated, streaming and +# non-streaming, and an upstream failure in both modes. +FRESH_INTERPRETER_CASES = [ + { + "method": method, + "endpoint": endpoint, + "request": str(CASSETTES / f"{request}.yaml"), + "response": str(CASSETTES / f"{response or request}.yaml"), + } + for method, endpoint, request, response in [ + ("responses", "responses", "matched/simple_text/openai-responses-stream-00", None), + ("responses", "responses", "matched/simple_text/openai-responses-non_stream-00", None), + ( + "responses", + "chat/completions", + "matched/simple_text/openai-responses-stream-00", + "matched/simple_text/openai-chat_completions-stream-00", + ), + ( + "messages", + "responses", + "matched/simple_text/anthropic-messages-non_stream-00", + "matched/simple_text/openai-responses-stream-00", + ), + ( + "responses", + "responses", + "matched/simple_text/openai-responses-non_stream-00", + "errors/not_found/openai-responses-non_stream-00", + ), + ( + "responses", + "responses", + "matched/simple_text/openai-responses-stream-00", + "errors/not_found/openai-responses-stream-00", + ), + ] +] +_FRESH_INTERPRETER_PREAMBLE = ( + "import asyncio, json, sys\n" + "from molto.utils.test import replayed_outcomes\n" + "cases = json.loads(sys.argv[1])\n" +) + + +def _run_in_fresh_interpreter(code: str, cases: list[dict[str, Any]]) -> Any: + """Run `code` in a new interpreter with `cases` bound, and return its JSON stdout. + + A new interpreter has no tracer provider unless `code` installs one; the API + lets a process set its global provider only once. The conftest network block does + not apply there, but `replayed_outcomes` serves every response through + `MockTransport`, so no request leaves the process. + """ + completed = subprocess.run( + [sys.executable, "-c", _FRESH_INTERPRETER_PREAMBLE + code, json_dumps(cases)], + capture_output=True, + text=True, + timeout=60, + ) + assert completed.returncode == 0, completed.stderr + return json_loads(completed.stdout) + + +def _without_item_ids(value: Any) -> Any: + if isinstance(value, list): + return [_without_item_ids(item) for item in value] + if not isinstance(value, dict): + return value + return { + key: _without_item_ids(item) + for key, item in value.items() + if key != "item_id" and not (key == "id" and "type" in value) + } + + +def _comparable(case: dict[str, Any], outcome: dict[str, Any]) -> dict[str, Any]: + """Drop the output-item IDs a translated Responses route generates afresh on every call. + + They differ between any two runs, traced or not (`python/ARCHITECTURE.md`, + "Responses through Chat Completions"); every other value is compared exactly. + """ + translated_responses = case["method"] == "responses" and case["endpoint"] != "responses" + return _without_item_ids(outcome) if translated_responses else outcome + + +async def test_calls_behave_identically_without_an_sdk(exporter: InMemorySpanExporter): + """Asserts that with no SDK configured, every call yields exactly what a traced call yields. + + Covers native and translated routes, streaming and non-streaming, and failures, + comparing events, responses, and error classes and messages. The SDK must never be + imported, because Molto depends only on the API. + """ + traced = await replayed_outcomes(FRESH_INTERPRETER_CASES) + + untraced = _run_in_fresh_interpreter( + "outcomes = asyncio.run(replayed_outcomes(cases))\n" + "sdk_imported = 'opentelemetry.sdk' in sys.modules\n" + "print(json.dumps({'outcomes': outcomes, 'sdk_imported': sdk_imported}))\n", + FRESH_INTERPRETER_CASES, + ) + + assert [ + _comparable(case, outcome) + for case, outcome in zip(FRESH_INTERPRETER_CASES, untraced["outcomes"], strict=True) + ] == [ + _comparable(case, outcome) + for case, outcome in zip(FRESH_INTERPRETER_CASES, traced, strict=True) + ] + assert untraced["sdk_imported"] is False + assert [outcome.keys() & {"events", "response", "error"} for outcome in traced] == [ + {"events"}, + {"response"}, + {"events"}, + {"response"}, + {"error"}, + {"error"}, + ] + assert len(exporter.get_finished_spans()) == len(FRESH_INTERPRETER_CASES) + + +def test_spans_use_the_application_tracer_provider(): + """Ensures Molto's tracer comes from the application's global provider, scoped as `molto`. + + The other tests route the tracer to a local provider; this checks the real + accessor in an interpreter whose application installs the SDK globally. + """ + result = _run_in_fresh_interpreter( + "from opentelemetry import trace\n" + "from opentelemetry.sdk.trace import TracerProvider\n" + "from opentelemetry.sdk.trace.export import SimpleSpanProcessor\n" + "from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter\n" + "exporter = InMemorySpanExporter()\n" + "provider = TracerProvider()\n" + "provider.add_span_processor(SimpleSpanProcessor(exporter))\n" + "trace.set_tracer_provider(provider)\n" + "asyncio.run(replayed_outcomes(cases))\n" + "(span,) = exporter.get_finished_spans()\n" + "scope = span.instrumentation_scope\n" + "print(json.dumps({'name': span.name, 'scope': scope.name, 'version': scope.version}))\n", + FRESH_INTERPRETER_CASES[:1], + ) + + assert result == {"name": "chat gpt-4.1-nano", "scope": "molto", "version": __version__} + + +async def test_consumed_stream_reports_completed(exporter: InMemorySpanExporter): + """Ensures a stream the caller reads to its end reports `molto.stream.outcome=completed`.""" + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + await _consume( + await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + ) + + assert _outcome(exporter) == "completed" + assert _molto_span(exporter).status.status_code is StatusCode.UNSET + + +async def test_stream_closed_before_its_end_reports_closed(exporter: InMemorySpanExporter): + """Ensures a stream the caller closes before its end reports `molto.stream.outcome=closed`.""" + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + await anext(stream) + await stream.aclose() + + assert _outcome(exporter) == "closed" + assert _molto_span(exporter).status.status_code is StatusCode.UNSET + + +async def test_closing_a_finished_stream_keeps_completed(exporter: InMemorySpanExporter): + """Asserts that closing a stream after its end does not relabel it `closed`.""" + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + [event async for event in stream] + await stream.aclose() + + assert _outcome(exporter) == "completed" + + +async def test_stream_failing_mid_generation_reports_error(exporter: InMemorySpanExporter): + """Asserts that an upstream failure while the caller reads a stream reports `error`.""" + name = "errors/context_length_exceeded/openai-responses-stream-00" + async with Molto(http_client=_replay(name)) as molto: + stream = await molto.responses(body=_request_body(name), api_key="test-key") + with pytest.raises(openai.APIError): + await _consume(stream) + + span = _molto_span(exporter) + assert span.attributes["molto.stream.outcome"] == "error" + assert span.attributes["molto.error.phase"] == "stream" + assert span.status.status_code is StatusCode.ERROR + + +async def test_streaming_call_failing_before_its_first_event_reports_error( + exporter: InMemorySpanExporter, +): + """Asserts that a streaming call failing before it returns a stream reports `error`.""" + name = "errors/not_found/openai-responses-stream-00" + async with Molto(http_client=_replay(name)) as molto: + with pytest.raises(openai.NotFoundError): + await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + + span = _molto_span(exporter) + assert span.attributes["molto.stream.outcome"] == "error" + assert span.attributes["molto.error.phase"] == "setup" + + +async def test_dropped_stream_ends_its_span_as_abandoned_at_its_last_event( + exporter: InMemorySpanExporter, +): + """Ensures a stream dropped without being consumed or closed still ends its span. + + A leaked stream reports `molto.stream.outcome=abandoned` when it is garbage + collected, ending at the last event the caller received rather than at collection + time, so its duration is not inflated. + """ + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + await anext(stream) + before_last_event = time_ns() + await anext(stream) + after_last_event = time_ns() + await asyncio.sleep(0.05) + del stream + gc.collect() + span = _molto_span(exporter) + + assert span.attributes["molto.stream.outcome"] == "abandoned" + assert before_last_event <= span.end_time <= after_last_event + assert span.status.status_code is StatusCode.UNSET + + +class _TrackedBody(AsyncByteStream): + """A recorded response body that notes when `httpx` releases it.""" + + def __init__(self, content: bytes, closed: list[bool]) -> None: + self._content = content + self._closed = closed + + async def __aiter__(self) -> AsyncIterator[bytes]: + # Small chunks, so the dropped stream still has unread body left. + for start in range(0, len(self._content), 64): + yield self._content[start : start + 64] + + async def aclose(self) -> None: + self._closed.append(True) + + +@pytest.mark.parametrize( + ("method", "endpoint", "cassette"), + [ + ("responses", "responses", SIMPLE_STREAM), + ("chat_completions", "messages", "matched/simple_text/anthropic-messages-stream-00"), + ], + ids=["native", "translated"], +) +async def test_dropped_stream_releases_its_upstream_response( + exporter: InMemorySpanExporter, method: str, endpoint: str, cassette: str +): + """Ensures a dropped, unclosed stream still releases its upstream HTTP response. + + `__del__` cannot await `aclose()`, but the event loop's async-generator finalizer + closes the dropped source on its next turns, so the docs must not call it a leak. + """ + recorded = _cassette(cassette) + closed: list[bool] = [] + + def handler(request: Request) -> Response: + return Response( + 200, + stream=_TrackedBody("".join(recorded["response"]["sse"]).encode(), closed), + headers={"content-type": "text/event-stream"}, + ) + + async with Molto(http_client=AsyncClient(transport=MockTransport(handler))) as molto: + stream = await getattr(molto, method)( + body=recorded["request"]["body"], endpoint=endpoint, api_key="test-key" + ) + await anext(stream) + del stream + gc.collect() + for _ in range(5): + await asyncio.sleep(0) + + assert _outcome(exporter) == "abandoned" + assert closed == [True] + + +async def test_dropped_stream_swallows_telemetry_failures( + monkeypatch: pytest.MonkeyPatch, +): + """Ensures a failure while ending an abandoned span never escapes `__del__`. + + Python only prints exceptions from `__del__`, and at interpreter shutdown the SDK + or exporter may already be gone, so telemetry must stay silent there. + """ + unraisable = [] + monkeypatch.setattr(sys, "unraisablehook", unraisable.append) + + class BrokenSpan: + def is_recording(self) -> bool: + return True + + def set_attribute(self, key: str, value: object) -> None: + raise RuntimeError("exporter already shut down") + + stream = MoltoStream([], _events(), Headers()) + stream.attach_span(BrokenSpan()) + del stream + gc.collect() + + assert unraisable == [] + + +async def test_unread_dropped_stream_ends_at_its_first_event(exporter: InMemorySpanExporter): + """Ensures a stream dropped before any read ends at the prefetched first event.""" + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + before_call = time_ns() + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + after_call = time_ns() + await asyncio.sleep(0.05) + del stream + gc.collect() + span = _molto_span(exporter) + + assert span.attributes["molto.stream.outcome"] == "abandoned" + assert before_call <= span.end_time <= after_call + + +@pytest.mark.parametrize("close", [False, True], ids=["consumed", "closed"]) +async def test_dropping_a_finished_stream_keeps_its_outcome( + exporter: InMemorySpanExporter, close: bool +): + """Asserts that garbage collection never relabels or re-ends a stream that already ended.""" + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + if close: + await stream.aclose() + else: + [event async for event in stream] + del stream + gc.collect() + + assert _outcome(exporter) == ("closed" if close else "completed") + + +NON_STREAM_BODY_CASSETTE = "matched/simple_text/openai-responses-non_stream-00" + + +async def test_successful_non_streaming_call_reports_no_stream_outcome( + exporter: InMemorySpanExporter, +): + """Asserts that `molto.stream.outcome` is absent from a successful non-streaming span.""" + async with Molto(http_client=_replay(NON_STREAM_BODY_CASSETTE)) as molto: + await molto.responses(body=_request_body(NON_STREAM_BODY_CASSETTE), api_key="test-key") + + assert _outcome(exporter) is None + + +async def test_failed_non_streaming_call_reports_no_stream_outcome( + exporter: InMemorySpanExporter, +): + """Asserts that `molto.stream.outcome` is absent from a failed non-streaming span.""" + async with Molto( + http_client=_replay("errors/not_found/openai-responses-non_stream-00") + ) as molto: + with pytest.raises(openai.NotFoundError): + await molto.responses(body=_request_body(NON_STREAM_BODY_CASSETTE), api_key="test-key") + + span = _molto_span(exporter) + assert span.attributes["error.type"] == "ResourceNotFoundError" + assert _outcome(exporter) is None + + +@pytest.mark.parametrize( + ("endpoint", "cassette"), + [ + ("responses", SIMPLE_STREAM), + ("chat/completions", "matched/simple_text/openai-chat_completions-stream-00"), + ], + ids=["native", "translated"], +) +async def test_integer_stream_is_streamed_and_traced_as_streaming( + exporter: InMemorySpanExporter, endpoint: str, cassette: str +): + """Ensures `stream: 1` returns a stream on both routes and is traced as streaming. + + The SDKs stream for any truthy `stream`, so native passthrough must expect a + stream rather than fail on it; translated validation also accepts `1`. + """ + body = {**_request_body(NON_STREAM_BODY_CASSETTE), "stream": 1} + async with Molto(http_client=_replay(cassette)) as molto: + result = await molto.responses(body=body, endpoint=endpoint, api_key="test-key") + await _consume(result) + + span = _molto_span(exporter) + assert isinstance(result, MoltoStream) + assert span.attributes["gen_ai.request.stream"] is True + assert _outcome(exporter) == "completed" + assert [event.name for event in span.events] == ["molto.first_token"] + + +@pytest.mark.parametrize( + ("native", "stream", "expected"), + [ + (True, True, True), + (True, 1, True), + (True, "false", True), + (True, 0, False), + (True, None, False), + (False, True, True), + (False, 1, True), + (False, 1.0, True), + (False, "true", False), + (False, 0, False), + ], + ids=[ + "native-true", + "native-1", + "native-string", + "native-0", + "native-absent", + "translated-true", + "translated-1", + "translated-float", + "translated-string", + "translated-0", + ], +) +def test_streaming_rule_matches_each_route(native: bool, stream: object, expected: bool): + """Ensures the one streaming rule shared by routing and the span matches each route. + + Native passthrough streams for any truthy value, as the SDKs do; translated + validation accepts `1` and `1.0` but rejects strings. + """ + assert _streams({"model": "gpt-5", "stream": stream}, native=native) is expected + + +async def test_recording_failure_never_fails_a_native_call( + exporter: InMemorySpanExporter, monkeypatch: pytest.MonkeyPatch +): + """Asserts that an extractor raising on unexpected native output still returns the response. + + Native output is unvalidated, so telemetry may lose attributes but must never turn + a successful call into a failure. + """ + + def broken(response: object) -> dict: + raise TypeError("unexpected provider output") + + monkeypatch.setattr("molto.core.responses_response_attributes", broken) + async with Molto(http_client=_replay(NON_STREAM_BODY_CASSETTE)) as molto: + response = await molto.responses( + body=_request_body(NON_STREAM_BODY_CASSETTE), api_key="test-key" + ) + + assert response.id == _recorded_response_id(NON_STREAM_BODY_CASSETTE) + assert _molto_span(exporter).status.status_code is StatusCode.UNSET + + +async def test_recording_failure_never_fails_a_native_stream( + exporter: InMemorySpanExporter, monkeypatch: pytest.MonkeyPatch +): + """Asserts that a recorder raising on every native event still delivers the whole stream.""" + + def broken(recorder: object, event: object) -> None: + raise TypeError("unexpected provider output") + + async def read() -> list[Any]: + async with Molto(http_client=_replay(SIMPLE_STREAM)) as molto: + stream = await molto.responses(body=_request_body(SIMPLE_STREAM), api_key="test-key") + return [event async for event in stream] + + expected = await read() + monkeypatch.setattr("molto.core.SpanRecorder.record_responses_event", broken) + events = await read() + + assert events == expected + assert [span.attributes["molto.stream.outcome"] for span in exporter.get_finished_spans()] == [ + "completed", + "completed", + ] + + +HTTPX_SCOPE = "opentelemetry.instrumentation.httpx" + + +class HttpSpanCase(NamedTuple): + """One coordinator call whose upstream HTTP request should be traced.""" + + method: str + endpoint: str + request: str + stream: bool + response_override: str | None = None + + @property + def response(self) -> str: + """The recording that answers the call: the override, or the request's own.""" + return self.response_override or self.request + + +HTTP_SPAN_CASES = [ + HttpSpanCase("responses", "responses", "matched/simple_text/openai-responses-stream-00", True), + HttpSpanCase( + "responses", "responses", "matched/simple_text/openai-responses-non_stream-00", False + ), + HttpSpanCase( + "responses", + "messages", + "matched/simple_text/openai-responses-stream-00", + True, + "matched/simple_text/anthropic-messages-stream-00", + ), + HttpSpanCase( + "messages", + "responses", + "matched/simple_text/anthropic-messages-non_stream-00", + False, + "matched/simple_text/openai-responses-stream-00", + ), +] + + +def _case_id(case: HttpSpanCase) -> str: + route = "native" if case.method.replace("_", "/") == case.endpoint else "translated" + return f"{case.method}-{route}-{'stream' if case.stream else 'non_stream'}" + + +async def _call(molto: Molto, case: HttpSpanCase) -> None: + """Make the case's coordinator call and consume any stream.""" + await _consume( + await getattr(molto, case.method)( + body=_request_body(case.request), endpoint=case.endpoint, api_key="test-key" + ) + ) + + +@contextmanager +def _instrumented_httpx(tracer_provider: TracerProvider) -> Iterator[None]: + """Instrument `httpx` globally, as host applications do, for the duration of the block. + + A no-op meter provider keeps the instrumentor's duration histograms off the + process-wide meter provider, so no other test sees them. + """ + instrumentor = HTTPXClientInstrumentor() + instrumentor.instrument(tracer_provider=tracer_provider, meter_provider=NoOpMeterProvider()) + try: + yield + finally: + instrumentor.uninstrument() + + +def _replay_network(monkeypatch: pytest.MonkeyPatch, cassette_name: str) -> None: + """Answer httpx's network transport from a recording, so no request leaves the process. + + Global instrumentation wraps this transport, which every default `AsyncClient` uses. + """ + replay = replay_transport(_cassette(cassette_name)) + + async def handle_async_request(self: AsyncHTTPTransport, request: Request) -> Response: + return await replay.handle_async_request(request) + + monkeypatch.setattr(AsyncHTTPTransport, "handle_async_request", handle_async_request) + + +def _http_spans(exporter: InMemorySpanExporter) -> list[ReadableSpan]: + return [ + span + for span in exporter.get_finished_spans() + if span.instrumentation_scope.name == HTTPX_SCOPE + ] + + +def _assert_http_span_nests_under_molto_span( + exporter: InMemorySpanExporter, case: HttpSpanCase +) -> None: + """Assert one HTTP CLIENT span is a child of the Molto span, ending at the response headers. + + For a stream, the HTTP span ends before the first token: it measures only the time + to first byte, while the Molto span covers the whole generation. + """ + (http_span,) = _http_spans(exporter) + (molto_span,) = [ + span + for span in exporter.get_finished_spans() + if span.instrumentation_scope.name == "molto" + ] + assert http_span.kind is SpanKind.CLIENT + assert http_span.parent.span_id == molto_span.context.span_id + assert http_span.context.trace_id == molto_span.context.trace_id + first_tokens = [event for event in molto_span.events if event.name == "molto.first_token"] + if case.stream: + (first_token,) = first_tokens + assert http_span.end_time <= first_token.timestamp + else: + assert first_tokens == [] + + +@pytest.mark.parametrize("injected", [False, True], ids=["default_client", "injected_client"]) +@pytest.mark.parametrize("case", HTTP_SPAN_CASES, ids=_case_id) +async def test_global_httpx_instrumentation_nests_http_spans_under_the_molto_span( + exporter: InMemorySpanExporter, + tracer_provider: TracerProvider, + monkeypatch: pytest.MonkeyPatch, + injected: bool, + case: HttpSpanCase, +): + """Ensures globally instrumented `httpx` spans parent to the Molto span on every route. + + Covers Molto's default client and an injected one, both created before the + instrumentation is installed, for native and translated, streaming and + non-streaming calls. This guards the rule that the Molto span is current while + the upstream request is sent. + """ + _replay_network(monkeypatch, case.response) + molto = Molto(http_client=AsyncClient() if injected else None) + with _instrumented_httpx(tracer_provider): + async with molto: + await _call(molto, case) + + _assert_http_span_nests_under_molto_span(exporter, case) + + +@pytest.mark.parametrize("case", HTTP_SPAN_CASES, ids=_case_id) +async def test_instrumented_client_nests_http_spans_under_the_molto_span( + exporter: InMemorySpanExporter, tracer_provider: TracerProvider, case: HttpSpanCase +): + """Ensures an injected client instrumented with `instrument_client` nests its spans under Molto's. + + This is the per-client setup `docs/observability.md` shows, which works with any + `httpx` transport. + """ + http_client = _replay(case.response) + HTTPXClientInstrumentor.instrument_client( + http_client, tracer_provider=tracer_provider, meter_provider=NoOpMeterProvider() + ) + async with Molto(http_client=http_client) as molto: + await _call(molto, case) + + _assert_http_span_nests_under_molto_span(exporter, case) + + +async def test_global_httpx_instrumentation_ignores_custom_transports( + exporter: InMemorySpanExporter, tracer_provider: TracerProvider +): + """Asserts that global instrumentation does not trace a client built on a custom transport. + + It wraps only `httpx`'s network transport, so a `MockTransport` client produces + no HTTP span; `docs/observability.md` points such clients to `instrument_client`. + """ + case = HTTP_SPAN_CASES[0] + with _instrumented_httpx(tracer_provider): + async with Molto(http_client=_replay(case.response)) as molto: + await _call(molto, case) + + assert _http_spans(exporter) == [] + assert len(exporter.get_finished_spans()) == 1 diff --git a/tests/utils/test_telemetry.py b/tests/utils/test_telemetry.py new file mode 100644 index 0000000..679f61d --- /dev/null +++ b/tests/utils/test_telemetry.py @@ -0,0 +1,796 @@ +from dataclasses import replace +from types import SimpleNamespace +from typing import Any, get_args + +import openai +import pytest +from anthropic.types import Message +from httpx import Request, Response +from openai.types.chat import ChatCompletion, ChatCompletionChunk +from openai.types.chat.chat_completion import Choice as ChatChoice +from openai.types.chat.chat_completion_chunk import Choice as ChunkChoice +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor +from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter +from pydantic import BaseModel + +from molto.providers import Provider +from molto.types.anthropic_messages import AnthropicMessagesCreateRequest +from molto.types.exceptions import AuthenticationError, ContextLengthExceededError +from molto.types.normalized import PartDeltaEvent, TextDelta, ThinkingDelta +from molto.types.openai_chat_completions import OpenAIChatCompletionsCreateRequest +from molto.types.openai_responses import OpenAIResponsesCreateRequest +from molto.utils.telemetry import ( + MessagesUsage, + ResolvedRoute, + SpanRecorder, + chat_completions_response_attributes, + error_attributes, + exception_event_attributes, + is_chat_completions_content_delta, + is_messages_content_delta, + is_responses_content_delta, + messages_response_attributes, + observe, + responses_response_attributes, + span_name, + start_attributes, +) + +NATIVE_RESPONSES = ResolvedRoute( + protocol="responses", + endpoint="responses", + provider="openai", + base_url="https://api.openai.com/v1", +) + + +def _start( + body: dict[str, Any] | BaseModel, *, streaming: bool = False, **route: Any +) -> dict[str, Any]: + """Map `body` on the native Responses route, with any route fields overridden.""" + return start_attributes(replace(NATIVE_RESPONSES, **route), body, streaming=streaming) + + +def test_native_request_maps_every_start_attribute(): + """Asserts that a native request yields the full documented start-attribute set.""" + attributes = _start( + { + "model": "gpt-5", + "input": "Hello!", + "max_output_tokens": 256, + "temperature": 0.5, + "top_p": 0.9, + "stream": True, + }, + streaming=True, + ) + + assert attributes == { + "gen_ai.operation.name": "chat", + "gen_ai.provider.name": "openai", + "gen_ai.request.model": "gpt-5", + "gen_ai.request.max_tokens": 256, + "gen_ai.request.temperature": 0.5, + "gen_ai.request.top_p": 0.9, + "gen_ai.request.stream": True, + "server.address": "api.openai.com", + "server.port": 443, + "molto.downstream.protocol": "responses", + "molto.upstream.endpoint": "responses", + "molto.route": "native", + } + + +@pytest.mark.parametrize( + ("protocol", "body", "max_tokens"), + [ + ( + "responses", + OpenAIResponsesCreateRequest( + model="gpt-5", + input="Hello!", + max_output_tokens=256, + temperature=0.5, + top_p=0.9, + stream=True, + ), + 256, + ), + ( + "chat/completions", + OpenAIChatCompletionsCreateRequest( + model="gpt-5", + messages=[{"role": "user", "content": "Hello!"}], + max_completion_tokens=256, + temperature=0.5, + top_p=0.9, + stream=True, + ), + 256, + ), + ( + "messages", + AnthropicMessagesCreateRequest( + model="claude-sonnet-5", + max_tokens=256, + messages=[{"role": "user", "content": "Hello!"}], + temperature=0.5, + top_p=0.9, + stream=True, + ), + 256, + ), + ], +) +def test_translated_request_models_map_request_fields( + protocol: str, body: BaseModel, max_tokens: int +): + """Ensures each protocol's validated request model is read for every request attribute. + + Translated coordinators pass the validated model rather than a dictionary, so the + model path must cover each protocol's output-token field and the shared fields. + """ + attributes = _start(body, streaming=True, protocol=protocol, endpoint="messages") + + assert attributes["gen_ai.request.max_tokens"] == max_tokens + assert attributes["gen_ai.request.temperature"] == 0.5 + assert attributes["gen_ai.request.top_p"] == 0.9 + assert attributes["gen_ai.request.stream"] is True + assert attributes["molto.downstream.protocol"] == protocol + + +def test_route_is_translated_when_endpoint_differs_from_protocol(): + """Asserts that the route is native only when the upstream endpoint matches the protocol.""" + attributes = _start({"model": "gpt-5"}, protocol="messages", endpoint="responses") + + assert attributes["molto.downstream.protocol"] == "messages" + assert attributes["molto.upstream.endpoint"] == "responses" + assert attributes["molto.route"] == "translated" + + +def test_non_streaming_call_omits_stream(): + """Ensures `gen_ai.request.stream` is set only for streaming calls, as the convention requires. + + The coordinator decides whether a call streams; telemetry never reads `stream` itself. + """ + body = OpenAIResponsesCreateRequest(model="gpt-5", input="Hello!", stream=True) + + assert "gen_ai.request.stream" not in _start(body, streaming=False) + + +def test_unset_optional_fields_are_omitted(): + """Asserts that optional request attributes appear only when the request sets them.""" + attributes = _start( + {"model": "qwen3", "messages": []}, + protocol="chat/completions", + endpoint="chat/completions", + provider="vllm", + ) + + assert set(attributes) == { + "gen_ai.operation.name", + "gen_ai.provider.name", + "gen_ai.request.model", + "server.address", + "server.port", + "molto.downstream.protocol", + "molto.upstream.endpoint", + "molto.route", + } + + +@pytest.mark.parametrize( + ("body", "expected"), + [ + ({"max_completion_tokens": 64, "max_tokens": 32}, 64), + ({"max_tokens": 32}, 32), + ({"max_completion_tokens": "64", "max_tokens": 32}, 32), + ], +) +def test_chat_max_tokens_prefers_valid_max_completion_tokens(body: dict, expected: int): + """Ensures Chat reports `max_completion_tokens`, falling back to the deprecated `max_tokens`. + + A mistyped preferred field must not hide a valid fallback value. + """ + attributes = _start( + {"model": "gpt-5", "messages": [], **body}, + protocol="chat/completions", + endpoint="chat/completions", + ) + + assert attributes["gen_ai.request.max_tokens"] == expected + + +def test_chat_request_model_reads_deprecated_max_tokens(): + """Ensures the Chat fallback also applies to a validated request model.""" + body = OpenAIChatCompletionsCreateRequest(model="gpt-5", messages=[], max_tokens=32) + + attributes = _start(body, protocol="chat/completions", endpoint="messages") + + assert attributes["gen_ai.request.max_tokens"] == 32 + + +def test_integer_sampling_values_are_reported_as_floats(): + """Ensures temperature and top_p keep the convention's double type for integer input.""" + attributes = _start( + {"model": "claude-sonnet-5", "max_tokens": 8, "temperature": 1, "top_p": 1}, + protocol="messages", + endpoint="messages", + provider="anthropic", + ) + + assert attributes["gen_ai.request.temperature"] == 1.0 + assert isinstance(attributes["gen_ai.request.temperature"], float) + assert isinstance(attributes["gen_ai.request.top_p"], float) + + +def test_messages_top_k_is_reported_on_native_and_validated_requests(): + """Ensures `top_k`, which only Messages defines, is reported as the convention's integer. + + A translated route rejects `top_k`, so its span must still show the field that + caused the rejection; that path passes the validated request model. + """ + native = _start( + {"model": "claude-sonnet-5", "max_tokens": 8, "top_k": 40}, + protocol="messages", + endpoint="messages", + provider="anthropic", + ) + translated = _start( + AnthropicMessagesCreateRequest( + model="claude-sonnet-5", max_tokens=8, messages=[], top_k=40 + ), + protocol="messages", + endpoint="responses", + ) + + assert native["gen_ai.request.top_k"] == 40 + assert translated["gen_ai.request.top_k"] == 40 + + +def test_native_body_reports_a_field_its_request_type_does_not_define(): + """Ensures a native body's sampling field is reported as sent, whatever protocol defines it. + + Native passthrough forwards fields its request type does not define, such as `top_k` + on a vLLM Chat request, so the span must report what was actually sent. + """ + attributes = _start( + {"model": "qwen3", "messages": [], "top_k": 20}, + protocol="chat/completions", + endpoint="chat/completions", + provider="vllm", + ) + + assert attributes["gen_ai.request.top_k"] == 20 + + +def test_chat_sampling_controls_are_reported_with_convention_types(): + """Ensures Chat penalties, seed, and choice count use the convention's keys and types. + + Penalties are doubles even for integer input, and `seed` and `n` stay integers. + """ + attributes = _start( + { + "model": "gpt-5", + "messages": [], + "frequency_penalty": 1, + "presence_penalty": 0.5, + "seed": 7, + "n": 3, + }, + protocol="chat/completions", + endpoint="chat/completions", + ) + + assert attributes["gen_ai.request.frequency_penalty"] == 1.0 + assert isinstance(attributes["gen_ai.request.frequency_penalty"], float) + assert attributes["gen_ai.request.presence_penalty"] == 0.5 + assert attributes["gen_ai.request.seed"] == 7 + assert attributes["gen_ai.request.choice.count"] == 3 + + +def test_default_choice_count_is_omitted(): + """Asserts that `n=1` is not reported, since the convention reports only a count other than one.""" + attributes = _start( + {"model": "gpt-5", "messages": [], "n": 1}, + protocol="chat/completions", + endpoint="chat/completions", + ) + + assert "gen_ai.request.choice.count" not in attributes + + +def test_mistyped_native_fields_are_omitted_without_raising(): + """Asserts that unvalidated native values of the wrong type never become attributes. + + Native bodies reach the SDK unvalidated; telemetry must not reject them or + report a value that does not match the convention's attribute type. Booleans + are rejected as numbers even though `bool` subclasses `int`. + """ + attributes = _start( + { + "model": 123, + "max_output_tokens": True, + "temperature": "0.5", + "top_p": True, + "top_k": 40.0, + "frequency_penalty": "0.1", + "presence_penalty": False, + "seed": "7", + "n": True, + } + ) + + for key in ( + "gen_ai.request.model", + "gen_ai.request.max_tokens", + "gen_ai.request.temperature", + "gen_ai.request.top_p", + "gen_ai.request.top_k", + "gen_ai.request.frequency_penalty", + "gen_ai.request.presence_penalty", + "gen_ai.request.seed", + "gen_ai.request.choice.count", + ): + assert key not in attributes + + +def test_empty_model_is_treated_as_absent(): + """Ensures an empty model neither becomes an attribute nor leaves a trailing space in the name.""" + attributes = _start({"model": ""}) + + assert "gen_ai.request.model" not in attributes + assert span_name(attributes) == "chat" + + +@pytest.mark.parametrize( + ("base_url", "expected"), + [ + ("http://localhost:8000/v1", {"server.address": "localhost", "server.port": 8000}), + ("http://127.0.0.1/v1", {"server.address": "127.0.0.1", "server.port": 80}), + ("https://api.anthropic.com", {"server.address": "api.anthropic.com", "server.port": 443}), + ], +) +def test_server_attributes_use_explicit_or_scheme_default_port(base_url: str, expected: dict): + """Ensures `server.port` is always set with `server.address`, as the convention requires.""" + attributes = _start({"model": "gpt-5"}, base_url=base_url) + + assert {key: attributes[key] for key in expected} == expected + + +@pytest.mark.parametrize("base_url", ["http://[::1", "localhost:8000", ""]) +def test_unusable_base_url_omits_server_attributes_without_raising(base_url: str): + """Asserts that a malformed or host-less base URL leaves its error to the SDK at transport.""" + attributes = _start({"model": "gpt-5"}, base_url=base_url) + + assert "server.address" not in attributes + assert "server.port" not in attributes + + +def test_span_name_includes_model_when_present(): + """Ensures spans are named `{operation} {model}`, the GenAI inference-span format.""" + assert span_name({"gen_ai.request.model": "gpt-5"}) == "chat gpt-5" + assert span_name({}) == "chat" + + +def test_provider_set_is_pinned_for_gen_ai_naming(): + """Ensures adding a provider forces a decision about its gen_ai.provider.name. + + Molto reports the provider name unchanged. Well-known convention values such as + `aws.bedrock` or `gcp.gemini` differ from the obvious Molto names, so a new provider + must either match the convention or add a mapping before this test is updated. + """ + assert set(get_args(Provider)) == {"openai", "anthropic", "vllm"} + + +REQUEST = Request("POST", "https://api.openai.com/v1/responses") + + +def _raised_from(error: Exception, cause: Exception) -> Exception: + """Return `error` raised from `cause`, with tracebacks, as Molto's error chain does.""" + try: + try: + raise cause + except Exception as caught: + raise error from caught + except Exception as raised: + return raised + + +def test_missing_credentials_omit_the_synthetic_status_code(): + """Asserts that the synthetic 401 on a missing-credentials error is not reported as HTTP. + + No HTTP call happens: the 401 exists only on the outward SDK exception, and the + normalized cause has no response. + """ + outward = openai.AuthenticationError( + "Missing credentials.", response=Response(401, request=REQUEST), body=None + ) + error = _raised_from( + outward, AuthenticationError("Missing credentials.", response=None, body=None) + ) + + attributes = error_attributes(error, "setup") + + assert attributes == {"error.type": "AuthenticationError", "molto.error.phase": "setup"} + + +def test_error_on_a_successful_stream_response_omits_the_status_code(): + """Asserts that a failure arriving on an HTTP 200 stream does not report status 200.""" + cause = ContextLengthExceededError( + "context length exceeded", response=Response(200, request=REQUEST), body=None + ) + error = _raised_from(openai.APIError("context length exceeded", REQUEST, body=None), cause) + + attributes = error_attributes(error, "stream") + + assert attributes == { + "error.type": "ContextLengthExceededError", + "molto.error.phase": "stream", + } + + +def test_unrecognized_sdk_error_reports_its_class_and_status(): + """Ensures an error that passes through unchanged keeps its own class and HTTP status.""" + error = openai.RateLimitError( + "rate limited", response=Response(429, request=REQUEST), body=None + ) + + attributes = error_attributes(error, "setup") + + assert attributes == { + "error.type": "RateLimitError", + "http.response.status_code": 429, + "molto.error.phase": "setup", + } + + +def test_exception_event_attributes_carry_no_message_from_the_cause_chain(): + """Asserts that the recorded exception keeps types and frames but no chained message text.""" + error = _raised_from( + RuntimeError("outer secret-request-marker"), + ValueError("inner secret-request-marker"), + ) + + attributes = exception_event_attributes(error) + + assert attributes["exception.message"] == "RuntimeError" + stacktrace = attributes["exception.stacktrace"] + assert "secret-request-marker" not in stacktrace + assert "builtins.RuntimeError" in stacktrace + assert "builtins.ValueError" in stacktrace + assert "_raised_from" in stacktrace + + +def test_messages_usage_includes_cached_tokens_in_input(): + """Ensures native Messages input tokens include cache reads and writes, as normalized usage does.""" + usage = MessagesUsage( + input_tokens=10, output_tokens=5, cache_read_input_tokens=3, cache_creation_input_tokens=2 + ) + + assert usage.attributes() == { + "gen_ai.usage.input_tokens": 15, + "gen_ai.usage.output_tokens": 5, + "gen_ai.usage.cache_read.input_tokens": 3, + "gen_ai.usage.cache_write.input_tokens": 2, + } + + +def test_messages_usage_keeps_thinking_tokens_across_stream_updates(): + """Ensures native Messages reports thinking tokens, and a later delta without details keeps them. + + `message_delta` usage updates the `message_start` snapshot, so a delta that omits + `output_tokens_details` must not erase a count reported earlier. + """ + usage = MessagesUsage() + usage.update( + SimpleNamespace( + input_tokens=10, + output_tokens=50, + output_tokens_details=SimpleNamespace(thinking_tokens=36), + ) + ) + usage.update(SimpleNamespace(output_tokens=60, output_tokens_details=None)) + + attributes = usage.attributes() + + assert attributes["gen_ai.usage.output_tokens"] == 60 + assert attributes["gen_ai.usage.reasoning.output_tokens"] == 36 + + +def test_messages_usage_reports_web_search_requests(): + """Ensures native Messages reports `server_tool_use.web_search_requests` as Molto's search count.""" + usage = MessagesUsage( + input_tokens=10, + output_tokens=5, + server_tool_use=SimpleNamespace(web_search_requests=2), + ) + + assert usage.attributes()["molto.usage.web_search_requests"] == 2 + + +@pytest.mark.parametrize("reasoning_tokens", [0, None]) +def test_zero_or_missing_reasoning_tokens_are_omitted(reasoning_tokens: int | None): + """Asserts that reasoning tokens appear only when non-zero, as the convention recommends.""" + completion = ChatCompletion( + id="chatcmpl-1", + object="chat.completion", + created=0, + model="gpt-5", + choices=[], + usage={ + "prompt_tokens": 1, + "completion_tokens": 2, + "total_tokens": 3, + "completion_tokens_details": {"reasoning_tokens": reasoning_tokens}, + }, + ) + + assert "gen_ai.usage.reasoning.output_tokens" not in chat_completions_response_attributes( + completion + ) + + +def test_incomplete_responses_response_reports_its_reason(): + """Ensures a native Responses span reports the incomplete reason, since Responses has no finish reason.""" + response = SimpleNamespace( + id="resp_1", + model="gpt-5", + status="incomplete", + incomplete_details=SimpleNamespace(reason="max_output_tokens"), + usage=None, + ) + + attributes = responses_response_attributes(response) + + assert attributes["gen_ai.response.finish_reasons"] == ["max_output_tokens"] + + +def _recorded_span(streaming: bool, record) -> Any: + provider = TracerProvider() + exporter = InMemorySpanExporter() + provider.add_span_processor(SimpleSpanProcessor(exporter)) + span = provider.get_tracer("test").start_span("chat") + record(SpanRecorder(span, streaming=streaming)) + span.end() + (finished,) = exporter.get_finished_spans() + return finished + + +def test_failed_responses_stream_event_reports_failed(): + """Ensures a native `response.failed` event records the failed response's end attributes. + + The SDK raises only on a separate `error` event, so a stream ending in + `response.failed` alone would otherwise look like a success on its span. + """ + event = SimpleNamespace( + type="response.failed", + response=SimpleNamespace( + id="resp_1", model="gpt-5", status="failed", incomplete_details=None, usage=None + ), + ) + + span = _recorded_span(True, lambda recorder: recorder.record_responses_event(event)) + + assert span.attributes["gen_ai.response.id"] == "resp_1" + assert span.attributes["gen_ai.response.finish_reasons"] == ("failed",) + + +def _chunk(index: int, *, content: str | None = None, finish_reason: str | None = None): + delta = SimpleNamespace(content=content, tool_calls=None) + choice = SimpleNamespace(index=index, delta=delta, finish_reason=finish_reason) + return SimpleNamespace(id="chatcmpl-1", model="gpt-5", choices=[choice], usage=None) + + +def test_chat_stream_reports_every_choice_finish_reason_without_usage(): + """Ensures Chat streams report finish reasons per choice and omit usage the caller did not request. + + Usage appears only with `stream_options.include_usage`, which native passthrough + never injects. + """ + + def record(recorder: SpanRecorder) -> None: + recorder.record_chat_completions_chunk(_chunk(1, finish_reason="length")) + recorder.record_chat_completions_chunk(_chunk(0, finish_reason="stop")) + + span = _recorded_span(True, record) + + assert span.attributes["gen_ai.response.finish_reasons"] == ("stop", "length") + assert "gen_ai.usage.input_tokens" not in span.attributes + + +def test_first_token_is_recorded_once_and_only_for_streaming_callers(): + """Ensures `molto.first_token` marks the first content delta, once, for streaming callers only.""" + + def record(recorder: SpanRecorder) -> None: + recorder.record_normalized(PartDeltaEvent(index=0, delta=TextDelta(content_delta="Hi"))) + recorder.record_normalized(PartDeltaEvent(index=0, delta=TextDelta(content_delta="!"))) + + assert [event.name for event in _recorded_span(True, record).events] == ["molto.first_token"] + assert _recorded_span(False, record).events == () + + +def test_signature_only_reasoning_delta_is_not_a_first_token(): + """Asserts that a delta replacing opaque reasoning state carries no content for time to first token.""" + + def record(recorder: SpanRecorder) -> None: + recorder.record_normalized( + PartDeltaEvent(index=0, delta=ThinkingDelta(signature_delta="opaque")) + ) + + assert _recorded_span(True, record).events == () + + +# Native passthrough does not validate upstream output, so these build SDK models +# without validation, as the SDKs do for stream chunks a provider sends. + + +def test_chat_chunk_choice_without_delta_is_recorded_without_raising(): + """Asserts that a Chat chunk whose choice has no delta still records its finish reason. + + Telemetry must never turn a successful native call into a failure. + """ + chunk = ChatCompletionChunk.model_construct( + id="chatcmpl-1", + model="gpt-5", + choices=[ChunkChoice.model_construct(index=0, finish_reason="stop")], + usage=None, + ) + + span = _recorded_span(True, lambda recorder: recorder.record_chat_completions_chunk(chunk)) + + assert span.attributes["gen_ai.response.finish_reasons"] == ("stop",) + assert span.events == () + + +def test_chat_completion_without_choice_indexes_or_choices_is_recorded_without_raising(): + """Asserts that Chat responses missing choice indexes, or all choices, never raise.""" + unindexed = ChatCompletion.model_construct( + id="chatcmpl-1", + model="gpt-5", + choices=[ChatChoice.model_construct(finish_reason="stop")], + usage=None, + ) + choiceless = ChatCompletion.model_construct( + id="chatcmpl-2", model="gpt-5", choices=None, usage=None + ) + + assert chat_completions_response_attributes(unindexed)["gen_ai.response.finish_reasons"] == [ + "stop" + ] + assert chat_completions_response_attributes(choiceless)["gen_ai.response.id"] == "chatcmpl-2" + + +def test_message_without_usage_is_recorded_without_raising(): + """Asserts that a native Messages response without usage reports no usage instead of raising.""" + message = Message.model_construct(id="msg_1", model="claude-sonnet-5", stop_reason="end_turn") + + attributes = messages_response_attributes(message) + + assert attributes["gen_ai.response.finish_reasons"] == ["end_turn"] + assert "gen_ai.usage.input_tokens" not in attributes + + +async def test_observe_closes_its_source_when_closed_early(): + """Ensures closing the recording wrapper closes the wrapped stream, releasing its response.""" + closed = [] + + async def source(): + try: + yield "a" + yield "b" + finally: + closed.append(True) + + recorded = [] + wrapped = observe(source(), recorded.append) + + assert await anext(wrapped) == "a" + await wrapped.aclose() + + assert closed == [True] + assert recorded == ["a"] + + +async def test_observe_yields_every_event_when_recording_fails(): + """Asserts that a recorder raising on unexpected output never interrupts the stream. + + Native Messages output is unvalidated, so a non-integer cache count makes usage + arithmetic raise; the caller must still receive every event. + """ + + async def source(): + yield "a" + yield "b" + + def record(event: str) -> None: + MessagesUsage(input_tokens=1, cache_read_input_tokens="2").attributes() + + assert [event async for event in observe(source(), record)] == ["a", "b"] + + +async def test_observe_propagates_source_errors_unchanged(): + """Asserts that an error from the wrapped stream reaches the caller as the same exception.""" + failure = RuntimeError("upstream failed") + + async def source(): + yield "a" + raise failure + + with pytest.raises(RuntimeError) as raised: + async for _ in observe(source(), lambda event: None): + pass + + assert raised.value is failure + + +@pytest.mark.parametrize( + ("event_type", "delta", "expected"), + [ + ("response.output_text.delta", "Hi", True), + ("response.reasoning_text.delta", "Thinking", True), + ("response.reasoning_summary_text.delta", "Summary", True), + ("response.function_call_arguments.delta", '{"city"', True), + ("response.custom_tool_call_input.delta", "input", True), + ("response.mcp_call_arguments.delta", '{"q"', True), + ("response.refusal.delta", "I can't", True), + ("response.output_text.delta", "", False), + ("response.audio.delta", "audio", False), + ], +) +def test_responses_content_deltas(event_type: str, delta: str, expected: bool): + """Ensures native Responses first-token detection counts text, reasoning, tool, and refusal deltas.""" + event = SimpleNamespace(type=event_type, delta=delta) + + assert is_responses_content_delta(event) is expected + + +@pytest.mark.parametrize( + ("delta", "expected"), + [ + ({"content": "Hi"}, True), + ({"refusal": "I can't"}, True), + ({"reasoning_content": "Thinking"}, True), + ( + {"tool_calls": [{"index": 0, "function": {"arguments": '{"city"'}}]}, + True, + ), + ({"tool_calls": [{"index": 0, "id": "call_1", "function": {"name": "f"}}]}, False), + ({"role": "assistant", "content": ""}, False), + ], +) +def test_chat_completions_content_deltas(delta: dict, expected: bool): + """Ensures native Chat first-token detection counts text, refusal, reasoning, and tool arguments. + + A tool-call delta carrying only its name or ID, or an empty role chunk, carries + no generated content. + """ + chunk = ChatCompletionChunk.model_validate( + { + "id": "chatcmpl-1", + "object": "chat.completion.chunk", + "created": 0, + "model": "gpt-5", + "choices": [{"index": 0, "delta": delta, "finish_reason": None}], + } + ) + + assert is_chat_completions_content_delta(chunk) is expected + + +@pytest.mark.parametrize( + ("delta", "expected"), + [ + ({"type": "text_delta", "text": "Hi"}, True), + ({"type": "thinking_delta", "thinking": "Thinking"}, True), + ({"type": "input_json_delta", "partial_json": '{"city"'}, True), + ({"type": "signature_delta", "signature": "opaque"}, False), + ({"type": "text_delta", "text": ""}, False), + ], +) +def test_messages_content_deltas(delta: dict, expected: bool): + """Ensures native Messages first-token detection counts text, thinking, and tool-input deltas only.""" + event = SimpleNamespace(type="content_block_delta", delta=SimpleNamespace(**delta)) + + assert is_messages_content_delta(event) is expected