Skip to content

feat: Add OpenTelemetry tracing for coordinator calls - #106

Draft
koladefaj wants to merge 13 commits into
mainfrom
feat/otel-tracing
Draft

koladefaj wants to merge 13 commits into
mainfrom
feat/otel-tracing

Conversation

@koladefaj

@koladefaj koladefaj commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Every Molto.responses(), Molto.chat_completions(), and Molto.messages() call now emits one OpenTelemetry CLIENT span named chat {model}. It records the route Molto chose (native or translated), the downstream protocol and upstream endpoint, the provider and destination, the response identity, finish reasons, token usage, errors, and a molto.first_token event for streaming callers.
  • Molto depends only on opentelemetry-api and never configures a tracer provider. Without an SDK the API returns no-op spans and calls behave exactly as before; applications own the SDK, exporters, and OTEL_* configuration.
  • The coordinator owns the span. Adapters, normalizers, and upstream clients stay telemetry-free, and telemetry never changes routing, validation, request construction, projection, or errors, and never reads extra_body or extra_headers.
Molto.responses(body, endpoint, ...)
  _resolve_provider()                        # unchanged
  _traced(protocol, endpoint, provider, base_url, body, call)
    start_attributes(route, body)            # utils/telemetry.py, pure
    start_span(kind=CLIENT)                  # parent: caller's active span
    use_span(span)                           # current only while the upstream request is sent
      _run_responses(..., recorder)          # the previous coordinator body
        observe(upstream events, recorder)   # reads output for end attributes and first token
    non-streaming -> end span
    streaming     -> MoltoStream ends it on exhaustion, error, aclose(), or garbage collection
  • Added docs/observability.md, the span contract: name, kind, status, parent and current-span rules, start and end attributes, error attributes, events, the content policy, and how applications enable export and httpx child spans.
  • Added opentelemetry-api>=1.36.0,<1.46 to core dependencies, and pinned opentelemetry-api==1.45.2, opentelemetry-sdk==1.45.2, and opentelemetry-instrumentation-httpx==0.66b2 in test. The floor is the lowest version on which the full suite passes unchanged, and the <1.46 ceiling stops just above 1.45.2, the newest release, on which it also passes. Each opentelemetry-sdk release requires the matching API version, so the ceiling must be re-derived with the suite when OpenTelemetry releases 1.46.
  • Added python/molto/utils/telemetry.py, which maps the resolved route, request, and upstream output to GenAI semantic-convention attributes. Keys are plain strings that follow the current spec, including gen_ai.usage.cache_write.input_tokens, rather than imports from the semconv package, which still uses older names.
  • Finish reasons keep each route's vocabulary: native spans report the provider's value (end_turn, stop, Responses completed, failed, or its incomplete reason), and translated spans report Molto's normalized termination reason. Native Messages input tokens include cached tokens, so native and translated spans report the same usage for the same call. Cache attributes appear only when non-zero.
  • error.type reports the normalized APIException class, so native and translated routes agree; http.response.status_code comes only from a real upstream error response of 400 or higher; molto.error.phase is setup before the coordinator returns, or stream after it returned a stream, including a failing aclose(). Exceptions are recorded without messages, which can quote request content: the event holds the class name and a frames-only stack trace across the __cause__ chain.
  • Every streaming span records how it ended in molto.stream.outcome: completed, closed (aclose() before the end), cancelled (the caller's task cancelled, including before the first event), error, or abandoned. Cancellation ends the span with UNSET status. A stream dropped without being consumed or closed ends its span when it is garbage-collected, as abandoned and at the time of its last event; the event loop's async-generator finalizer then closes the dropped source, which releases its upstream response.
  • Native output is not validated, so telemetry reads every field a provider may omit with a default. record_safely wraps every recording call, so output no extractor expects loses its attributes, with only the exception class logged at DEBUG through Molto's loguru logger, and never turns a successful call into a failure.
  • A native Responses stream that ends in response.failed without a separate error event returns normally; its span records the failed response with finish reason failed, as the same non-streaming response does.
  • Native passthrough now streams for any truthy stream, the rule the OpenAI and Anthropic SDKs use (stream=stream or False). It previously streamed only for True, so stream: 1 on a provider that accepted it returned an SDK stream to the non-streaming path and failed with TypeError: Responses request did not return a non-streaming response. A provider that rejects the value still raises its own error. One helper, _streams in core.py, holds each route's rule and decides both native routing and the span's gen_ai.request.stream, so they cannot drift: truthy on native routes, and on translated routes, which validate stream as Literal[True, False], true, 1, or 1.0. Telemetry never reads stream itself.
  • Ending a span skips error attributes and stack-trace formatting when the span is not recording, and native Chat streams set the response identity and finish reasons only when they change rather than on every chunk.
  • Added BaseClient.resolve_base_url(), so the clients and span attributes resolve the default destination in one place, and shared replay helpers in molto.utils.test.

Verification

  • python -m pytest on Python 3.12.13, 3.13.13, and 3.14.3: 2772 passed on each.
  • uv pip install -e . followed by python -c "import molto" (CI "Check import with core dependencies only"): passed on Python 3.12, 3.13, and 3.14.
  • opentelemetry-api floor: uv pip install -e ".[test]" --overrides with each candidate's matching opentelemetry-sdk and opentelemetry-instrumentation-httpx, then python -m pytest. 1.36.0 (with instrumentor 0.57b0) passed all 2743 tests on Python 3.12.13, 3.13.13, and 3.14.3, and all 2769 on Python 3.12.13 after the review fixes. 1.28.0 (with 0.49b0), pydantic-ai-slim's floor, passed 2739 on 3.12 and 3.14; the 4 failures are the test-only instrumentor's instrument_client() rejecting meter_provider.
  • ruff check --config pyproject.toml python tests and ruff format --diff --config pyproject.toml python tests: clean. A repository-wide run also flags test_molto.py, an untracked local file outside this change.
  • npx prettier@3.8 --check on the changed Markdown: passed.
  • tests/core/test_tracing.py replays committed cassettes through all three coordinators on native and translated routes, streaming and non-streaming. It covers span name, kind, parenting, start and end attributes, usage, status, the first-token event, every lifecycle edge (exhaustion, early and repeated aclose(), mid-stream failure, failing close, prefetch failure, cancellation of a real call and stream, and dropped streams collected with gc.collect()), each molto.stream.outcome value, error attributes on native and translated routes, and that spans hold only contract keys with no request content, extensions, headers, or credentials.
  • A fresh interpreter with no SDK produces identical stream events, responses, and errors, and never imports opentelemetry.sdk; another with a global SDK provider records the span under scope molto and the package version.
  • opentelemetry-instrumentation-httpx spans nest under the Molto span with global instrument(), on Molto's default client and an injected one created before instrumentation, and with instrument_client(); for streams, the HTTP span ends before the first token.
  • Each fix from the PR review fails its regression tests when reverted: dropping response.failed from the terminal events (1 test), the translated stream rule (2, before the rule moved into _streams), the observe guard (2), and the non-streaming response guard (1), and restoring the native is True stream rule (7).
  • Dropping an unconsumed, unclosed native or translated stream closes its upstream response body within five event-loop turns after gc.collect(), asserted by test_dropped_stream_releases_its_upstream_response.
  • Mutation checks: removing the end-on-exhaustion branch fails 6 tests, removing the end-once guard fails 7, removing the use_span call fails all 12 HTTP-span tests, reporting cancellation as error fails both cancellation tests, ending abandoned spans at collection time fails both end-time tests, and removing the __del__ guard fails its test.
  • ruff check --config pyproject.toml . and ruff format --diff --config pyproject.toml . (the repository-wide lint.yml commands): clean on every tracked file; the only findings are in test_molto.py, an untracked local file, which CI does not check out.
  • npx prettier@3.8 --check .: passed.
  • Design principles 1-5 checked against git diff origin/main...HEAD: no SDK import or set_tracer_provider under python/; start_span, use_span, and end() only in core.py; no OpenTelemetry or telemetry import in adapters/, upstream/, or types/; telemetry.py never references extra_body or extra_headers; no semantic-conventions import; no request or completion content read into an attribute.
  • Cassettes were replayed only; none were recorded or modified.

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Coverage

Coverage Report (Python 3.14) •
FileStmtsMissCoverMissing
core.py238498%185, 495, 602, 721
upstream
   anthropic.py5082795%98, 104, 117–118, 241, 345, 362, 393, 485, 565, 585, 631–632, 735–736, 739, 745, 756, 765, 810, 827, 840, 844, 896–897, 932, 934
   base.py61493%124, 129, 143, 148
   openai.py7643695%156, 158, 185, 199, 211, 223, 287–288, 297, 435, 462, 620, 628, 630, 633, 667–668, 740, 752, 768, 808, 832, 915, 993, 1012, 1171, 1203, 1245–1246, 1383, 1425, 1429, 1442–1443, 1460, 1463
utils
   telemetry.py2990100% 
   test.py1910100% 
TOTAL398719395% 

Tests Skipped Failures Errors Time
2794 0 💤 0 ❌ 0 🔥 36.607s ⏱️

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Coverage

Coverage Report (Python 3.13) •
FileStmtsMissCoverMissing
core.py238498%185, 495, 602, 721
upstream
   anthropic.py5082795%98, 104, 117–118, 241, 345, 362, 393, 485, 565, 585, 631–632, 735–736, 739, 745, 756, 765, 810, 827, 840, 844, 896–897, 932, 934
   base.py63494%124, 129, 143, 148
   openai.py7643695%156, 158, 185, 199, 211, 223, 287–288, 297, 435, 462, 620, 628, 630, 633, 667–668, 740, 752, 768, 808, 832, 915, 993, 1012, 1171, 1203, 1245–1246, 1383, 1425, 1429, 1442–1443, 1460, 1463
utils
   telemetry.py3030100% 
   test.py1910100% 
TOTAL403719395% 

Tests Skipped Failures Errors Time
2794 0 💤 0 ❌ 0 🔥 1m 14s ⏱️

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Coverage

Coverage Report (Python 3.12) •
FileStmtsMissCoverMissing
core.py238498%185, 495, 602, 721
upstream
   anthropic.py5082795%98, 104, 117–118, 241, 345, 362, 393, 485, 565, 585, 631–632, 735–736, 739, 745, 756, 765, 810, 827, 840, 844, 896–897, 932, 934
   base.py63494%124, 129, 143, 148
   openai.py7643695%156, 158, 185, 199, 211, 223, 287–288, 297, 435, 462, 620, 628, 630, 633, 667–668, 740, 752, 768, 808, 832, 915, 993, 1012, 1171, 1203, 1245–1246, 1383, 1425, 1429, 1442–1443, 1460, 1463
utils
   telemetry.py3030100% 
   test.py1910100% 
TOTAL403719395% 

Tests Skipped Failures Errors Time
2794 0 💤 0 ❌ 0 🔥 1m 18s ⏱️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant