Repository navigation
feat(telemetry): trace each MCP request and tool call under the caller's trace - #103
Open
braghettos wants to merge 1 commit into
Open
braghettos wants to merge 1 commit into
braghettos wants to merge 1 commit into
Conversation
…r's trace Wrap the MCP HTTP handler in otelhttp, so every request gets a server span that is a child of the incoming W3C traceparent. The standalone SSE stream (GET) is filtered out: it stays open for the whole session. The tools/call span is now named "tools/call <tool>" and carries mcp.method.name and gen_ai.tool.name next to the existing mcp.tool.* attributes. It is parented on the server span of the request that made the call. The SDK runs handlers on the session's context, which holds the trace context of the initialize request, so the middleware writes the server span's context into a cloned request's traceparent header and the tool middleware extracts it from CallToolRequest.Extra.Header. No header, argument or result values are added to spans. Part of kagent-dev#102 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016RAxvkp7wRXwXDx4Lx8STi Signed-off-by: Diego Braga <diego.braga86@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #102
What
With tracing enabled, a tool call made over MCP HTTP now shows up in the caller's trace:
telemetry.HTTPMiddlewarenow wraps the handler inotelhttp.NewHandler(go.opentelemetry.io/contrib/instrumentation/net/http/otelhttpv0.72.0, the release that matches the otel v1.47.0 SDK already ingo.mod). The span is a child of the incoming W3Ctraceparent. The standalone SSE stream (GET) is filtered out, because it stays open for the whole session and its span would last for hours and parent nothing. POST responses that stream SSE work as before: otelhttp wraps the writer with httpsnoop, soFlushis kept.ToolMiddleware(the single receiving middleware every tool already goes through) names its spantools/call <tool name>(wasmcp.tool.<tool name>) and adds the MCP semconv attributesmcp.method.nameandgen_ai.tool.name. The existingmcp.tool.*attributes are still there. Error status on anIsErrorresult or a Go error is unchanged.Why the tool span needed more than a server span
The SDK does not run handlers on the HTTP request's context. A stateful session's handlers run on the context of the request that opened the session (
initialize). So even before this change,mcp.tool.*spans joined the trace of the session'sinitializerequest, not the trace of thetools/callthat triggered them. What does reach the handler for each request is that request's header (CallToolRequest.Extra.Header). So:HTTPMiddlewareclones the request and writes the server span's context into the clone'straceparentheader. The caller's request is not modified.ToolMiddlewaredrops whatever span is in the session context and extracts the parent fromExtra.Header. With no header (stdio, in-memory) it keeps the old behaviour.The middleware also no longer re-extracts trace context on top of an active span. That would have replaced the otelhttp server span with its remote parent.
What is not recorded
No request header, tool argument or tool result is added as a span attribute. Headers can carry credentials, and #66 removed header attributes for that reason. otelhttp's defaults record method, path, status, sizes, peer address and
user_agent.original, and no other header.Tests
cmd/tracing_test.gobuilds the HTTP handler the wayrun()does (new helpernewMCPHTTPHandler) and records spans withtracetest.InMemoryExporter. It connects a real go-sdk client whose transport sets a differenttraceparentoninitializeand on eachtools/call, plus anAuthorizationheader. It calls a succeeding tool, a tool that returns an error result, and a tool that returns a Go error. It asserts:tools/callrequest, with the incoming trace ID and a remote parent with the incoming span ID;tools/call <name>span, a child of that server span and in thetools/callrequest's trace (not theinitializetrace);Okfor the succeeding tool andErrorfor both failing tools;mcp.method.nameandgen_ai.tool.nameare set;header,authorizationortoken, and no attribute value contains the bearer credential or the tool argument.When
toolSpanParentis disabled, the test fails on the trace-ID and parent assertions, which confirms the session-context problem described above.make lint(golangci-lint v2.15.0): 0 issues.go vet ./...is clean.go test -tags=test ./pkg/... ./internal/... ./cmd/...passes.Follow-up (not in this PR)
This PR covers only the first half of #102. The second half is the k8s tools: they run
kubectlas a subprocess, which cannot carry trace context. That is left for a follow-up, which would use client-go in-process for the read verbs, with anotelhttptransport, sotraceparentreaches the API server.Note
Dashboards or queries that match the old span name
mcp.tool.<tool>need to switch totools/call <tool>or filter on themcp.tool.name/gen_ai.tool.nameattribute.🤖 Generated with Claude Code
https://claude.ai/code/session_016RAxvkp7wRXwXDx4Lx8STi