Skip to content

fix(adk-bridge): preserve tool event Agent identities - #267

Merged
krisztianfekete merged 2 commits into
agentevals-dev:mainfrom
erauner12:fix/preserve-tool-agent-identities
Nov 3, 2026
Merged

krisztianfekete merged 2 commits into
agentevals-dev:mainfrom
erauner12:fix/preserve-tool-agent-identities

Conversation

@erauner12

@erauner12 erauner12 commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Closes #268.

Trace conversion currently flattens tool calls before the evaluation adapter assigns event authors. That can replace a captured Agent identity with agent, causing native judges to fail when app_details uses the captured name.

Preserve each tool call/result owner as native InvocationEvents. Missing metadata receives a names-only fallback without replacing captured declarations, and model-call accounting stays idempotent. The existing flat ADK eval-set/API format keeps its current exports and single-owner adaptation; this is an ADK data-model contract, separate from deprecated producer telemetry. Ambiguous flat multi-Agent evidence fails explicitly.

This repair targets upstream main independently. #264 is stacked on it and also targets upstream main because its dependency branch lives in my fork. Merge #267 first; then I will rebase #264 onto updated upstream main to leave only its tool-definition changes.

The replay test now uses the pure-semconv v2proj_span_only/S2.json fixture. A fake-judge regression completes native final-response evaluation both with captured Agent metadata and with names-only enrichment, asserting result.error is None and a complete score. Synthetic tests also cover multiple Agents, correlated pairs, both ADK data variants, unchanged exports and accounting.

Local validation: 2,014 model-free unit tests and 29 HTTP integration tests passed; Ruff lint, format and diff checks passed. No live model calls.

The in-memory intermediate_data variant changes; callers should use ADK’s variant-neutral readers.

AI assistance: non-trivial code and test assistance.

@erauner12
erauner12 marked this pull request as ready for review November 1, 2026 23:38
@krisztianfekete
krisztianfekete merged commit 890dc65 into agentevals-dev:main Nov 3, 2026
9 checks passed
@erauner12
erauner12 deleted the fix/preserve-tool-agent-identities branch November 3, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native evaluation can lose the Agent identity of captured tool activity

2 participants