Repository navigation
feat: pass captured tool declarations to ADK judges - #264
krisztianfekete merged 3 commits into
Conversation
b22d890 to
9c93c10
Compare
krisztianfekete
left a comment
There was a problem hiding this comment.
Thanks, couple of small things:
rubric_based_final_response_quality_v1now errors on every ADK trace with a tool call (see inline). CI misses it because no test runs that metric on a converted trace.- We're dropping legacy formats in favour of OTel GenAI semconv, which is also where ADK telemetry v2 is going. Please remove the parts built on OpenLLMetry
gen_ai.prompt.N, ADK's deprecatedcall_llmspan and the Gemini Schema format. Please also use the semconv fixtures for tests.
9c93c10 to
c982ece
Compare
|
Rebased this PR onto #267. Removed the legacy compatibility branch and dedicated tests, converted only @krisztianfekete Would you accept a minimal synthetic kagent 0.10 compatibility fixture, or should semconv be the only supported path? |
c982ece to
50c25af
Compare
Optimally no, semconv is the only supported path. Older producers should upgrade or normalize to OTel GenAI before export. Would that work with you? |
krisztianfekete
left a comment
There was a problem hiding this comment.
Let's rebase it to main, then it LGTM!
50c25af to
cfbf71f
Compare
Closes #263.
Summary
ADK judges need to know which tools an agent had available, not just which ones it called.
This PR passes captured tool definitions into
Invocation.app_details, including descriptions and parameter schemas, so judges have the context needed to evaluate tool use.It also:
Testing
Added regression coverage using OTel GenAI semconv fixtures, including
rubric_based_final_response_quality_v1with tool calls.Validation after rebase: 2,059 model-free tests passed on Python 3.13.15, including 29 integration tests; 14 skipped. E2E tests excluded. Ruff, lockfile, and diff checks passed. No live model calls.
Base
Rebased directly onto upstream
mainat890dc65, which includes merged #267. The identity repair is inherited from upstream; this PR now adds only captured tool declarations and their tests. Producers use canonical GenAI semconv; no legacy producer fixture or private extraction override is added.AI assistance: non-trivial code and test assistance.