Repository navigation
fix: keep file_search reasoning before its call on continuations - #542
justrightnow wants to merge 1 commit into
Conversation
When a gateway-run file search produced reasoning before the function call, continuing with previous_response_id persisted the model-facing function call/output pair ahead of the turn's output, so the next turn showed the model call, output, reasoning instead of reasoning, call, output. Hosted tool/web/MCP searches were fixed for this in vllm-project#505 with regression coverage in vllm-project#519; file search was the remaining gap tracked in vllm-project#482 because its public file_search_call omits results by default. Record each file search's model-facing call/output pair in public-output order and splice it at the public file_search_call's position when assembling turn history, so a continuation sees the same prefix the live round did, without changing the public response shape. Fixes vllm-project#482. Signed-off-by: Jack Wei <weike@chinamobile.com>
Verification (before vs after)Local, deterministic mocks only — mock Turn 1 upstream output: Round 2 model input — live (already correct, both before & after): Continuation turn model input — before the fix: (call + output jumped ahead of the reasoning — #482) Continuation turn model input — after the fix: The continuation prefix is now byte-identical to the live round (same function cargo test -p agentic-server-core --test file_search_history_order
# before: FAILED ("continuation shares the live round's prefix")
# after: ok. 1 passed
cargo test -p agentic-server-core # all pass
cargo clippy --all-targets -- -D warnings && cargo fmt -- --check # clean |
|
@justrightnow Thanks however we cant accept this. #536 is already working on acceptable solution with proper cassettes recording for file history and continuation. |
|
OK |
Summary
Continuations reorder a turn's reasoning around gateway-executed calls (#482). Hosted tool/web/MCP searches were fixed in #505 with regression coverage in #519.
file_search— added later in #486 — still had the remaining gap, as flagged in the issue's follow-up.When a gateway-run file search emits reasoning before the function call, the live second inference round already sees the model-facing items in the right order:
user, reasoning, function_call(file_search), function_call_outputBut continuing with
previous_response_idpersisted that model-facing call/output pair at the front of the turn's history (new_input_items), ahead of the turn's public output (which holds the reasoning). The next turn therefore saw:user, function_call, function_call_output, reasoning, message, userThis happens because the public
file_search_callomits retrieved results by default (include_resultsdefaults to false), andOutputItem::to_input_itemreturnsNonefor it, so the model-facing pair (whose output carriesretrieved_passages) had to be carried separately — it was just carried in the wrong place.This change:
RequestContext::file_search_history, recording each file search's model-facing call/output pair in public-output order.file_search_callitem's position when assembling turn history (turn_items), instead of prepending it. Pairs are still consumed in order under in-turn compaction so multiple file searches stay aligned.enriched_request.input) path and the public response shape untouched: the public response still exposesreasoning, file_search_call, message, withfile_search_call.resultsomitted by default.After the fix a continuation shows the model exactly the same prefix the live round did.
Test plan
Pure local mocks (mock
/v1/responses+ deterministic/v1/embeddings, in-memory shared-cache SQLite, temp file store) — no cassette and no OpenAI key required.agentic-server-core/tests/file_search_history_order.rs::file_search_continuation_keeps_reasoning_before_the_call. It:[reasoning, file_search_call] -> messagestored turn, then aprevious_response_idturn;user, reasoning, function_call(file_search), function_call_outputwith matchingcall_id;reasoning, file_search_call, messageand that the publicfile_search_callomitsresults.cargo test -p agentic-server-core— all targets pass (includingfile_search_service,hosted_tool_search_*,web_search_tool_test,mcp_tool_test,stateful_conversation_integration,stateful_responses_integration,session_continuation_test,storage_*,compaction, and the replay/cassette suites).cargo test -p agentic-server— passes, includingfile_search_http_test,conversations_test,responses_test, and compaction.cargo clippy --all-targets -- -D warningsandcargo fmt -- --check— clean.messages_test"unreachable upstream" cases expect502/connection-refused but the local transparent proxy (ALL_PROXY=socks5h://127.0.0.1:1024) turns the bound-but-unlistened loopback port into a504timeout; and oneresponses_websocket_teststeering timing case passes 3/3 in isolation.Thanks @Than0316 for the file-search follow-up and the failing-test analysis in #482, and to the #505/#519 authors whose fix this mirrors.