Repository navigation
fix(openai): don't consume iterator arguments before calling OpenAI - #2075
Open
mosafariuk wants to merge 1 commit into
Open
mosafariuk wants to merge 1 commit into
mosafariuk wants to merge 1 commit into
Conversation
OpenAI accepts any iterable for messages. Langfuse read a generator while extracting the prompt and then passed the exhausted generator to OpenAI, so the request went out with messages=[] and the API returned a 400. One-shot iterators in the call kwargs are now turned into lists before they are read.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
With the
langfuse.openaidrop-in, passingmessagesas a generator breaks the call. The OpenAI SDK typesmessagesasIterable[...], and plainopenaiaccepts a generator. ButOpenAiArgsExtractorreadskwargs["messages"]to record the prompt (_extract_chat_prompt), which uses the generator up before the originalcreateruns. OpenAI then receivesmessages: []and rejects the request:The async client is affected the same way, because it goes through the same extractor.
This PR turns one-shot iterator arguments into lists in
OpenAiArgsExtractor.__init__, so tracing and the OpenAI call both see the same values. Lists, tuples, dicts and other re-iterable values are left as they are. Only the chat, completions, responses and embeddings methods are wrapped, and none of them take file arguments, so no file handle is ever read here.Fixes # (no existing issue)
Type of change
Verification
The new
test_chat_completion_does_not_consume_messages_iteratorpatches the client's_post, returns a realChatCompletion, and checks both the request body sent to OpenAI and the recorded span input. Onmainit fails withassert [] == [{...}].I also ran the async client by hand against an
httpx.MockTransport. Onmainit sendsmessages: [], and with this change it sends the real messages.Checklist
code_review.md..env.templateif needed.

Confidence Score: 5/5
The PR appears safe to merge.Summary
The PR converts iterator-valued keyword arguments to lists before tracing reads them. OpenAI then receives the same values instead of an exhausted iterator.
Reviews (1) · Last reviewed commit: "fix(openai): don't consume iterator argu..." · Reviewed by Greptile