Skip to content

fix(openai): don't consume iterator arguments before calling OpenAI - #2075

Open
mosafariuk wants to merge 1 commit into
langfuse:mainfrom
mosafariuk:fix/openai-iterator-messages
Open

mosafariuk wants to merge 1 commit into
langfuse:mainfrom
mosafariuk:fix/openai-iterator-messages

Conversation

@mosafariuk

@mosafariuk mosafariuk commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do?

With the langfuse.openai drop-in, passing messages as a generator breaks the call. The OpenAI SDK types messages as Iterable[...], and plain openai accepts a generator. But OpenAiArgsExtractor reads kwargs["messages"] to record the prompt (_extract_chat_prompt), which uses the generator up before the original create runs. OpenAI then receives messages: [] and rejects the request:

plain openai:      messages sent to OpenAI: [{'role': 'user', 'content': 'hello'}]  -> OK
langfuse.openai:   messages sent to OpenAI: []  -> BadRequestError 400: "[] is too short - 'messages'"

The async client is affected the same way, because it goes through the same extractor.

This PR turns one-shot iterator arguments into lists in OpenAiArgsExtractor.__init__, so tracing and the OpenAI call both see the same values. Lists, tuples, dicts and other re-iterable values are left as they are. Only the chat, completions, responses and embeddings methods are wrapped, and none of them take file arguments, so no file handle is ever read here.

Fixes # (no existing issue)

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Documentation update
  • Tooling, CI, or repo maintenance

Verification

uv run pytest tests/unit/test_openai.py   # 39 passed; the new test fails on main
uv run pytest tests/unit                  # 1006 passed; the 2 failures (test_prompt_atexit) and 18 errors (test_prompt) are identical on unmodified main (1005 passed)
uv run ruff check . && uv run ruff format --check .
uv run mypy langfuse

The new test_chat_completion_does_not_consume_messages_iterator patches the client's _post, returns a real ChatCompletion, and checks both the request body sent to OpenAI and the recorded span input. On main it fails with assert [] == [{...}].

I also ran the async client by hand against an httpx.MockTransport. On main it sends messages: [], and with this change it sends the real messages.

Checklist

  • I self-reviewed the diff using code_review.md.
  • I added or updated tests for behavior changes.
  • I updated docs, examples, or .env.template if needed.
  • I did not hand-edit generated files; if generated files changed, I used the upstream regeneration path.
  • I did not commit secrets or credentials.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

Summary

The PR converts iterator-valued keyword arguments to lists before tracing reads them. OpenAI then receives the same values instead of an exhausted iterator.

  • OpenAI calls keep iterator arguments for both tracing and requests.

Reviews (1) · Last reviewed commit: "fix(openai): don't consume iterator argu..." · Reviewed by Greptile

OpenAI accepts any iterable for messages. Langfuse read a generator while extracting the prompt and then passed the exhausted generator to OpenAI, so the request went out with messages=[] and the API returned a 400. One-shot iterators in the call kwargs are now turned into lists before they are read.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant