Repository navigation
fix(llm): map openai_compat structured output messages like generate - #258
Merged
Merged
Conversation
Structured output sent the prompt's messages unmapped, so image parts kept
the universal {type=image, source} shape and developer messages kept their
role. OpenAI-compatible servers reject both (vLLM: "Unsupported chat content
part type: 'image'"). The handler now maps messages with the driver mapper,
as generate does.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cause
wippy.llm.openai_compat:structured_outputput the prompt's messages on the wire unmapped.generatemaps them throughopenai_mapper.map_messages. Because structured output skipped that step, it sent:{type = "image", source = {...}}shape instead of{type = "image_url", image_url = {url}};developermessages with their role unchanged, instead of assystemmessages.OpenAI-compatible servers reject the image shape. vLLM returns:
Fix
The handler now maps messages with the driver mapper, the same way
generatedoes.map_messagesalready drops blank text, so thewithout_blank_textcall and thepromptimport are removed.Tests (red before the fix, green after)
In
test/openai_compat/structured_output_test.lua, underMessage mapping:sends image parts as image_url content parts: before the fix,expected "image_url", got "image"sends developer messages as system messages: before the fix,expected "system", got "developer"Full llm suite: 2907 passed, 9 skipped.
wippy lint --level error: no issues.