Repository navigation
FIX Export the converted data type in JSON conversations and label HTML report pieces by it - #3782
Open
Shivam Sharma (ShivamSharma43) wants to merge 3 commits into
Conversation
…ieces by it JsonConversationPrinter exported only the original value's data type, so the HTML scenario report labelled a converted value by its original type: a text-to-image conversion rendered the image path as plain text, and an image-to-text conversion rendered the text as an [image_path] placeholder. Export original_value_data_type and converted_value_data_type next to the existing data_type (kept as the original type), and have the HTML template switch on the type of the value it displays. Fixes microsoft#3742 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
The only red check is |
…rt-converted-type
…rt-converted-type Resolve the conflict with microsoft#3771 (Original/Converted labels in the HTML report): keep its side-by-side rendering and label each value with its own data type (original_value_data_type / converted_value_data_type). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #3742.
JsonConversationPrinter._build_piece_asyncexported each piece with a singledata_type, set tooriginal_value_data_type, next to bothoriginal_valueandconverted_value. The converted value's type was lost. The HTML scenario report switched on that originaldata_typefor both values, so a converted value was labelled with the wrong type.Since #3771 merged, the report shows "Original:" and "Converted:" side by side when the two values differ. With this change each of them gets its own type:
maintext"draw a cat" →image_pathmedia/cat.png<div class="piece">media/cat.png</div><div class="piece media">[image_path] media/cat.png</div>image_pathmedia/dog.png→text"a dog on a log"<div class="piece media">[image_path] a dog on a log</div><div class="piece">a dog on a log</div>The original value keeps its own type label (
draw a catstays plain text,[image_path] media/dog.pngstays media).pyrit/output/conversation/json.py: each non-reasoning piece now also carriesoriginal_value_data_typeandconverted_value_data_type.data_typestays as it was (the original type), so existing consumers of the JSON keep working, and reasoning pieces are unchanged.pyrit/output/scenario_result/html.py: the template labels the original value withoriginal_value_data_typeand the converted value withconverted_value_data_type. When only one value is shown (no converted value, or it matches the original), it uses the type of the value it shows. Both fall back todata_type, so older JSON documents without the new fields render as before.Tests and Documentation
New tests:
tests/unit/output/conversation/test_json.py::test_build_async_exports_original_and_converted_data_types: a text → image_path piece exports both types, anddata_typestaystext.tests/unit/output/scenario_result/test_html.py::test_html_report_labels_converted_media_by_converted_type: a text → image_path piece renders the original as plain text and the converted value as[image_path] media/cat.png.tests/unit/output/scenario_result/test_html.py::test_html_report_renders_converted_text_of_media_original_as_text: an image_path → text piece renders the original as[image_path] media/cat.pngand the converted value as plain text.tests/unit/output/scenario_result/test_html.py::test_html_report_labels_unchanged_value_by_converted_type: when the value is unchanged but the converted type differs, the single shown value uses the converted type.tests/unit/output/scenario_result/test_html.py::test_html_report_labels_converted_types_from_real_builders: both directions built from realMessagePieces byJsonConversationPrinterand rendered to HTML.tests/unit/outputpasses (410 tests, including the #3771 tests), and so do the JSON export consumers intests/unit/cli/test_output.py,test_scenario_report_parity.py,test_results.pyandtests/unit/analytics/test_scenario_statistics_parity.py.ruff check,ruff format --checkandty checkare clean on the changed files.No documentation changes: the docs don't list the per-piece JSON fields. I didn't run JupyText because no notebooks are affected.
This change was prepared with the help of Claude Code (an AI coding assistant). I reviewed the change and the tests.
🤖 Generated with Claude Code