Repository navigation
FIX: Show nonboolean automated scores by value, not as Failure - #3922
Merged
Roman Lutz (romanlutz) merged 2 commits intoNov 5, 2026
Merged
Roman Lutz (romanlutz) merged 2 commits into
Roman Lutz (romanlutz) merged 2 commits into
Conversation
Attack Result Details compared score_value against the literal "true" without checking score_type, so an unknown-type or float_scale score with a value was labeled Failure. Only true_false scores carry a verdict now; every other type shows the value the scorer recorded, as the message score details already do. Missing and undetermined scores stay distinct and no numeric value is interpreted. Closes microsoft#3916 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the change to display raw nonboolean scores in Attack Result Details.
No open issues 🎉
What changed in this PR
Fixes #3916 by showing raw nonboolean scores in Attack Result Details instead of incorrectly labeling them as Success or Failure, aligning this view with message score details.
Changes:
- Restricts verdicts to
true_falsescores while preserving missing and valueless score labels. - Adds regression coverage for numeric and unknown scores, accessible labels, and human-score selection.
| File | Description |
|---|---|
frontend/src/components/Chat/ObjectiveHeader.tsx |
Checks score type before assigning a verdict and displays raw nonboolean values. |
frontend/src/components/Chat/ObjectiveHeader.test.tsx |
Adds parameterized tests for nonboolean score display and radio selection. |
Review effort: Balanced
Roman Lutz (romanlutz)
approved these changes
Nov 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
scoreVerdict()inObjectiveHeader.tsxcomparedscore_valueagainst the literal"true"without looking atscore_type, so every nonboolean score with a value becameFailurein the Attack Result Details popover:Score(score_type="unknown", score_value="unknown")readFailure, and so did afloat_scalescore of0.9. The message score details already show the raw value for the same score, so the two views disagreed.Only
true_falsecarries a verdict (Score.get_value()returns a bool for it and a float forfloat_scale, and_check_score_value()restrictstrue_falsevalues totrue/false). The display helper now follows that: atrue_falsescore still readsSuccess/Failure, every other type shows the value the scorer recorded, a score without a value staysUndetermined, and no score at all staysNot set. No numeric value is interpreted, so there is no threshold in the presentation layer. The human-score radio group reads the same helper, so a nonboolean automated score no longer preselectsFailurefor the reviewer either.Closes #3916
Tests and Documentation
Added a parameterized case to
ObjectiveHeader.test.tsxcoveringfloat_scale0and0.9,unknown"unknown", andunknown"true"— the literal that used to produce a verdict from the wrong type. Each case asserts the raw value is shown, that the automated-score row contains noSuccess/Failure/Undeterminedtext, that the button's accessible name matches, and that the human verdict is not preselected toFailure. The existingdistinguishes automated scorecases keep coveringtrue_falseTrue/False, the valuelessUndeterminedscore and the missingNot setscore.Verified with
npx jest src/components/Chat/ObjectiveHeader.test.tsxinfrontend/. The four new cases fail onmain(they readFailure) and pass with this change; the file's other tests pass unchanged. No documentation change: this is a display fix with no API surface.🤖 Generated with Claude Code