Skip to content

FIX: Show nonboolean automated scores by value, not as Failure - #3922

Merged
Roman Lutz (romanlutz) merged 2 commits into
microsoft:mainfrom
qwist1233-cpu:fix/3916-nonboolean-automated-score
Nov 5, 2026
Merged

Roman Lutz (romanlutz) merged 2 commits into
microsoft:mainfrom
qwist1233-cpu:fix/3916-nonboolean-automated-score

Conversation

@qwist1233-cpu

@qwist1233-cpu qwist1233-cpu commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Description

scoreVerdict() in ObjectiveHeader.tsx compared score_value against the literal "true" without looking at score_type, so every nonboolean score with a value became Failure in the Attack Result Details popover: Score(score_type="unknown", score_value="unknown") read Failure, and so did a float_scale score of 0.9. The message score details already show the raw value for the same score, so the two views disagreed.

Only true_false carries a verdict (Score.get_value() returns a bool for it and a float for float_scale, and _check_score_value() restricts true_false values to true/false). The display helper now follows that: a true_false score still reads Success/Failure, every other type shows the value the scorer recorded, a score without a value stays Undetermined, and no score at all stays Not set. No numeric value is interpreted, so there is no threshold in the presentation layer. The human-score radio group reads the same helper, so a nonboolean automated score no longer preselects Failure for the reviewer either.

Closes #3916

Tests and Documentation

Added a parameterized case to ObjectiveHeader.test.tsx covering float_scale 0 and 0.9, unknown "unknown", and unknown "true" — the literal that used to produce a verdict from the wrong type. Each case asserts the raw value is shown, that the automated-score row contains no Success/Failure/Undetermined text, that the button's accessible name matches, and that the human verdict is not preselected to Failure. The existing distinguishes automated score cases keep covering true_false True/False, the valueless Undetermined score and the missing Not set score.

Verified with npx jest src/components/Chat/ObjectiveHeader.test.tsx in frontend/. The four new cases fail on main (they read Failure) and pass with this change; the file's other tests pass unchanged. No documentation change: this is a display fix with no API surface.

🤖 Generated with Claude Code

Attack Result Details compared score_value against the literal "true"
without checking score_type, so an unknown-type or float_scale score with
a value was labeled Failure. Only true_false scores carry a verdict now;
every other type shows the value the scorer recorded, as the message score
details already do. Missing and undetermined scores stay distinct and no
numeric value is interpreted.

Closes microsoft#3916

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 4, 2026 08:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the change to display raw nonboolean scores in Attack Result Details.

No open issues 🎉

What changed in this PR

Fixes #3916 by showing raw nonboolean scores in Attack Result Details instead of incorrectly labeling them as Success or Failure, aligning this view with message score details.

Changes:

  • Restricts verdicts to true_false scores while preserving missing and valueless score labels.
  • Adds regression coverage for numeric and unknown scores, accessible labels, and human-score selection.
File Description
frontend/​src/​components/​Chat/​ObjectiveHeader.tsx Checks score type before assigning a verdict and displays raw nonboolean values.
frontend/​src/​components/​Chat/​ObjectiveHeader.test.tsx Adds parameterized tests for nonboolean score display and radio selection.

Review effort: Balanced

@romanlutz Roman Lutz (romanlutz) self-assigned this Nov 5, 2026
@romanlutz
Roman Lutz (romanlutz) merged commit 42290d0 into microsoft:main Nov 5, 2026
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: Attack Result Details labels nonboolean automated scores as Failure

3 participants