Skip to content

test(repo): fail the run on skipped tests - #200

Merged
mrcsin merged 1 commit into
masterfrom
xunit-fail-skips
Nov 18, 2026
Merged

mrcsin merged 1 commit into
masterfrom
xunit-fail-skips

Conversation

@mrcsin

@mrcsin mrcsin commented Nov 18, 2026

Copy link
Copy Markdown
Member

Problem

A test marked Skip stops running and the suite still reports green, so a disabled test can go unnoticed.

What changed

  • SemiStep.Tests/xunit.runner.json sets failSkips: true (same as SemiSpec), copied to the output by the test project.
  • docs/perf/README.md notes that explicit perf probes still count as not run and fail nothing.

Verification

  • No test has Skip today; dotnet test: 1444 passed, 0 failed, same as master.
  • Proof: adding Skip to one test made the run fail (FAIL_SKIP, exit 1); the edit was reverted.
  • The 8 explicit perf probes stay out of the default run and fail nothing.

🤖 Generated with Claude Code

A test marked Skip passed the run silently, so a test could stop
running and nobody would notice. xunit.runner.json now sets failSkips,
as in SemiSpec, and a skipped test fails dotnet test. Explicit
performance probes count as not run, not skipped, and stay out of the
default run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mrcsin
mrcsin merged commit 4203848 into master Nov 18, 2026
1 check passed
@mrcsin
mrcsin deleted the xunit-fail-skips branch November 18, 2026 14:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant