[Roman Lutz (romanlutz)]

Description

Close the Scenario GUI dataset/estimate acceptance gap with real backend handlers and fix the concrete display defects it exposed.

• Derive the displayed dataset selection limit from current edits and defaults, rather than a retained successful estimate. Keep configured budgets distinct from population counts.
• Identify a retained estimate as the last successful estimate, announce refreshing/stale states on the existing elements, and keep metadata/loading status visible on narrow screens.
• Add isolated desktop/narrow acceptance for dataset search, per-source/default/total caps and provenance, preset save/reload, target-free estimates, target-required launch, keyboard operation, contract-supplied precision, and explicitly injected loading/error states. Successful estimate and launch responses use actual handlers and deterministic local data/targets. Estimate-only cases do not prepare/download datasets or call providers.

The fixture explicitly binds private configuration, SQLite and preset storage, and checks actual source/document/lock containment before mutations. The dedicated Playwright command, default-suite exclusion, CI shard and README instructions are additive.

Merged and adapted to #3526 at upstream 0f5f6589e92824268b5151409ad19c8593e1dc5a. This keeps the extracted sections and single shared launch input, with estimate status rendering in ScenarioEstimateSection. There are no backend API changes or unrelated technique/History fixes.

Independent review found two acceptance reliability issues: a debounced reset could consume a reload fault, and fixed preset names collided on repetitions/retries. Both are fixed by awaiting the reset response before fault arming and using attempt-unique preset names. A fresh post-split Reviewer review of all 15 changed files at e6ebebcf26cd3b6bdafcce3633dfa5aa72cb5584 returned APPROVE, with no actionable findings. This was an independent source review, not an independently executed test run.

Tests and Documentation

All results below are from clean head e6ebebcf26cd3b6bdafcce3633dfa5aa72cb5584, after the #3526 merge. $evidence denotes the external, head-specific artifact directory, not a source-tree change.

Build passed:

$env:PYRIT_PYTHON = (Resolve-Path .venv\Scripts\python.exe).Path
npm --prefix frontend run build

Frontend units passed: 306 tests in 10 suites:

npm --prefix frontend test -- --runInBand --runTestsByPath src\components\Scenarios\ScenarioDetail.test.tsx src\components\Scenarios\ScenarioDatasetTable.test.tsx src\components\Scenarios\datasetPresentation.test.ts src\components\Scenarios\ScenarioRunEstimate.test.tsx src\components\ScenarioPresets\ScenarioPresetEditor.test.tsx src\components\Scenarios\scenarioLaunchForm.test.ts src\components\Scenarios\scenarioLaunchRequest.test.ts src\hooks\useScenarioDetail.test.tsx src\hooks\useScenarioLaunch.test.tsx src\hooks\useScenarioRunEstimate.test.tsx --json --outputFile="$evidence\frontend-unit-tests.json"

Real-backend browser acceptance passed: 12 attempts, no retries, flaky results or skips:

# From frontend; desktop and narrow, repeated twice.
$env:PLAYWRIGHT_JSON_OUTPUT_NAME = "$evidence\playwright-report.json"
npx --no-install playwright test --config playwright.scenario-datasets.config.ts --workers=1 --repeat-each=2 --retries=0 --fail-on-flaky-tests --reporter=line,json --output="$evidence\playwright-results"

Containment passed: seven tests, including rejected storage escapes:

uv run --no-sync pytest tests\unit\backend\test_scenario_dataset_fixture.py --basetemp .pyrit_cache\dataset-post-split-containment -q

Python lint, typing and async naming passed:

uv run --no-sync ruff check frontend\e2e\fixtures\scenario_dataset_backend.py tests\unit\backend\test_scenario_dataset_fixture.py
uv run --no-sync ty check frontend\e2e\fixtures\scenario_dataset_backend.py tests\unit\backend\test_scenario_dataset_fixture.py
uv run --no-sync python build_scripts\check_async_suffix.py frontend\e2e\fixtures\scenario_dataset_backend.py tests\unit\backend\test_scenario_dataset_fixture.py

Frontend lint and strict browser-spec typing passed:

# From frontend.
npx --no-install eslint e2e\scenario-datasets.spec.ts playwright.scenario-datasets.config.ts src\components\Scenarios\datasetPresentation.ts src\components\Scenarios\datasetPresentation.test.ts src\components\Scenarios\ScenarioDatasetTable.tsx src\components\Scenarios\ScenarioDatasetTable.test.tsx src\components\Scenarios\ScenarioDetail.tsx src\components\Scenarios\ScenarioDetail.test.tsx src\components\Scenarios\ScenarioEstimateSection.tsx
# From repository root, using the temporary config retained with the evidence.
npm --prefix frontend exec -- tsc --project .pyrit_cache\scenario-datasets.tsconfig.json
git diff --check origin/main...HEAD

Retained exact-head evidence includes 60 PNGs, 24 actual-handler JSON attachments, SHA256/dimension indexing, the clean compatibility stamp, test reports and review record. Sixteen saved presets are distinct across the two repetitions. The GUI run-plan captures show execution in progress, not completed-GUI evidence; separate backend JSON proves completed totals of four and six. Loading/error captures are explicitly identified as fault injection. Earlier captures retain their original head binding.

Updated frontend/README.md documents the feature-specific command, isolation and acceptance coverage. Local validation used Windows, Python 3.14, Node 26 and Chromium. Remote Linux CI, full repository suites, physical touch input and screen-reader audio have not been claimed as locally verified. Existing build chunk-size and Pydantic field-shadowing warnings remain unchanged.

JupyText: not run, because no executable documentation or notebooks changed.

[Copilot]

🟢 Approval recommended

No active review issues remain for the dataset-limit and estimate-status display changes.

No open issues 🎉

What changed in this PR

Fixes Scenario dataset-limit and estimate-status displays, while adding isolated browser coverage using real backend handlers. Preserves the shared launch input and extracted components introduced in #3526 without changing backend APIs.

Changes:

• Derives displayed dataset limits from current edits and defaults, independently of retained estimates.
• Announces refreshing and stale estimates and keeps dataset-loading status visible on narrow screens.
• Adds desktop and narrow browser acceptance tests, guarded local fixtures, containment tests, CI integration, and documentation.

File	Description
tests/​unit/​backend/​test_scenario_dataset_fixture.py	Tests preset storage containment and rejected path escapes.
frontend/​src/​components/​Scenarios/​ScenarioEstimateSection.tsx	Announces refreshing and last-successful estimate states.
frontend/​src/​components/​Scenarios/​ScenarioDetail.tsx	Displays the current configured dataset limit.
frontend/​src/​components/​Scenarios/​ScenarioDetail.test.tsx	Covers current limits during refreshing and stale estimates.
frontend/​src/​components/​Scenarios/​ScenarioDatasetTable.tsx	Shares limit resolution and exposes loading status outside responsive columns.
frontend/​src/​components/​Scenarios/​ScenarioDatasetTable.test.tsx	Tests visible loading and metadata-refresh announcements.
frontend/​src/​components/​Scenarios/​datasetPresentation.ts	Adds a shared total-limit resolver.
frontend/​src/​components/​Scenarios/​datasetPresentation.test.ts	Tests explicit limits and default fallback behavior.
frontend/​README.md	Documents isolated acceptance checks and their coverage.
frontend/​playwright.scenario-datasets.config.ts	Configures desktop and narrow fixture-backed browser projects.
frontend/​playwright.config.ts	Excludes fixture-only tests from the default suite.
frontend/​package.json	Adds the dedicated dataset acceptance command.
frontend/​e2e/​scenario-datasets.spec.ts	Exercises dataset configuration, presets, estimates, launches, and injected failures.
frontend/​e2e/​fixtures/​scenario_dataset_backend.py	Provides deterministic local data, real handlers, and storage guards.
.github/​workflows/​frontend_tests.yml	Adds dataset acceptance projects to CI.

_{Review effort: Balanced}