Repository navigation
TEST: technique acceptance - #4016
Open
Roman Lutz (romanlutz) wants to merge 4 commits into
Open
Roman Lutz (romanlutz) wants to merge 4 commits into
Roman Lutz (romanlutz) wants to merge 4 commits into
Conversation
Exercise real preset editing, scenario defaults, estimate recovery and launch with isolated provider-free data. Retain desktop/narrow provenance and execution evidence, explain disabled choices, and use existing touch targets for preview actions. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve the technique descriptions, existing estimate status roles, and preview touch targets in their extracted components. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues block approval of the technique-selection acceptance coverage and accessibility changes.
No open issues 🎉
What changed in this PR
Adds real-backend acceptance coverage for Scenario GUI technique selection, building on the ScenarioDetail component split in #3526. UI changes focus on accessible explanations, estimate status announcements, and touch-friendly preview actions.
Changes:
- Adds eight desktop/narrow browser journeys covering presets, defaults, baseline selection, estimate recovery, and persisted execution.
- Introduces a provider-free backend fixture with isolated storage and containment regression tests.
- Adds dedicated Playwright/CI wiring and documents execution and evidence capture.
| File | Description |
|---|---|
| tests/unit/backend/test_scenario_techniques_fixture.py | Tests preset-write and lock-file containment. |
| frontend/src/components/Scenarios/ScenarioTechniqueSelector.tsx | Associates unavailable baseline controls with explanations. |
| frontend/src/components/Scenarios/ScenarioTechniqueSection.tsx | Adds accessible technique-limit and baseline explanations. |
| frontend/src/components/Scenarios/ScenarioRunPreviewDialog.tsx | Applies touch-target sizing to preview actions. |
| frontend/src/components/Scenarios/ScenarioEstimateSection.tsx | Announces updating and stale estimate states. |
| frontend/src/components/Scenarios/ScenarioDetail.test.tsx | Covers keyboard navigation, status announcements, and descriptions. |
| frontend/src/components/ScenarioPresets/ScenarioPresetEditor.test.tsx | Tests the unavailable baseline’s accessible description. |
| frontend/README.md | Documents isolated checks and evidence provenance. |
| frontend/playwright.scenario-techniques.config.ts | Configures the dedicated real-backend browser suite. |
| frontend/playwright.config.ts | Excludes the dedicated suite from ordinary discovery. |
| frontend/package.json | Adds the technique acceptance test command. |
| frontend/e2e/scenario-techniques.spec.ts | Implements eight browser journeys and evidence capture. |
| frontend/e2e/fixtures/scenario_techniques_backend.py | Provides isolated local scenarios and estimate fault controls. |
| .github/workflows/frontend_tests.yml | Adds a dedicated technique acceptance CI shard. |
Review effort: Balanced
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes the remaining Scenario GUI technique-selection acceptance gap with eight desktop/narrow Playwright journeys against the real backend, registry, preset storage, estimate service and execution path. The fixture uses deterministic local targets, converters and scoring, private SQLite/configuration/preset storage, and no external providers or developer environment files. Successful API responses are not intercepted. Only the loading/error cases deliberately hold one estimate or return one HTTP 503.
The journeys cover an actual preset edit, scenario-relative defaults, untouched preset values, aggregate expansion, independent baseline settings, keyboard/focus recovery, retained selections and displayed totals through failed/pending estimates, recovered totals/status, Run preview and persisted execution. Each primary journey completes exactly the two selected techniques plus baseline and verifies the actual request, saved configuration, plan and successful results.
The tightly scoped UI fixes associate unavailable technique/baseline controls with their existing explanations, add
role="status"to the two existing updating/stale estimate elements, and apply the existing touch-target class to both preview buttons. Coarse-pointer checks wait for the dialog to settle and measure at least 44px without fixed sleeps. Backend sizing/compatibility ownership and desktop behavior are unchanged. CI/package/Playwright wiring is feature-specific and additive.This is an independent leaf based on main including the ScenarioDetail split in #3526 (
0f5f6589e92824268b5151409ad19c8593e1dc5a), not stacked on the dataset or History follow-ups.ScenarioDetail.tsxexactly matches that upstream revision; the approved attributes live in the extracted technique, estimate and preview components. Dataset-label/count/metadata semantics are not changed.Tests and Documentation
Validated head:
aaaad689981ec0dd48c67f0309179ba277dc9501, clean worktree. Independent Reviewer critically reviewed all 14 leaf files after the split against upstream0f5f6589e92824268b5151409ad19c8593e1dc5a: APPROVE, no actionable findings. Its initial finding about missing rendered estimate-recovery proof was fixed and explicitly confirmed resolved in the post-split review.From
frontend:npm run test:e2e:scenario-techniques -- --reporter=line,jsonnpm test -- --runInBand src\components\Scenarios\ScenarioDetail.test.tsx src\components\ScenarioPresets\ScenarioPresetEditor.test.tsx --silent --json --outputFile="$evidence\jest-report.json"$evidenceis the retained exact-head evidence directory.npm run buildnpx eslint e2e\scenario-techniques.spec.ts playwright.scenario-techniques.config.ts src\components\Scenarios\ScenarioDetail.tsx src\components\Scenarios\ScenarioTechniqueSection.tsx src\components\Scenarios\ScenarioEstimateSection.tsx src\components\Scenarios\ScenarioRunPreviewDialog.tsx src\components\Scenarios\ScenarioTechniqueSelector.tsx src\components\Scenarios\ScenarioDetail.test.tsx src\components\ScenarioPresets\ScenarioPresetEditor.test.tsxnpx playwright test --list --reporter=lineExplicit acceptance/config TypeScript checking also passed with this command from
frontend:From the repository root:
uv run --no-sync pytest tests\unit\backend\test_scenario_techniques_fixture.py --basetemp .pyrit_cache\technique-fixture-post-split-unit -quv run --no-sync ruff check frontend\e2e\fixtures\scenario_techniques_backend.py tests\unit\backend\test_scenario_techniques_fixture.pyuv run --no-sync ruff format --check frontend\e2e\fixtures\scenario_techniques_backend.py tests\unit\backend\test_scenario_techniques_fixture.pyuv run --no-sync ty check frontend\e2e\fixtures\scenario_techniques_backend.py tests\unit\backend\test_scenario_techniques_fixture.py --output-format conciseuv run --no-sync python build_scripts\check_async_suffix.py frontend\e2e\fixtures\scenario_techniques_backend.pyThe README documents fixture isolation, deliberate fault injection, the acceptance command and screenshot provenance. Eighteen screenshot/provenance pairs, two successful execution records, both test reports and a checksummed verification manifest are retained in the session Library under
scenario-techniques\aaaad689981ec0dd48c67f0309179ba277dc9501. Each viewport captures preset edit, loading, failure, keyboard focus, preview, completed run and all three baseline states. Verification confirmed the clean source/compatibility head, all expected request/result shapes and private fixture paths. All six named external synthetic preset JSON paths and their six hidden lock sidecars were absent; no external artifacts were changed. Earlier captures retain their original head binding.No unresolved review or acceptance blocker. Existing Vite chunk-size/Pydantic warnings and Windows connection-reset logging did not fail the final suite. External-provider/live suites and JupyText were not run because no provider integration or notebooks changed.