Repository navigation
Generate Cheat Sheet rows in golden dataset builder - #1198
KaavyaIndhu wants to merge 1 commit into
Conversation
|
Important Review skippedReview was skipped as selected files did not have any reviewable changes.
|
| Layer / File(s) | Summary |
|---|---|
Generate and append positive rows scripts/build_golden_dataset.py |
Adds Authorization, REST Security, and Server Side Request Forgery Prevention fixtures. build_positive_cheatsheets() maps their sections to CRE identifiers and raises ValueError when a fixture section has no mapping. build() appends the generated rows. |
Estimated code review effort: 2 (Simple) | ~10 minutes
Change: Bug fix
Possibly related issues
- Golden dataset determinism test fails: fixture has rows the generator can’t produce #1194: Identifies missing generator support for the three Cheat Sheet rows that this change adds.
Possibly related PRs
- OWASP/OpenCRE#1088: Includes golden-dataset work and identifies a fixture follow-up that this change addresses.
Suggested reviewers: northdpole, pa04rth
Merge Risk: 
Minimal · previous review 1936b
-
The existing formatting drift is not caused or worsened by this change. No actionable merge-blocking risk remains after normal checks.
Pre-merge checks: 
5
-
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 1 files.
-
Finishing Touches
-
Generate unit tests (beta)
-
- Create a new PR
-
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
CodeRabbit Verify
- Run CodeRabbit Verify to test the changes in this pull request.
OWASP#1036 added three OWASP Cheat Sheet rows to golden_dataset.json, but build_golden_dataset.py had no code to produce them, so --check could never match the committed fixture. The builder now generates them, reading CRE mappings from owasp_cheatsheets_supplement.json. Part of OWASP#1194
1936b7f to
3309aed
Compare
|
@KaavyaIndhu I think adding I would suggest separating the deterministic dataset generation from live database retrieval. We could maintain version-controlled input fixtures for CWE records, their CRE relationships, and Cheat Sheet cases, and have the generator consume those fixtures. The generated output can then be compared against Separately, we should retain integration tests against the actual database to ensure that changes in extraction, adapters, and relationship retrieval are still handled correctly. A few additional things worth considering:
This would solve the current nondeterminism without sacrificing database integration coverage. @PRAteek-singHWY , review this PR and also suggest what you think .... |
|
Thanks for the review @Pa04rth ! That makes total sense, and I’ll start working on it right away and put up a PR. One more thing I thought I could add is once the generator only uses fixtures, the --check test no longer needs a DB, so it could run in CI on every PR and catch drift early. |
|
Thanks @KaavyaIndhu. Adding I ran it locally against my
#1216 already includes this commit and also fixes the CWE half, so I'd treat #1216 as the PR to review and merge. Once it lands this one has nothing left to add and can be closed. My review of the combined change is on #1216. @northdpole for merge order: #1216 contains this commit, so this one can close once #1216 lands. |
Part of #1194.
#1036 added 3 Cheat Sheet rows to
golden_dataset.json(authorization, rest-security, ssrf-prevention), butbuild_golden_dataset.pyhad no code for them, so--checkcouldn’t match the fixture.What changed:
build_positive_cheatsheets(), which builds the 3 rows and takes the CRE IDs fromowasp_cheatsheets_supplement.json(the source named in the rows’ provenance).build(), matching their position in the fixture.Verification:
--checkstill reports drift, but only for the CWE rows picked bybuild_positive_multilink(). That’s the second half of Golden dataset determinism test fails: fixture has rows the generator can’t produce #1194, and I’ll send it as a separate PR once we agree on the approach.