Repository navigation
Conversation
Signed-off-by: Nirmal Khedkar <nirmalhk7@gmail.com>
Signed-off-by: Nirmal Khedkar <nirmalhk7@gmail.com>
Signed-off-by: Nirmal Khedkar <nirmalhk7@gmail.com>
NetworkTheoryAppliedResearchInstitute
left a comment
There was a problem hiding this comment.
Thanks, Nirmal. This is careful CI work, and the docs are honest about what fuzzing does and doesn't prove.
I reviewed only the top commit, 1b084aa, against its parent f3e09c5 (the #13 head). It changes .github/workflows/reliability.yml, RELIABILITY.md and the new CONFORMANCE.md section.
What was not run: I did not build or run any code from this stack locally. The three fork runs for 1b084aa (CI, dependency-leaf and Protocol reliability) are still waiting for a maintainer to approve them, so this head has no build, test, race or fuzz results yet. The only reported check is DCO, which passes. Everything below comes from reading the code.
What looks right
- Safe for fork PRs: it uses
pull_request, notpull_request_target, setspermissions: contents: read, uses no secrets, and passes matrix values throughenvrather than inline inrun:. shell: bashon the fuzz step makes GitHub add-o pipefail, sogo test ... | teestill fails the step when the fuzzer finds something.- Fuzzing is time-bounded: 10s on PRs, pushes and manual runs, 60s on the schedule, with a 15-minute job limit. Each matrix entry matches exactly one fuzz function, and
go test -race ./...covers every package. - Putting this in its own workflow leaves
ci.ymlunchanged, so the lean-code invariant test still findsgo test ./...and the Python reproducer.
Before approval
- The fork runs need to be approved and to pass: CI, dependency-leaf and all 6 reliability jobs. The race job has never run in this repo. A new push, for example with the doc fixes below, would also give you runs against current main, which now includes #15.
- Merge order is #12, then #13, then #14. This PR carries #13's
Verifychanges; #13 is now approved at f3e09c5. If you rebase, please keep #12's commit: #13'sdecode_fuzz_test.gousestestCoordinator,roundTripFunc,trackedBodyandclientCallsfrom #12'stransport_test.go.
Should fix (fine in the same push)
- RELIABILITY.md:66-84, "Bug-Build-Test report draft": this section is notes about this PR, such as "On the parent implementation…" and "No Slack receipt was submitted…", and it will go stale once merged. Please remove it from the file and file it through the Bug-Build-Test workflow, so the find gets credited, or move it into the PR description. Lines 1-64 are good.
Nits
- CONFORMANCE.md:60: "Reliability checks" comes after "Product-specific notes (last, per the ordering rule)" at :56. Please move it up, for example after "Dependency declaration" (:52-54).
- reliability.yml:14-16: push, schedule and manual runs on main share one concurrency group with
cancel-in-progress: true. A merge can cancel the daily 60s run, and the daily run can cancel a push run. Suggestcancel-in-progress: ${{ github.event_name == 'pull_request' }}. - reliability.yml:33 and :57-63: if a fuzz job ever hits the 15-minute job limit, GitHub cancels it and the
if: failure()upload is skipped, so RELIABILITY.md:59 wouldn't hold in that case. Addtimeout-minutes: 10to the "Bounded fuzz campaign" step (a step timeout counts as a failure), or useif: failure() || cancelled(). - RELIABILITY.md:51-52 says the existing checks "remain mandatory", but only DCO is a required check on main. Suggest "continue to run in ci.yml and dependency-leaf.yml".
- RELIABILITY.md:48-49: beyond the seeds, FuzzOperatorVerification can in practice only check panic safety, because random inputs won't produce a valid ed25519 signature. FuzzClientJSON's success path only checks that the body was closed. One sentence saying so would keep the doc's claims exact.
- Optional: add
persist-credentials: falseto both checkouts. SHA-pinning the actions would be better as a separate PR that covers all three workflows.
There are no wire, record or identity changes here. Once the runs are green and #12 and #13 are in, this looks ready apart from the items above.
Signed-off-by: Nirmal Khedkar <nirmalhk7@gmail.com>
Signed-off-by: Nirmal Khedkar <nirmalhk7@gmail.com>
|
Resolved the integration conflict in The PR diff is now only Verification on the new head:
Tests ran from an isolated copy of committed tree |
Existing CI runs protocol tests but does not exercise the race detector or active fuzz campaigns. This adds one race job and five bounded fuzz jobs, with documented reproduction commands and failure evidence.
Each fuzz target runs for 10 seconds on pull requests, pushes to main and manual runs, or 60 seconds on the daily schedule. Fuzz steps have a 10-minute timeout inside a 15-minute job, and failures attempt to upload logs and corpus inputs for 14 days. Only pull-request runs cancel in-progress jobs; both reliability checkouts disable credential persistence. The workflow uses pull_request, read-only contents permissions and no secrets.
RELIABILITY.md documents transport coverage, timestamp regressions, the limited assertions made by operator/client fuzzing and the limits of artifact retention. CONFORMANCE.md links those checks before the final product-specific section. Historical bug-report notes are absent from the maintained guide.
#12 and #13 have merged. This branch incorporates current main at
cfe78d1399c1f395c77eda0be991b3206dfdb0cfthrough an append-only merge, preserving published history. The diff against main is now exactly three files: the new reliability workflow, RELIABILITY.md and the conformance link. All protocol source, existing tests/helpers, SPEC, dependency declaration and golden vectors match main.Verification on
78124faace9b38eeb3c6345c8d9f3d8e27fc6301:go test -race ./... -count=1, build and vet pass.Verification used an isolated copy of the exact committed tree with Go 1.27.1 and loopback access for real HTTP tests. Hosted CI remains a separate approval gate; prior-head results do not establish a pass on this revision. The daily schedule starts after the workflow reaches the default branch. No protocol release, consumer bump, wire change or production-readiness claim is included. The integration commit carries a DCO sign-off.