Skip to content

ci: reserve regtest host ports before e2e network steps - #1777

Merged
jvsena42 merged 1 commit into
masterfrom
ci/reserve-e2e-ports
Nov 5, 2026
Merged

jvsena42 merged 1 commit into
masterfrom
ci/reserve-e2e-ports

Conversation

@jvsena42

@jvsena42 jvsena42 commented Nov 4, 2026

Copy link
Copy Markdown
Member

This PR:

  1. Reserves the regtest stack's host ports at the start of each E2E shard, so docker compose up cannot fail to bind them

Description

The e2e-tests-local shards start the regtest stack from bitkit-e2e-tests' docker/docker-compose.yml. It publishes 39388 (bitcoind P2P), 43782 (bitcoind RPC) and 60001 (electrs) on the host. All three sit inside Linux's ephemeral port range (32768–60999). Earlier steps in the job make outbound connections (npm ci, image pulls), and one of them can be given one of those ports as its local port. When that happens, docker compose up fails before any test runs:

Error response from daemon: failed to set up container networking: driver failed programming external connectivity on endpoint bitcoin (…): failed to bind host port for 0.0.0.0:39388

This hit the onboarding_backup_numberpad shard twice in a row on #1763: run 41919439910 and run 41926929379, both on fresh ubuntu-latest runners.

The new first step sets net.ipv4.ip_local_reserved_ports=39388,43782,60001, so the kernel never hands these ports to an outbound connection. It runs before any network step, because a reservation does not free a port that a connection already holds. The other published ports are below 32768 and not at risk.

Out of Scope

  • bitkit-ios runs its E2E on macOS, whose ephemeral range starts at 49152, so 39388 and 43782 are safe there. 60001 is inside that range, but no iOS failure has been seen.
  • Not publishing 39388 at all would need a change in bitkit-e2e-tests' compose file, after checking that nothing on the host uses bitcoind P2P.

Preview

N/A

QA Notes

Automated Checks

  • The workflow YAML parses.
  • This PR touches .github/workflows/e2e.yml, which the change filter includes, so its own e2e-tests-local matrix runs with the new step. All seven shards passing on this PR is the check.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge.

Summary

Adds a first step to each local E2E shard that reserves host ports 39388, 43782, and 60001 before later steps make outbound connections.

  • Each E2E shard reserves the regtest ports before network setup.

Reviews (1) · Last reviewed commit: "ci: reserve regtest host ports before e2..." · Reviewed by Greptile

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Regtest APK

Built from ce5cae1 (run).

Download bitkit-dev-debug universal APK (expires in 30 days).

@talosmachina talosmachina left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No findings. Reserves the regtest stack's three ephemeral-range host ports (39388, 43782, 60001) with net.ipv4.ip_local_reserved_ports as the first step of each e2e-tests-local shard, so no earlier outbound connection can take one before docker compose up. Reviewed ce5cae1, light tier, static review (the branch was not built); the claims below were checked against the compose file and a kernel probe.

What I checked, and 2 candidates I ruled out

Read in full: .github/workflows/e2e.yml at ce5cae1, bitkit-e2e-tests docker/docker-compose.yml on main
Port list: the default-profile services publish 43782, 39388 (bitcoind), 60001 (electrs), 80, 8080, 9735, 10009, 3003 and 30001 (lnurl-server); only the three reserved ones fall in Linux's default ephemeral range 32768-60999, so the list is complete for the stack this job starts.
Mechanism, probed: in a container with ip_local_port_range=39380 39396 and ip_local_reserved_ports=39388, sixteen outbound connections were handed every port in the range except 39388, then exhausted the range, and an explicit bind(0.0.0.0:39388) still succeeded. So the reservation keeps the kernel's auto-assignment off the port while leaving docker-proxy's explicit bind untouched.
Ordering: the new step precedes every network step in the job (resolve-e2e-ref, both checkouts, caches, npm ci, docker compose pull).
CI: build-and-test green; build-local and the seven e2e shards that exercise the step were still pending at review time.

Ruled out

  • sysctl -w replaces rather than appends to an existing reservation list: no step in this job sets a reservation, so the write has nothing of the workflow's own to clobber.
  • The reservation breaks the stack's own bind: reserved ports block only automatic assignment; the explicit bind in the probe above succeeded.

Merge confidence: 4/5, the change is a CI step whose only test is this PR's own e2e shards, still running.

@jvsena42
jvsena42 requested a review from piotr-iohk November 5, 2026 11:15
@jvsena42 jvsena42 self-assigned this Nov 5, 2026
@jvsena42
jvsena42 enabled auto-merge November 5, 2026 11:16

@piotr-iohk piotr-iohk left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA review

Scope: full PR review at ce5cae13, against merge base 21f0b40e, covering port reservation, shard setup ordering, and the Compose port contract.

No new actionable code findings.

Validation: local YAML parsing, shell syntax, Compose port coverage, and diff whitespace checks passed. E2E run 41931029680 matches the pinned head/base; all seven shards completed the reservation, Docker setup, and first test attempt successfully. The onboarding shard log confirms the sysctl value. Dependency comparison used bitkit-e2e-tests 80e26d4e, matching that CI checkout. Reservation semantics were checked against the Linux kernel documentation.

Device testing: not performed in this review. Existing CI results were inspected; no device or stack was started locally. Additional app runtime testing is unnecessary for this CI-only change.

Reviewer model: gpt-6.1-sol
Reasoning effort: high

@jvsena42
jvsena42 merged commit 6ee86de into master Nov 5, 2026
26 of 28 checks passed
@jvsena42
jvsena42 deleted the ci/reserve-e2e-ports branch November 5, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants