Skip to content

Mirror Homebrew bottles - #610

Merged
andrew merged 4 commits into
git-pkgs:mainfrom
simonchrz:feat/mirror-brew
Oct 30, 2026
Merged

andrew merged 4 commits into
git-pkgs:mainfrom
simonchrz:feat/mirror-brew

Conversation

@simonchrz

@simonchrz simonchrz commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Part of #414.

proxy mirror pkg:brew/hello@2.12.3 failed with unsupported ecosystem: brew, although listing brew versions already worked. Homebrew publishes a formula's bottles as an image on upstream.homebrew_artifact, and since #411 the mirror can fetch images. So a brew PURL now resolves to that image and is mirrored like one, under the keys brew reads through HOMEBREW_ARTIFACT_DOMAIN:

  • The image name is Homebrew's: homebrew/core/<formula>, with @ written as / and + as x.
  • The tag comes from the formula's JSON API on upstream.homebrew_api. It carries the formula's revision and the bottles' rebuild: hello 2.12.3 with rebuild 1 is tagged 2.12.3-1. The API only describes the current version, so any other version is used as the tag as given (2.12.2, or 2.12.3-1 directly).
  • --platform selects bottles the way it selects image platforms: linux/arm64, linux/amd64, or darwin/arm64 for each macOS release's arm64 bottle.
  • A formula name with @ has to be written %40: pkg:brew/openssl%403. pkg:brew/openssl@3 is version 3 of openssl.

Listing a formula's versions also returned its versioned siblings (openssl@3.6, openssl@3.0 for openssl@3), which are other formulae. git-pkgs/registries#120 fixes that in the Homebrew client; until the proxy picks it up with a registries bump, mirroring an unversioned pkg:brew/openssl%403 also tries those tags.

Refused, each with a message: unknown formulae, a current version without bottles, and invalid names or versions (checked before any request). Casks, source archives and other taps aren't mirrored. Dependencies aren't mirrored with a formula, so they have to be listed too.

Tests:

  • TestHomebrewBottleImage resolves names and tags against a fake JSON API, including revision, rebuild, @, +, older versions and the refusals.
  • TestMirrorBrewMirrorsBottleImage checks that versioned and listed PURLs reach the image mirror with the image, tag and platforms.
  • TestMirrorBrewFailures covers a mirror without a resolver and an unknown formula.

The image mirroring itself is the code #411 tests.

I also ran it for real:

  • proxy mirror --platform linux/arm64 pkg:brew/hello pkg:brew/openssl%403 pkg:brew/nosuchformula@1.0 mirrored hello 2.12.3 (tag 2.12.3-1) and openssl@3 3.5.9 (11.5 MB) from ghcr.io, and failed the third with homebrew/core has no formula "nosuchformula".
  • I then ran serve with the upstream unreachable, and brew install hello in the homebrew/brew container with HOMEBREW_ARTIFACT_DOMAIN pointed at the proxy and fallback off. It got the manifest and the bottle from the cache, and hello printed "Hello, world!". The JSON API came from the metadata cache, filled by an earlier brew info.

pkg:brew PURLs failed with "unsupported ecosystem: brew", although listing their versions worked. Homebrew publishes a formula's bottles as an image on its artifact upstream, which proxy mirror can already fetch, so a brew PURL is now resolved to that image and mirrored like one: the formula's JSON API gives the tag, which carries the revision and the bottles' rebuild, and --platform selects bottles like image platforms.

Listing a formula's versions returned the versioned siblings the Homebrew client adds (openssl@3.6 for openssl@3) as versions of it; they are now left out.
git-pkgs/registries#120 stops listing versioned formulae as versions.
Enumerate went over the cognitive complexity limit after main added the
Julia uuid qualifier.
@andrew

andrew commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

@simonchrz sorry claude made a complete mess of this, I'm trying to clean it up.

@simonchrz

Copy link
Copy Markdown
Contributor Author

No worries, the content matches what I had. The Windows failure here and on #608 is TestCacheMissSlowClientFailsWithoutStoredCopy from #609, which depends on timing; #630 makes it deterministic.

@simonchrz simonchrz added the enhancement New feature or request label Oct 29, 2026
@andrew
andrew merged commit e09844e into git-pkgs:main Oct 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants