Skip to content

Add an Alpine APK repository client for v2 indexes - #107

Open
andrew wants to merge 4 commits into
mainfrom
apk-registry
Open

andrew wants to merge 4 commits into
mainfrom
apk-registry

Conversation

@andrew

@andrew andrew commented Oct 18, 2026

Copy link
Copy Markdown
Contributor

Part of #62: the v2 APKINDEX.tar.gz side. v3 Packages.adb decoding waits on an ADB decoder shared with git-pkgs/archives#30 and a fixture generated with apk mkndx.

Adds an apk registry with no default URL, so a missing URL gets the existing "no registry URL configured" error. The URL is one architecture directory such as https://dl-cdn.alpinelinux.org/alpine/v3.23/main/x86_64, or an exact index URL in one. Nothing is inferred from a mirror root. For a directory the client asks for Packages.adb first and falls back to APKINDEX.tar.gz only on a 404, as apk-tools does. If a Packages.adb is served, it returns ErrV3IndexUnsupported rather than reading the other index, so v3 support can later turn that error into a result without changing what any repository resolves to today. The index is fetched once per registry instance.

The parser reads the APKINDEX member across the signature and index gzip streams and maps the fields from apk-tools' apk_adb.c: name, version, description, URL, license, arch, build time (PublishedAt), maintainer, file and installed size, origin, commit, provider priority, depends, provides, replaces and install-if. Malformed lines, records with no P or V, bad numbers, truncated archives and archives with no APKINDEX member return ErrMalformedIndex. Versions sort with the apk scheme from vers, so -r10 sorts above -r9, and lookups match the exact version including the revision.

FetchDependencies returns D entries as runtime dependencies with the operator kept in Requirements (libcurl / =8.22.0-r0), and leaves out ! conflicts. Everything is also in version metadata. Maintainers in Name <email> form are split; anything else is kept whole as the name. Names may carry the PURL vendor namespace: alpine/curl looks up curl and FetchPackage returns Namespace: "alpine".

The C: checksum is not used as integrity. On curl-8.22.0-r0.apk, the Q1 value 25ef42c2…8161 is the SHA-1 of the package's second gzip stream (the control section), while the whole file's SHA-1 is b9119fc5…1e73. Putting it in Integrity would make a consumer that verifies downloads, such as git-pkgs/proxy, reject every package. It's exposed in metadata as checksum (raw), checksum_algorithm (sha1, or md5 for legacy hex), checksum_hex and checksum_covers: control.

Each version has one artifact, <name>-<version>.apk beside the index, with S: as its size and arch in its metadata. URLs().Download builds the same URL without loading the index, so fetch.Resolver resolves APKs with no request.

The test fixture is four records trimmed from the live v3.23 main/x86_64 index, packed into the same two-stream layout in the test. Against dl-cdn.alpinelinux.org, v3.23/main/x86_64 and v3.23/main/aarch64/APKINDEX.tar.gz resolve curl 8.22.0-r0 with sizes matching the files, and edge/community/x86_64 (about 2.5 MB compressed) loads in under a second.

https://claude.ai/code/session_01NxsA75HBjdWQKcdx7FDTDc

Copilot AI balanced review requested due to automatic review settings October 18, 2026 18:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@simonchrz

simonchrz commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

I read this from the proxy's side, for proxy mirror (git-pkgs/proxy#414). It fits: the artifact beside the index is the path the proxy's APK handler caches under ({repository}/{release}/{component}/{arch}/{name}-{version}.apk), so the mirror can build the directory URL from the upstream and the PURL's qualifiers and use FetchVersions as is. Leaving C: out of Integrity is right; the proxy would reject every package otherwise. A few things I noticed:

  1. The v3 probe downloads Packages.adb with GetBody only to see whether it exists. Today that's a 404 (edge/main/x86_64 answers 404, 153 bytes). Once a repository publishes one, every lookup downloads it, up to the 10 MB GetBody limit, before returning ErrV3IndexUnsupported. A HEAD, or a GET that doesn't read the body, would do.
  2. GetBody truncates at 10 MB without an error. Correction: that's only the state this branch is based on. On main, client: silent truncation at maxResponseSize produces confusing downstream errors #29 made it fail with ErrResponseTooLarge, so after a rebase an oversized index gets a clear error. With that, item 1's oversized Packages.adb would surface as ErrResponseTooLarge rather than ErrV3IndexUnsupported.
  3. The index is cached per Registry instance, and registries.New makes a new one each time. The proxy's version listing creates an instance per PURL, so mirroring 100 unversioned APK PURLs from one repository downloads the index 100 times (plus 100 v3 probes). Either a note that callers should reuse an instance per repository, or a small per-URL cache with revalidation, would help.
  4. URLs().Download builds the URL from the name and version without checking them. If fetch.Resolver gets them straight from a PURL, a version such as ../../edge/main/x86_64/other-1.0-r0 (percent-encoded in the PURL) would point at a file elsewhere in the repository. I haven't tried that through the resolver. The proxy validates on its side, but rejecting / and .. here would keep other callers safe too.

Minor: URLs().PURL leaves out arch, and distro where known, so the PURL it returns doesn't name the file it came from.

Probe Packages.adb with HEAD and refuse package file names containing path or URL characters.
@simonchrz

Copy link
Copy Markdown
Contributor

Thanks, all four points are covered.

One more: Resolver.ResolveWithOptions takes URLs().Download first, and for apk that always returns a URL, so selectArtifact and the arch qualifier check never run. pkg:apk/alpine/curl@8.22.0-r0?arch=aarch64 against an x86_64 registry resolves to .../x86_64/curl-8.22.0-r0.apk, and since the filename has no architecture, the wrong binary would be fetched without an error. Returning "" from Download would send it through the cached index instead.

Smaller: a 403 on the Packages.adb probe stops the lookup. S3 buckets without list permission answer 403 for missing keys, so repositories hosted there would never fall back to APKINDEX.tar.gz.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants