Skip to content

[RFD]: Package signing and repository publishing #43

Description

@seantronsen

Decision Goal

Choose how OpenCHAMI release packages (rpm/deb) are securely signed and published to dnf/apt repositories.

Category

Tooling

Stakeholders / Affected Areas

  • Maintainers of OpenCHAMI service repos (release pipelines)
  • Infrastructure owners (repository hosting, signing keys)
  • Downstream users installing OpenCHAMI packages

Decision Needed By

https://github.com/OpenCHAMI/github-actions/milestone/1 (v4.0.0)

Problem Statement

We need one secure way to sign OpenCHAMI's rpm/deb packages (30+ repos) and publish them to dnf/apt repositories that work with stock tooling.

Current approaches:

  • Three-key chain (gpg-sign-artifacts, release-signed-artifacts): per-run keys certified up to an offline master. rpm doesn't follow the chain, so clients must import every per-run key, and the repo-cert key is a CI secret.
  • Per-repo publishing (Adding publish-rpm-repository action/workflow #16): each repo pushes to S3. Publishers can race, re-runs collide, and every publish regenerates the shared apt metadata.

Constraints:

  • Scales to 30+ repos without per-repo key handling.
  • Supports rpm (signed packages) and deb (signed repository metadata).

Preferences:

  • Runs entirely in GitHub Actions.

Proposed Solution

Note

TBD. The chosen solution will be recorded here once discussion has finished and consensus is reached.

Signing and publishing are independent choices. Pick one of each (A–D, 1–3), then compare the combinations. Build-time signing (B, C) can use the nfpm signing available in the GoReleaser workflows from #41.

Signing

  • A. Three-key chain (current): offline master -> repo-cert -> per-run key signs each RPM.
  • B. Single key at build, from a secret: an org/repo secret, optionally gated by a tag-only Environment.
  • C. Single key at build, from Vault/KMS: the key is released via OIDC only to the release workflow on v* tags (or never leaves the KMS).
  • D. At publish: builds are unsigned; the publisher signs packages and metadata.

Warning

Distros verify differently: rpm checks per-package signatures, apt checks signed repo metadata (Release), pacman checks per-package signatures and optionally the repo database. #41 gives us all of these formats almost for free, so pick an approach that holds up on security and works across deb, rpm, and possibly pacman distros, not just rpm.

Publishing

  1. Per-repo push (Adding publish-rpm-repository action/workflow #16): each service repo publishes to the archive directly.
  2. Central GitHub repo: one tightly held repo collects releases (e.g. from draft releases) and publishes serially.
  3. External pull server: a server we run polls releases, verifies, signs, and publishes (not all-GitHub).

Combinations

1. Per-repo push 2. Central repo 3. Pull server
A current #16 design verify chain, re-sign with one archive key verify chain, re-sign with one archive key
B key in every repo's CI packages signed at build; central repo signs metadata packages signed at build; server signs metadata
C key held in Vault/KMS packages signed at build; central repo signs metadata packages signed at build; server signs metadata
D n/a central repo signs everything server signs everything

Archive key: one long-lived publisher key that signs admitted packages and repository metadata; it's the only key clients import (like Debian's debian-archive-keyring).

Evaluation criteria: where durable keys live, number of client-facing keys, publish races and apt metadata scaling, operational effort, and the GitHub Actions preference.

Feedback wanted: comment with your preferred combination, where the repositories should be hosted, and any criteria we've missed.

Alternatives Considered

Options outside the matrix above:

  • Hosted package services (Cloudsmith, Packagecloud, Fedora COPR, openSUSE OBS): CI pushes packages; the service hosts the repositories, generates metadata, and serializes publishes. Some sign with their own keys. Trades control and trust for little to no infrastructure.
  • Off-the-shelf repository managers (Pulp, aptly, reprepro, createrepo_c): not a separate option, but a likely implementation behind a central repo (2) or pull server (3), leaving us to write only verification and glue.
  • Per-project archives: each service repo owns its own repository path, so there's no shared metadata to race on or rebuild. Clients add one repo/source per project.
  • Sigstore/cosign: already used for image and build attestations, but dnf, apt, and pacman verify GPG signatures, so it can't replace package signing.
  • GitHub Packages: has no rpm/deb registry, so it can't host the repositories.

Other Considerations

  • Key custody and rotation: whichever option wins, one durable key ends up client-facing. We need an owner, a rotation/revocation plan, and a way to ship a new key to clients.
  • Trust anchors: anything that verifies upstream signatures (e.g. the chain in A) must pin its trust anchor out of band, never fetch it alongside the packages it verifies.
  • Storage integrity: object storage like S3 gives durability, not integrity. Repository metadata signing (repo_gpgcheck=1, signed Release) is what protects the archive.
  • Channels and retention: stable vs. pre-release repositories, and how long old versions stay published.
  • Hosting: where the repositories live (e.g. S3/CloudFront, GitHub Pages, a hosted service), under which domain, and who owns the account. Independent of the options above.

Related Docs / PRs

Activity

  1. added this to the v4.0.0 milestone on Oct 12, 2026
  2. self-assigned this
    on Oct 12, 2026
  3. changed the title [-][RFD]: Automated Package Signing and Deployment.[/-] [+][RFD]: Package signing and repository publishing[/+] on Oct 12, 2026
  4. seantronsen commented on Oct 13, 2026

    @seantronsen
    ContributorAuthor

    Meeting outcome: recommended approach (D2)

    The recent meeting recommended D2 from the matrix: sign at publish, from a central repo.

    • Signing key: one GPG keyset stored as an org secret, scoped to the central repo only. Service repos never see it.
    • Service repos: on release, build and create the GitHub release as today (packages unsigned). Nothing else.
    • Central repo: a scheduled workflow (every 15 minutes) polls a statically configured list of service repos for new releases, downloads their packages, signs packages and repository metadata with the org key, and publishes. Publishes run one at a time, so there are no races or concurrent metadata rebuilds.

    Note

    The meeting discussed having service repos trigger the central repo on release. GITHUB_TOKEN can't dispatch workflows in another repo, so that would need a GitHub App token or a PAT in every service repo. Polling avoids those extra credentials.

    Pros

    • Runs entirely in GitHub Actions.
    • The signing key lives in one place, and clients import a single key (the archive key).
    • Service repos need no signing secrets or cross-repo tokens.
    • Serial publishing: no races or concurrent metadata rebuilds.
    • Adding a repo is a one-line change to the central repo's list.

    Cons

    • The central repo and its workflows are publicly readable.
    • The signing key sits in a public repo's secrets, so the repo needs heavy protection (branch protection, required reviews, restricted write access) and should only be changed by well-trusted maintainers.
    • Its workflows become the main extraction target and need close monitoring (workflow changes, runs, secret access).
    • Single point of failure: whoever compromises the central repo can sign anything.
    • Org owners can rescope an org secret to other repos, so org admin access also guards the key.
    • Publishing lags releases by up to 15 minutes, more when GitHub delays scheduled runs.
    • GitHub disables scheduled workflows in public repos after 60 days without repo activity, so polling can silently stop.
    • The static repo list has to be maintained.

    Open items

    • Admission: how the central repo verifies that a release came from an OpenCHAMI repo's release workflow before signing it (e.g. build provenance attestations).
    • Hosting and key custody/rotation (see Other Considerations).
  5. seantronsen commented on Oct 14, 2026

    @seantronsen
    ContributorAuthor

    Decision: A2, central repo verifies the chain and re-signs

    Today's meeting chose A2 from the matrix. This replaces the D2 recommendation above.

    • Service repos: keep the existing three-key chain (gpg-sign-artifacts / release-signed-artifacts). Packages are signed at build with per-run keys certified up to the offline master.
    • Central repo: on a schedule (about every 10 minutes; the interval is still open), checks a fixed list of service repos for new releases. For each new release it downloads the packages, verifies the signing chain against the pinned offline master key, re-signs packages and repository metadata with the archive key, and publishes to the S3-backed dist server.
    • Archive key: a secret on the central repo, which only a small set of maintainers can administer. It's the only key clients import, and it's rotated yearly.
    • Keyring package: the central repo builds and signs openchami-archive-keyring (rpm and deb). During a rotation it ships the current and next public keys, so clients pick up the new key through normal upgrades. First-time setup fetches the key from the dist server over HTTPS.

    The chain is no longer what clients trust. It proves a package came from an OpenCHAMI release workflow, which answers the admission question above.

    Open items

    • Polling interval.
    • Archive key rotation: manual or automated.
    • S3 hosting details (domain, account owner, how the repository metadata gets rebuilt).
    • What happens to the v4.0 deprecations of the chain workflows and actions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

rfdRequest for Discussion

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions