From 31dd42bb0d5e4bf6366056fd3bcf3dea4fcc9e09 Mon Sep 17 00:00:00 2001 From: yaythomas Date: Wed, 28 Oct 2026 23:07:48 +0000 Subject: [PATCH] docs: correct and shorten the release guide The note in How Publishing Works said PyPI rejects an already published filename. PyPI accepts a re-upload whose files match the published ones. It rejects the upload only when the files differ, for example after a README change without a version bump. That package's job then fails, and GitHub skips the emulator image job. The note and the checklist now describe both cases. The first-time publication section explained general PyPI and GitHub behavior. It now keeps the steps specific to this repository and links PyPI's guide. The protection rules appeared in four places. Release Permissions and Protection now states them, and other sections link to it. --- RELEASING.md | 215 +++++++++++++++++---------------------------------- 1 file changed, 70 insertions(+), 145 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index e0b4d6750..3de813882 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -28,129 +28,60 @@ or the initial version for a new distribution. ## Release Permissions and Protection -The GitHub `pypi` environment permits deployments only from **tags** matching: - -- `sdk-v*` -- `otel-v*` -- `testing-v*` -- `telemetry-exporter-v*` - -Branches, including `main` and temporary publishing branches, are not allowed. -GitHub checks the workflow run's ref; checking out an allowed tag or a merged -commit inside a branch-triggered workflow does not make that run eligible. - -The [protected tags ruleset](https://github.com/aws/aws-durable-execution-sdk-python/rules/25440902) -restricts creation, updates, deletion, and force pushes for the same tag -patterns. Have an actor permitted by that ruleset create the release tag at the -intended merged commit after CI passes. Treat published tags as immutable: use -a new version and tag for corrections instead of moving, deleting, or -recreating a published tag, even if your role permits those operations. - -When adding a package with a new tag prefix, configure its pattern in **both** -the [`pypi` environment's deployment rules](https://github.com/aws/aws-durable-execution-sdk-python/settings/environments) -and the protected tags ruleset before its first release. Updating only the -workflow matrix or this document is not enough. Keep these protections in -place for first-time publications as well as later releases. +The [`pypi-publish.yml`](.github/workflows/pypi-publish.yml) workflow publishes +to PyPI only for a release whose tag matches `sdk-v*`, `otel-v*`, +`testing-v*`, or `telemetry-exporter-v*`. Its `pypi-publish` jobs run in the +`pypi` environment, which accepts only runs on a matching tag. The +[protected tags ruleset](https://github.com/aws/aws-durable-execution-sdk-python/rules/25440902) +restricts who can create, update, delete, or force push a matching tag. + +If a release uses a tag that matches none of the patterns, GitHub creates the +tag and the release, but refuses to run the `pypi-publish` jobs. Before the +first release that uses a new prefix, add the prefix's pattern to both the +ruleset and the +[environment's deployment rules](https://github.com/aws/aws-durable-execution-sdk-python/settings/environments). + +Do not move, delete, or recreate a published tag. Correct a release with a +new version and a new tag. ## Publishing a Package for the First Time -Each distribution name needs its own PyPI project and trusted publisher. This -also applies when renaming a package: the old project's publisher does not -carry over to the new name. Trusted publishing uses GitHub Actions OIDC, so this -process does not require a PyPI API token on your machine or in GitHub secrets. - -### 1. Prepare the package - -Confirm the distribution name in the package's `pyproject.toml` (`[project].name`) -and its initial version in `__about__.py`. The distribution name is the name used -by `pip install`; it can differ from the Python import name. - -Add the package to **both** matrices in -[`pypi-publish.yml`](.github/workflows/pypi-publish.yml): its name and path in -`release-build`, and its name in `pypi-publish`. Add its path and tag prefix to -the [Packages](#packages) table above, and configure any new tag prefix as -described in [Release Permissions and Protection](#release-permissions-and-protection). -Merge the package and confirm CI passes before publishing. Follow -[CONTRIBUTING.md](CONTRIBUTING.md) for tests and run -`hatch build` from the package directory. From the repository root, run -[`check_dist_legal_files.py`](.github/scripts/check_dist_legal_files.py) with -`packages/` as its argument to verify the wheel and source archive -include `LICENSE` and `NOTICE`. - -### 2. Configure the PyPI publisher - -Sign in to the PyPI account that should own the new project. Open -[Publishing settings](https://pypi.org/manage/account/publishing/), find **Add a -new pending publisher**, select **GitHub**, and fill in: - -| Field | Value | -|-------|-------| -| PyPI Project Name | The exact `[project].name` from the new package's `pyproject.toml` | -| Owner | `aws` | -| Repository name | `aws-durable-execution-sdk-python` | -| Workflow name | `pypi-publish.yml` | -| Environment name | `pypi` | - -**Owner** is the GitHub organization, not the PyPI username. **Workflow name** -is the filename, without `.github/workflows/`; it is not the display name -`Upload PyPI Package`. Enter `pypi` for the environment even though the form -marks it optional, because the publishing job uses that GitHub environment. - -Click **Add**. The publisher remains pending until the first successful upload, -which creates the project and converts it to a regular trusted publisher. The -PyPI account that registered it becomes a project owner. A pending publisher -**does not reserve the name**. See PyPI's -[first-project publishing guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/). - -If you need to claim a name without uploading a release and have access to a -PyPI organization, use the organization's **Manage → Projects → Create and add -a new project** action instead ([instructions](https://docs.pypi.org/organization-accounts/actions/project-actions/#create-a-project)). -Then add the trusted publisher through that project's **Publishing** settings. -A public PyPI 404 alone does not prove a name is free; a project can exist -without any releases ([name availability](https://pypi.org/help/#project-name)). - -### 3. Publish the first version - -Follow [Cutting a Release](#cutting-a-release) below using an allowed release tag. -The initial version only needs to be unused under the new distribution name; -there is no need to bump it just to create the project. - -Check the entire release scope first. The current workflow publishes **all -packages in both matrices**, regardless of the release's tag prefix. After -PyPI publishing succeeds, it calls -[`ecr-release.yml`](.github/workflows/ecr-release.yml), which publishes the -testing emulator image only when the tag includes `testing-v`. Do not -create a GitHub release solely to claim one name if those other publications -are not intended. - -The current PyPI workflow has no package selector or manual-dispatch trigger, -and the environment rules prevent publishing from a temporary branch. If only -the new package should be uploaded, first review and merge workflow changes -that limit publication to the intended package and avoid the unrelated emulator -image release, while retaining the tag protections. Include those changes in -the release's tagged commit. For name reservation without an upload, use the -PyPI organization project-creation option above. - -### 4. Verify the project - -Check that the workflow ran on the intended release tag, that the tag and the -checkout step resolve to the selected merged source commit, and that both the -build and publish jobs succeeded. On PyPI, verify the project name, version, -wheel, and source archive. Compare their SHA-256 hashes with the workflow's -distribution artifact. Confirm the publisher now appears in the project's -**Publishing** settings and arrange the team's project ownership as needed. - -If publishing reports `invalid-pending-publisher` or `invalid-publisher`, check -the repository owner, repository name, workflow filename, and environment -against the form values above. Also check that the built distribution's name -matches **PyPI Project Name**. Correct the configuration before retrying; adding -the pending publisher by itself does not finish publication. - -If GitHub rejects access to the `pypi` environment, check the run's tag ref -against the allowed patterns. Retrying a branch run or changing its checkout -does not satisfy a tag-only deployment rule. If tag creation is rejected, have -an actor permitted by the protected tags ruleset create the intended release -tag. Do not relax the environment or tag protections to retry a publication. +A trusted publisher names the GitHub workflow that may upload to a PyPI +project. A pending publisher is a trusted publisher for a project that does +not exist yet. Each distribution name needs its own project and trusted +publisher. A renamed package needs a new project and publisher too. + +1. Set the package's initial version in its `__about__.py`. Choose its tag + prefix as [Tagging Convention](#tagging-convention) describes. In + [`pypi-publish.yml`](.github/workflows/pypi-publish.yml), add its name and + path to the `release-build` matrix and its name to the `pypi-publish` + matrix. Add it to the legal-files step in + [`ci.yml`](.github/workflows/ci.yml), the [Packages](#packages) table, and + the [Versioning](#versioning) list. +2. The PyPI account that adds the pending publisher becomes the project's + owner. So sign in as `durable-execution-dev`, the account that owns the + existing projects. In + [Publishing settings](https://pypi.org/manage/account/publishing/), add a + pending GitHub publisher + ([PyPI guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/)): + + | Field | Value | + |-------|-------| + | PyPI Project Name | `[project].name` from the package's `pyproject.toml` | + | Owner | `aws` | + | Repository name | `aws-durable-execution-sdk-python` | + | Workflow name | `pypi-publish.yml` | + | Environment name | `pypi` | + + Enter the environment name even though the form marks it optional. Without + it, PyPI accepts uploads from a run of `pypi-publish.yml` on any branch. + With it, PyPI accepts only jobs in the `pypi` environment, so only runs on + a matching tag can upload. Another PyPI user can still register the name + until the first upload. +3. Check the other packages as the note in + [How Publishing Works](#how-publishing-works) describes. Then + [cut a release](#cutting-a-release). The first successful upload creates + the project. ## Cutting a Release @@ -162,11 +93,6 @@ use the initial version prepared above. Commit and merge to `main`. ### 2. Create a GitHub Release -If the release creates a new tag, the actor creating it must be permitted by -the [protected tags ruleset](#release-permissions-and-protection). The tag must -point to the merged commit whose CI passed, including any publishing workflow -changes needed for this release. - 1. Go to the [Releases page](https://github.com/aws/aws-durable-execution-sdk-python/releases) on GitHub. 2. Click **Draft a new release**. 3. Select the intended merged commit as the target and create a new tag @@ -211,17 +137,21 @@ Creating a GitHub Release triggers the [`pypi-publish.yml`](.github/workflows/py 2. **Uploads** the built distributions as artifacts. 3. **Publishes** each package to [PyPI](https://pypi.org/) using trusted publishing (OIDC-based, no API tokens required). -The workflow runs on the `release: [published]` event. Its publishing jobs use -the protected `pypi` environment, so the release tag must satisfy that -environment's deployment rules before they can upload to PyPI. The workflow -and package source come from the tagged revision; later changes on `main` do -not change an existing release's source. - -> **Note:** The tag prefix does not select packages. The workflow builds and -> publishes every package in its matrices, including any unrelated version -> that has not been published yet. Check every package's version before -> creating a release. PyPI rejects an already published filename; with -> `fail-fast: false`, that job can fail while other packages still publish. +The workflow runs on the `release: [published]` event. The workflow file and +the package source both come from the tagged revision. So a re-run of an +existing release uses its original workflow and source, not later changes on +`main`. The `pypi-publish` jobs run in the `pypi` environment, which +[Release Permissions and Protection](#release-permissions-and-protection) +describes. + +> **Note:** The tag prefix does not select packages. The workflow uploads +> every package in its matrices on every release. PyPI publishes each version +> that it does not have yet. So check that each unpublished version belongs in +> this release. PyPI accepts a re-upload of a version it already has only if +> the files match the published ones. If the package changed after PyPI +> published that version, even only its README, PyPI rejects the upload with +> `400 File already exists`. That package's job then fails, and GitHub skips +> the emulator image job. Re-running the failed job repeats the rejection. The [`ecr-release.yml`](.github/workflows/ecr-release.yml) workflow publishes the testing emulator image only when the release tag names `testing-v`, @@ -321,17 +251,12 @@ Before publishing a release: `aws-durable-execution-sdk-python-telemetry-exporter`), follow [Publishing a Package for the First Time](#publishing-a-package-for-the-first-time) and configure its PyPI publisher before uploading -- [ ] Every unpublished version in the PyPI matrices is intended for this - release, along with the testing emulator image publication when the tag - includes `testing-v` +- [ ] Every unpublished version in the PyPI matrices belongs in this release +- [ ] For a release tag that includes `testing-v`, every package that + changed since its latest PyPI version has a new version, or GitHub + skips the emulator image job - [ ] Changes merged to `main` - [ ] CI checks pass on `main` -- [ ] Release tag points to the selected merged, CI-passing commit, including - the publishing workflow used for the release -- [ ] Release tag matches an allowed `pypi` deployment pattern and is covered - by the protected tags ruleset; any new prefix is configured in both -- [ ] The actor creating the release tag is permitted by the protected tags - ruleset - [ ] Release notes written with separate sections per package - [ ] Tag follows the naming convention (`sdk-vX.Y.Z`, `otel-vX.Y.Z`, `testing-vX.Y.Z`, `telemetry-exporter-vX.Y.Z`, or comma-separated)