Skip to content

WPB-22956: migrate team metadata to PostGreSQL - #5631

Open
blackheaven wants to merge 5 commits into
developfrom
gdifolco/WPB-22956-migration-postgres-team-metadata
Open

blackheaven wants to merge 5 commits into
developfrom
gdifolco/WPB-22956-migration-postgres-team-metadata

Conversation

@blackheaven

Copy link
Copy Markdown
Contributor

https://wearezeta.atlassian.net/browse/WPB-22956

Checklist

  • Add a new entry in an appropriate subdirectory of changelog.d
  • Read and follow the PR guidelines

@blackheaven
blackheaven requested review from a team as code owners October 22, 2026 10:56
@zebot zebot added the ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist label Oct 22, 2026
@blackheaven

Copy link
Copy Markdown
Contributor Author

Heads-up: the integration test added here (Test/Migration/Team.hs) reuses admin (the owner of tid1) for the tid2-tid4 assertions, but every createTeam mints a fresh owner and team-member listing requires membership — those assertions would 404. Fixed on the stacked branch in #5635 (this PR should be merged together with or after #5635, or the fix cherry-picked).

@blackheaven
blackheaven force-pushed the gdifolco/WPB-22956-migration-postgres-team-metadata branch from f43b28d to 49db27d Compare October 22, 2026 14:30
@blackheaven
blackheaven added this pull request to stack #5636 October 22, 2026 14:34
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22956-migration-postgres-team-metadata branch 2 times, most recently from a69b376 to d4d9a96 Compare November 4, 2026 12:45
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22956-migration-postgres-team-metadata branch from d4d9a96 to 30ee8c4 Compare November 5, 2026 12:36
@VeryMilkyJoe
VeryMilkyJoe requested a balanced review from Copilot November 6, 2026 16:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical write-ordering and deletion-state bugs can overwrite acknowledged updates, leave Cassandra and PostgreSQL inconsistent, and make interrupted deletions unrecoverable or allow deleted teams to be restored.

5 open issues
What changed in this PR

This PR adds staged migration of team metadata from Cassandra to PostgreSQL, while keeping team membership data in Cassandra.

Changes:

  • Adds PostgreSQL storage, dual-write support, and a background migration with metrics.
  • Wires storage selection and migration controls into services, Helm configuration, and documentation.
  • Adds serialization tests and integration coverage for migration and cutover.
File Description
services/​galley/​src/​Galley/​App.hs Selects the configured team-store interpreter.
services/​galley/​galley.integration.yaml Enables PostgreSQL team metadata in integration tests.
services/​brig/​brig.integration.yaml Adds the team storage setting.
services/​background-worker/​test/​Test/​Wire/​Util.hs Updates test configuration with team storage.
services/​background-worker/​test/​Test/​Wire/​BackendNotificationPusherSpec.hs Updates migration-option fixtures.
services/​background-worker/​src/​Wire/​PostgresMigrations.hs Starts team migration and registers metrics.
services/​background-worker/​src/​Wire/​Effects.hs Selects the worker's team-store interpreter.
services/​background-worker/​src/​Wire/​BackgroundWorker/​Options.hs Adds the team migration flag.
services/​background-worker/​src/​Wire/​BackgroundWorker.hs Manages team migration startup and cleanup.
services/​background-worker/​background-worker.integration.yaml Configures team storage and migration defaults.
postgres-schema.sql Records the new team table and primary key.
libs/​wire-subsystems/​wire-subsystems.cabal Exposes the new team-store modules.
libs/​wire-subsystems/​src/​Wire/​TeamStore/​Postgres.hs Implements PostgreSQL team metadata operations.
libs/​wire-subsystems/​src/​Wire/​TeamStore/​Migration.hs Copies Cassandra team metadata into PostgreSQL.
libs/​wire-subsystems/​src/​Wire/​TeamStore/​DualWrite.hs Mirrors metadata writes during migration.
libs/​wire-subsystems/​src/​Wire/​TeamStore/​Cassandra/​Queries.hs Adds the migration scan query.
libs/​wire-subsystems/​src/​Wire/​PostgresMigrationOpts.hs Adds team storage selection.
libs/​wire-subsystems/​postgres-migrations/​20261022080836-create-team.sql Creates the PostgreSQL team metadata table.
libs/​wire-api/​test/​unit/​Test/​Wire/​API/​Roundtrip/​PostgresMarshall.hs Tests team-type serialization round trips.
libs/​wire-api/​src/​Wire/​API/​Team/​SearchVisibility.hs Adds PostgreSQL search-visibility encoding.
libs/​wire-api/​src/​Wire/​API/​Team.hs Adds PostgreSQL binding and icon encoding.
libs/​wire-api/​src/​Wire/​API/​Routes/​Internal/​Galley/​TeamsIntra.hs Adds PostgreSQL team-status encoding.
integration/​test/​Test/​Migration/​Team.hs Tests staged migration and PostgreSQL cutover.
integration/​integration.cabal Registers the migration test module.
hack/​helm_vars/​wire-server/​values.yaml.gotmpl Passes team storage selection into Galley.
hack/​helm_vars/​common.yaml.gotmpl Defines the preferred team storage value.
docs/​src/​developer/​reference/​config-options.md Documents team migration settings and metrics.
charts/​wire-server/​values.yaml Adds default storage and migration settings.
charts/​wire-server/​templates/​background-worker/​configmap.yaml Renders the team migration flag.
changelog.d/​5-internal/​WPB-22956 Records the team metadata migration.

Review effort: Balanced

@blackheaven
blackheaven force-pushed the gdifolco/WPB-22956-migration-postgres-team-metadata branch from 30ee8c4 to be6b2b8 Compare November 10, 2026 07:55
…LWW convergence, insert-if-absent + writetime-guarded copy reconciliation, PendingDelete park before Cassandra delete with terminal-stamp finalize, nullable writetime columns, 5-copilot-thread fixes
@blackheaven
blackheaven force-pushed the gdifolco/WPB-22956-migration-postgres-team-metadata branch from c0ff399 to a4bf8d5 Compare November 13, 2026 11:27

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

easy-to-review ok-to-test Approved for running tests in CI, overrides not-ok-to-test if both labels exist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants