Repository navigation
WPB-22956: migrate team metadata to PostGreSQL - #5631
blackheaven wants to merge 5 commits into
Conversation
|
Heads-up: the integration test added here ( |
f43b28d to
49db27d
Compare
a69b376 to
d4d9a96
Compare
d4d9a96 to
30ee8c4
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Critical write-ordering and deletion-state bugs can overwrite acknowledged updates, leave Cassandra and PostgreSQL inconsistent, and make interrupted deletions unrecoverable or allow deleted teams to be restored.
5 open issues
Concurrent writes can leave Cassandra and PostgreSQL out of sync · New Backfill can overwrite newer dual-written team metadata · New PostgreSQL deletion can strand incomplete Cassandra cleanup · New Later PostgreSQL writes can undo terminal team deletion · New Missing timestamps are converted to fabricated 1858 dates · New
What changed in this PR
This PR adds staged migration of team metadata from Cassandra to PostgreSQL, while keeping team membership data in Cassandra.
Changes:
- Adds PostgreSQL storage, dual-write support, and a background migration with metrics.
- Wires storage selection and migration controls into services, Helm configuration, and documentation.
- Adds serialization tests and integration coverage for migration and cutover.
| File | Description |
|---|---|
| services/galley/src/Galley/App.hs | Selects the configured team-store interpreter. |
| services/galley/galley.integration.yaml | Enables PostgreSQL team metadata in integration tests. |
| services/brig/brig.integration.yaml | Adds the team storage setting. |
| services/background-worker/test/Test/Wire/Util.hs | Updates test configuration with team storage. |
| services/background-worker/test/Test/Wire/BackendNotificationPusherSpec.hs | Updates migration-option fixtures. |
| services/background-worker/src/Wire/PostgresMigrations.hs | Starts team migration and registers metrics. |
| services/background-worker/src/Wire/Effects.hs | Selects the worker's team-store interpreter. |
| services/background-worker/src/Wire/BackgroundWorker/Options.hs | Adds the team migration flag. |
| services/background-worker/src/Wire/BackgroundWorker.hs | Manages team migration startup and cleanup. |
| services/background-worker/background-worker.integration.yaml | Configures team storage and migration defaults. |
| postgres-schema.sql | Records the new team table and primary key. |
| libs/wire-subsystems/wire-subsystems.cabal | Exposes the new team-store modules. |
| libs/wire-subsystems/src/Wire/TeamStore/Postgres.hs | Implements PostgreSQL team metadata operations. |
| libs/wire-subsystems/src/Wire/TeamStore/Migration.hs | Copies Cassandra team metadata into PostgreSQL. |
| libs/wire-subsystems/src/Wire/TeamStore/DualWrite.hs | Mirrors metadata writes during migration. |
| libs/wire-subsystems/src/Wire/TeamStore/Cassandra/Queries.hs | Adds the migration scan query. |
| libs/wire-subsystems/src/Wire/PostgresMigrationOpts.hs | Adds team storage selection. |
| libs/wire-subsystems/postgres-migrations/20261022080836-create-team.sql | Creates the PostgreSQL team metadata table. |
| libs/wire-api/test/unit/Test/Wire/API/Roundtrip/PostgresMarshall.hs | Tests team-type serialization round trips. |
| libs/wire-api/src/Wire/API/Team/SearchVisibility.hs | Adds PostgreSQL search-visibility encoding. |
| libs/wire-api/src/Wire/API/Team.hs | Adds PostgreSQL binding and icon encoding. |
| libs/wire-api/src/Wire/API/Routes/Internal/Galley/TeamsIntra.hs | Adds PostgreSQL team-status encoding. |
| integration/test/Test/Migration/Team.hs | Tests staged migration and PostgreSQL cutover. |
| integration/integration.cabal | Registers the migration test module. |
| hack/helm_vars/wire-server/values.yaml.gotmpl | Passes team storage selection into Galley. |
| hack/helm_vars/common.yaml.gotmpl | Defines the preferred team storage value. |
| docs/src/developer/reference/config-options.md | Documents team migration settings and metrics. |
| charts/wire-server/values.yaml | Adds default storage and migration settings. |
| charts/wire-server/templates/background-worker/configmap.yaml | Renders the team migration flag. |
| changelog.d/5-internal/WPB-22956 | Records the team metadata migration. |
Review effort: Balanced
30ee8c4 to
be6b2b8
Compare
…/binding, writetime columns)
…LWW convergence, insert-if-absent + writetime-guarded copy reconciliation, PendingDelete park before Cassandra delete with terminal-stamp finalize, nullable writetime columns, 5-copilot-thread fixes
…gs, migration tests)
c0ff399 to
a4bf8d5
Compare


https://wearezeta.atlassian.net/browse/WPB-22956
Checklist
changelog.d