Repository navigation
WPB-28665: add PostGreSQL support for query user search - #5539
blackheaven wants to merge 11 commits into
Conversation
zebot
added
the
ok-to-test
blackheaven
changed the title
fisx
left a comment
fisx
left a comment
There was a problem hiding this comment.
This would be awesome, but the ticket doesn't really say anything about re-architecting anything? Also, shouldn't we do some load testing before we merge this? Or maybe after, before the next release?
|
@fisx as discussed during the meeting this morning, we are not ready yet for load testing, instead, we should support both for a little while, I'll rework my PR in this direction. |
e2fa56a to
7cd6d34
Compare
7cd6d34 to
6e7477f
Compare
blackheaven
changed the title
82189a8 to
260c1f1
Compare
260c1f1 to
dbb0ef2
Compare
|
Fair points on both counts — thanks @fisx. Re-architecture: agreed, the ticket doesn't call for one, and this PR isn't one — it's a straight ES→Postgres backend swap behind Load testing: yes — explicitly planned after merge, before the next release that turns Meanwhile, in dbb0ef2: the keyset pagination null-cursor restart bug is fixed (branch on the null flag before decoding, so NULL sort values no longer restart pages), the |
…fault, Postgres optional) Rebased onto develop; restores the ElasticSearch stack removed by the original PR. Keeps the UserSearchStore effect with its Postgres interpreter and adds a thin ElasticSearch adapter delegating to IndexedUserStore, so all search reads (local search, team browse, team size, search-visibility) go through one interface. In Postgres mode the ES index write path is neutralized via a no-op IndexedUserStore interpreter; brig's Postgres user store now maintains wire_user.name_normalized on insert/update. brig-index regains the ES tooling plus a new backfill-normalized-names subcommand. ES backend is deprecated; migration steps documented.
CREATE INDEX CONCURRENTLY cannot run inside a transaction; the migration runner only exempts scripts listed in nonTransactionMigrations. Move the three index statements to their own script registered there, keeping the transactional DDL (ALTER TABLE wire_user, CREATE TABLE team_search_visibility) in the original file. Fixes the brig/galley PostgresMigrationUsageError crashloop in the integration environment.
Hasql.Session.script sends a whole script as one simple-query message, and Postgres runs multiple statements of one message in a single implicit transaction - so a script containing several CREATE INDEX CONCURRENTLY statements still fails with SQLSTATE 25001 even on the non-transactional path. Split the user-search index migration into three single-statement scripts, all registered in nonTransactionMigrations.
…fast on search/user-store mismatch - keysetPredicate: branch on the cursor null-flag BEFORE decoding the sort value. A cursor for a row whose sort value was NULL stores a non-decodable placeholder; decoding it made the predicate Nothing and silently RESTARTED the next page from the beginning (duplicate rows). Null-flagged cursors now continue within the NULL-valued block by id alone; the dead 'or (col is null and false and ...)' arm is gone. - runBackfillNormalizedNames: one batched UPDATE ... FROM unnest per page instead of one UPDATE per row. - newEnv: reject searchBackend=postgres with the user store still on Cassandra at startup instead of a lazy error in the CanonicalInterpreter chain that surfaced only on the first search.
dbb0ef2 to
83f7781
Compare
https://wearezeta.atlassian.net/browse/WPB-28665
Checklist
changelog.d