Skip to content

chore: merge upstream adiom-data/dsync main (#387) - #23

Merged
AndrewP-GH merged 3 commits into
mainfrom
chore/merge-upstream-387
Nov 16, 2026
Merged

AndrewP-GH merged 3 commits into
mainfrom
chore/merge-upstream-387

Conversation

@AndrewP-GH

@AndrewP-GH AndrewP-GH commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Merges upstream adiom-data/dsync main (fd27d7d, "Fixes for postgres and sink for sqlbatch (adiom-data#387)") into the fork, as a merge commit like #14.

What comes from upstream

  • PostgreSQL connector: sink rewritten (replacement upserts, _id kept when it is a destination column, generated and identity columns, UUID binary values, batch close error checked, partial updates rejected with InvalidArgument); change stream sends per-transaction batches with the commit LSN and handles unchanged TOAST values; live tests behind POSTGRES_TEST_URL.
  • sqlbatch: new sink with PostgreSQL, SQL Server and Oracle dialects.
  • MongoDB connector (changes behaviour of any image built from main): unknown and empty partial updates stay a successful no-op and do not take part in deduplication; the _id is taken as the last id part.

Fork-side fix

The textual merge is clean, but one upstream test called buildIndexedBulkModels without the namespace argument the fork added. Fixed in a separate commit.

Connection strings

Read, not only searched: NewConn keeps the fork's fixed message for an unparsable connection string, and the new sqlbatch sink code has no connect path of its own. The three upstream live tests call t.Fatal(err) on the connect error; pgx prints user, database and host there and redacts only the password. They run only when POSTGRES_TEST_URL is set, so use a local throwaway database.

Notes for the open stack

#20, #21 and #22 conflict with this merge in connectors/postgres/conn.go, connectors/postgres/pglib/stream.go and connectors/postgres/pglib/stream_test.go. They are resolved on those branches, not here.

Preferred order: this PR lands first, then the stack merges main. If #20 merges this branch earlier, its diff shows the upstream lines until this PR lands.

Merge this PR with a merge commit, not a squash, so the stacked branches that merge it keep a common history.

Test plan

  • go build ./...
  • go vet ./... (only the existing "unkeyed fields" notices)
  • go test ./... (all packages pass)
  • Live PostgreSQL tests with POSTGRES_TEST_URL set
  • Stack branches re-merged and their external suites green

@AndrewP-GH
AndrewP-GH merged commit 9c0ff50 into main Nov 16, 2026
@AndrewP-GH
AndrewP-GH deleted the chore/merge-upstream-387 branch November 16, 2026 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants