Skip to content

chore: merge upstream v0.30.0 - #14

Merged
AndrewP-GH merged 6 commits into
mainfrom
chore/merge-upstream-0.30.0
Nov 2, 2026
Merged

AndrewP-GH merged 6 commits into
mainfrom
chore/merge-upstream-0.30.0

Conversation

@AndrewP-GH

@AndrewP-GH AndrewP-GH commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Merges adiom-data/dsync main (v0.30.0) into the fork:

Conflict resolutions

  • coordinators/simple/coordinator.go: upstream mutex-guarded flowErr replaces fork sync.Once.
  • connectors/common/base.go: fork failFlow kept (superset of upstream NotifyError+cancel). Upstream test TestStandaloneFlowFailsAfterInitialCopyWhenStreamFails exposed a fork bug: Unimplemented StreamLSN (kafka/null/file/s3) failed the flow → failLSNTracking now logs and continues for CodeUnimplemented.
  • connectors/mongo/conn.go: upstream partial-results structure (indexedDocument, buildIndexedBulkModels, failuresFromBulkWrite) with fork changes ported in: Cosmos shard key in overwrite filter, cosmosValidTTL on inserts/updates, transient-code log levels, overwrite diagnostics. buildIndexedBulkModels takes the namespace for TTL.
  • External tests: NewConnWithClient returns (conn, error) in the fork.

Note: TestMongoConnectorSuite/TestConnectorReadResumeCDC is flaky in the full external suite — also fails on current origin/main; passes in isolation.

Test plan

  • go build ./..., go vet -composites=false -tags external ./...
  • go test ./...
  • go test -tags external ./connectors/mongo/ against local mongo 8.0 replset (only the pre-existing ResumeCDC flake)
  • Review: manual + fable + sol + astra (ship / ship / ship with upstream-only notes below)
  • Rebuild image and smoke on a test migration

Review follow-ups

  • Added TestFlowFailsWhenLSNTrackingFails (non-Unimplemented LSN error fails the flow; proven RED without failFlow).
  • Bounds check on overwrite diagnostics write-error index.
  • Upstream-only, not changed here (Cosmos is our sink, PartialWriteResults is never set by dsync): Cosmos source checkpoint may run ahead of sent events across namespaces; under PartialWriteResults ambiguous/transient write errors become ItemFailure and bypass retry — do not enable it in base.go without carrying transientServerCodes over.

Mark C and others added 6 commits October 12, 2026 10:47
)

* support partial write failures in protocol, mongodb impl
Resolutions:
- coordinator: take upstream mutex-guarded flowErr
- base.go: keep fork failFlow; treat Unimplemented StreamLSN as optional
  (kafka/null/file/s3 would otherwise fail the flow)
- mongo conn: upstream partial write results + fork shard-key overwrite
  filter, Cosmos TTL fix and write diagnostics
- external tests: NewConnWithClient returns an error in the fork
@AndrewP-GH
AndrewP-GH merged commit 86249b1 into main Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants