Repository navigation
Clarify RDI snapshot modes and missed deletes - #4238
ZdravkoDonev-redis wants to merge 7 commits into
Conversation
🧠 Redis MemoryFound 10 related items from repository history (1 new this commit):
Memory updated at d4cdff7 |
andy-stark-redis
left a comment
There was a problem hiding this comment.
Some suggestions (mainly adding the word "mode" after always, initial, etc) and I'd prefer not to use that new shortcode unless you really feel it's necessary. Otherwise looks good, so I'll approve to avoid holding you up.
When a saved source log position expires,
when_neededcan take a new snapshot and resume collection while leaving deleted rows in Redis. These docs explain that gap and the connector differences between snapshot modes.Compare all six listed modes for Debezium sources in one shared explanation used by latest and v1.19.1. Include a shared Mermaid missed-delete timeline and a readable ordered sequence in the Cloud FAQ. Update configuration references, deployment/reset guidance, troubleshooting, multiple-source guidance, and Cloud pages. The embed shortcode enables Mermaid rendering in the parent page.
Both Architecture pages retain the delivery guarantee and add a visible disclaimer about events that cannot be captured from expired source history. Reset and target cleanup remain separate.
Document the Kubernetes restart loop after
initial_onlycompletes. Explain where to set SnowflakeclearOffset, howINITIALversusNEVERdetermines whether existing rows reload, and why the flag should be disabled after recovery. Warn that missed deletes can leave stale records even after a recovery snapshot finishes.Validation:
no_data; release mapping checked for RDI 1.19.1 and 2.0.0.initial_onlynote and Snowflake recovery guidance verified at 1280 and 390 px without horizontal overflow; the collector-property editor anchor resolves.Companion schema-description PR: https://github.com/RedisLabs/redis-data-integration/pull/2886
Context: Slack discussion.
Note
Low Risk
Documentation-only changes that clarify existing CDC behavior; no runtime or configuration schema changes in this repo.
Overview
Documents Debezium
snapshot.modefor RDI CDC sources and the stale-target risk when source log history expires before the collector resumes.Adds shared embeds (
rdi-snapshot-modes.md,rdi-snapshot-gap.md) with a mode comparison table, Debezium 3.5 connector notes, and a Mermaid missed-delete timeline. Pipeline config gains a Choose a snapshot mode section; architecture adds a note that at-least-once delivery does not cover replay of expired history.Reset / start / stop wording is corrected:
redis-di resetclears saved positions without changingsnapshot.modeor flushing Redis; recovery behavior depends on the configured mode. New troubleshooting covers unavailable source log history; multiple-source and Redis Cloud (define, FAQ, view-edit) pages are aligned.The
embed-mdshortcode now propagateshasMermaidso embedded diagrams render on parent pages. Config reference text forsnapshot.modeis expanded and linked to the new guide.Reviewed by Cursor Bugbot for commit d87ebb8. Bugbot is set up for automated code reviews on this repo. Configure here.