Activity

1. vzakaznikov

   changed the title [-]CAS (content_addressed MergeTree) — consolidated audit findings (tracking)[/-] [+]CAS (content addressed storage for MergeTree) — consolidated audit findings (tracking)[/+] on Jul 9, 2026
2. vzakaznikov

   changed the title [-]CAS (content addressed storage for MergeTree) — consolidated audit findings (tracking)[/-] [+]CAS — consolidated audit findings (tracking)[/+] on Jul 9, 2026
3. vzakaznikov

   changed the title [-]CAS — consolidated audit findings (tracking)[/-] [+]CAS — consolidated static analysis audit findings (tracking)[/+] on Jul 9, 2026
4. vzakaznikov

   assigned

   alsugiliazova

   on Jul 9, 2026
5. alsugiliazova

   added

   CAS

   on Aug 7, 2026
6. alsugiliazova commented on Aug 13, 2026

   

   alsugiliazova

   on Aug 13, 2026

   Member

   More actions

   CAS-021 split out: #2207

   Ambiguous conditional-write outcomes are reported as definite ownership (content equality treated as authorship; lost ACK reported as foreign occupancy).
7. k-morozov commented on Aug 18, 2026

   

   k-morozov

   on Aug 18, 2026

   Last edited by k-morozov

   More actions

   CAS-002

   The problem is architectural: adoptEvidence saves dependencies for promote to check before we've reached a durable precommit state. Throughout the whole transaction we don't have the same checks that observeAndAdmit has - it doesn't read the condemn marker, and it doesn't require our own edge to already be durable before trusting an existing blob.

   Because of this, GC can fully complete its condemn - graduate - delete cycle before our reference even becomes visible to the protocol - and that decision is irreversible, it never rolls back. As a result, the new part gets published pointing at a blob that's already been deleted, and there's nothing to recover it from.

   CAS-003

   I agree that there is a scenario where a second GC can take the lease from the first one, while the first one keeps working - including deleting. It's important to say that true atomic deletes with a leadership check can't be built here, and the only real question is how to shrink the gap between checking leadership and deleting. The lease mechanism itself exists to avoid duplicate work, and if we trust that the algorithm proving something is safe to delete is correct, then the two-leader problem is just double work - not a critical issue. It can be handled by checking gc/state before deletes (for example, in phase 11) and by discussing whether the heartbeat should stay one-directional like it is now.
   I wouldn't call the missing TTL a real problem - the lease mechanism already relies on updates to gc/state and gc/hb, and in my view that's enough.

   CAS-005

   If we want to do 2 deletes inside one cas-transaction and we stop on the second one, with the current design we can't roll back the changes from the first one - because the ownership record in the ref-log for the old manifest has already been rewritten, and there's no operation to undo that. The blobs themselves aren't deleted yet at that point — they just become candidates for GC to delete later, asynchronously, as a separate process. So the user can end up seeing a partial state: the first part is already applied, the second one isn't, even though the whole transaction reports failure.

   CAS-006

   Known limitation. Seems very close to CAS-005.

   There is no native cross-namespace atomicity (object storage has no directory rename, unlike a
   non-CAS disk where RENAME TABLE is a single atomic directory rename).
   
8. filimonov commented on Aug 21, 2026

   

   filimonov

   on Aug 21, 2026

   Member

   More actions

   All 135 findings were triaged one by one against current head, each verified in the code rather than against the report's own reasoning. Priorities were re-assigned from scratch. Verdict distribution:

   Verdict	Count	Meaning
   partly	87	the code shape is described correctly, but the attached consequence is overstated, unreachable, or inverted
   by design	21	intentional behaviour, position on record
   confirmed	17	a real problem at head
   not a bug	8	the claim is factually wrong
   already fixed	1	was real, closed before this triage
   duplicate	1	same ground as another CAS-###

   Priorities: 4 × P1, 38 × P2, 83 × P3, 10 need no action.

   The four P1 items

   ID	Substance	Tracked as
   CAS-001	The shadow/FREEZE namespace is pool-global, so UNFREEZE on one server deletes another server's frozen parts	#2212
   CAS-040	A newline in a part-file path (reachable from plain DDL: a projection named with \n) leaves an undecodable orphan manifest, and every GC round in the pool then fails forever — reproduced live	filed for the pre-release pass
   CAS-058	freezeRemote is the only one of the three clone paths without a CAS transaction, so cross-disk ATTACH PARTITION FROM cannot work	#2173
   CAS-106	The enumerated non_cas_keys skip-set rejects legal S3 disk keys, so the server dies at startup with UNKNOWN_SETTING	field report on the #2243 mitigation

   Three of the four came in independently (two as their own issues, one from a field report). CAS-040 is the one the audit surfaced first — and its stated consequence was wrong in an instructive way: the INSERT fails closed, no data is lost and no unreadable part is committed, but the failed attempt wedges reclamation for the whole pool. Worth fixing exactly because the real effect is worse than the reported one in operational terms.

   Two notes for whoever reads the findings next

   The carried from prev CAS-### provenance lines are unreliable. In every case that was checked closely, the claimed lineage did not match the content of either finding. Treat them as noise.

   The CA/... anchors are stale — the code moved into subdirectories (Backend/, Pool/, Gc/, Parts/, Formats/, Primitives/, Tools/). Locate by symbol, not by path.

   Beyond the verdicts

   The triage also produced work that was not in the audit:

   • One alarm raised and cleared: UniversePolicy::kDefault had been flipped to Authoritative, which a recorded hard constraint forbade until incarnation-keyed cursors landed. They landed two days before the flip, so the sequencing held; the constraint is now marked satisfied with the verification in place.
   • Six stale internal tracking items closed on the evidence of reading the code, and one stale claim removed.
   • Findings the audit missed, spotted while refuting its consequences: a repeated FREEZE WITH NAME silently merges two snapshots into one frozen ref; fsck never enumerates gc/, so the backstop the code promises for a stranded generation prefix does not exist; and the GC ref-cleanup still requires whole-catalog token stillness — the same over-sensitivity that was removed from the ref-writer path.

   Recurring failure modes, for calibration if the audit is re-run: a real code shape with an invented consequence; "reported nowhere" where a SQL row or CLI surface does report it; test-only fields and hooks treated as production data; invented arithmetic; cited callers with no production call site; claims about vendored third-party code that reading the source refutes; and privilege claims that ignore the GLOBAL privileges guarding SYSTEM CAS *.

   Per-finding verdicts

   ID	Verdict	Priority	Pre-release fix?	Summary
   CAS-001	confirmed (split out as #2212)	P1	yes	The shadow/FREEZE namespace is pool-global, so UNFREEZE on one server deletes another server's frozen parts.
   CAS-002	by design	P3	no	The missing probe/condemn check in adoptEvidence is the deliberate §4 manifest-trust design (8fe6331a431); the claimed data-loss window is closed at every call site, leaving an accepted trade-off with an fsck backstop.
   CAS-003	partly	P3	no	The facts hold (no wall-clock TTL, theft by differential observation, a deposed leader learns only at the round-commit CAS) but this is design: catalog and ref phases revalidate the lease before every delete, so "destructive phases are never revalidated" is false; only liveness remains.
   CAS-004	partly	P2	no	Server-side SYSTEM CAS GC REBUILD is safe on a live disk (it condemns nothing, holds the GC lease) and the two entry points' opposed read-only postures are documented design; the only real gap is offline clickhouse-disks cas-gc-rebuild having no mount-lease interlock against a live server.
   CAS-005	partly	P3	no	The core finding was closed by the TXN-ONE-PIPELINE refactor (ref published only in commit, exact rollback via CommitOutcome); the tracked residuals are no multi-ref atomicity, no rollback of a committed-ref repoint, and immediate-class DDL.
   CAS-006	partly	P3	no	Mechanism confirmed (journal-less per-ref move, a crash mid-RENAME splits a table across namespaces, no reconciler) but data loss is heavily overstated: the move is idempotently redriven, nothing is physically deleted, and only non-Atomic (deprecated Ordinary) databases reach it.
   CAS-007	confirmed	P2	no	A nested server_root_id (a/b) passes validation while decommission ownership is decided by path prefix, so SYSTEM CAS DROP POOL MEMBER on a deletes live member a/b's namespaces and objects.
   CAS-008	by design	—	—	Position holds on HEAD: the hash is selectable (blob_hash = cityhash128 | xxh3-128 | sha256, default cityhash128, fixed at pool creation) and there is no re-hashing on read — a deliberate decision, not a bug.
   CAS-009	partly	P2	no	Mechanics correct (presence-only admit, no re-hash of any re-upload or staged body) but the digest half is by design; the real corruption window narrows to a length-preserving divergence of local scratch plus the already-tracked non-atomic emulated-backend write.
   CAS-010	partly	P2	no	Real mechanism — an empty Token in Native mode passes every check and goes on the wire without If-Match — but no call site has an empty token by construction: a missing fail-closed guard, not a proven data-loss path.
   CAS-011	partly	P3	no	Bypassing CasRequestController is real and already tracked, but the fence-bypass claim is false (the fence generation is checked before every durable PUT/DELETE and is tested); 100 sleepless attempts are a live-lock slowdown, not a retry policy.
   CAS-012	partly	P3	no	The settled by-design-plus-docs position holds, but the docs half is closed only for versioning: lifecycle expiration, Object Lock/WORM and storage-class transitions are undocumented in docs/en/antalya/cas/, and a Glacier read fails with a raw S3Exception without restore-and-retry.
   CAS-013	partly	P3	no	Mechanics confirmed (admission CAS-raises the pool-global min_reader_generation to G_BUILD without writing a blob) but there is no harm today — the format floor already equals G_BUILD, so old builds cannot read the pool anyway; the defect is latent.
   CAS-014	partly	P2	no	The classifier is indeed a closed allowlist unaware of primary.cidx, .mrk4/.cmrk4 and secondary-index files — but this is not corruption: there is a 1 MiB cap and blob spill, and the real cost is buffering the whole file in memory plus a double write.
   CAS-015	partly	P2	no	Waits on single-flight, leadership and recovery have no deadline and no query cancellation, but each sits behind time-bounded I/O (90 s/16 attempts, 120 s recovery), so nothing hangs forever; what remains is non-cancellability (KILL QUERY/max_execution_time) and minutes of summed bounded work.
   CAS-016	partly	P3	no	Both facts are literally true (attempt_timeout_ms never reaches the wire; payload reads take the ordinary object-storage path) but both are documented design; "no attempt timeout, no attempt limit, no classification" is false, and the real residual is no startup check of the budget against the disk's request_timeout_ms.
   CAS-017	partly	P3	no	Latch-before-durable ordering and Closed/Faulted terminality are confirmed, but the empty-catch, no-exit-from-Removing and permanently-broken-table claims are false; the residuals are a read answering "no ref" instead of retry-later inside the latch window, and one Faulted branch without auto-remount.
   CAS-018	partly	P3	no	The headline ref-queue leadership leak is already closed by a single exit point plus tests; of six anchors only theoretical allocations in noexcept code and destructors under a memory limit hold, and "renewal fences the mount" is plainly wrong.
   CAS-019	partly	P2	no	The single-flight key really is only ns+ref with no post-wait manifest-id check, but every issued view is internally consistent (one manifest) and single-flight runs only on stale-tolerant CachedForLoad, so the effect is a one-repoint lag, not two mixed manifests.
   CAS-020	confirmed	P2	no	Confirmed: getStorageObjects loses the payload offset and has no source-side CA guard, so a server-side copy-object (MOVE out of CA, BACKUP to s3 on the same host) copies envelope bytes; but "without error" is overstated — inline files yield an empty key and the whole operation fails loudly.
   CAS-021	partly	P3	no	All six controller quotes are textually accurate on HEAD but every dangerous consequence is already neutralized; the residual is a stale in-process condemn-marker memo, which the user declined to fix by re-reading.
   CAS-022	partly	P2	no	The "no catalog row" branch of the paged sweep planner checks neither watermark, coverage nor the §6 premise, and its rationale is wrong on HEAD (the manifest body is written before the row is created) — but the window is narrow and the outcome loud, not data loss.
   CAS-023	partly	P3	no	Both behaviours are real code but the DATA-LOSS class is not: gc_enabled=false is a documented debug mode and the vanished-pool no-op success is a deliberate fail-close contract; only "every manual reclamation verb is refused" is false (FSCK and GC STOP work).
   CAS-024	not a bug	P3	no	A config with two CAS disks on one pool sharing a server_root_id never reaches a write: the second disk fails the mount protocol with ABORTED (live double-start), so there is no MOVE PARTITION TO DISK data-loss path.
   CAS-025	by design	P3	no	Mechanics correct (rebuild starts from empty priors, edge-only fold, the condemn universe is reset, incremental fold never finds those blobs again) but this is a deliberate fail-closed trade-off already tracked as the REBUILD R4 residual: retention, not loss, visible as a non-draining fsck unaccounted.
   CAS-026	by design	P3	no	Relink identity is not just pool_uuid: it is pool_uuid + server_root_id + namespace + ref + part + an exact ManifestRef proven by publish-then-confirm at the source; the absent blob probe is deliberate §4 manifest-trust and check_consistency=false matches the upstream byte path exactly.
   CAS-027	by design	P3	no	HEAD matches the settled position exactly (no intra-pool authentication; the bucket credential is the entire trust boundary); the only real residual is that this trust boundary is documented nowhere in docs/en/antalya/cas/.
   CAS-028	by design	P3	no	Blob keys really are unsalted pool-global content hashes — that is CAS dedup itself; the dedup oracle via system.cas_log needs an explicit grant, but the absent crypto-shred primitive is confirmed and gets no line in the operator documentation.
   CAS-029	partly	P3	no	The central claim is false: the versioning precondition is enforced by a mandatory behavioural mount probe (created_delete_marker), not by the GCS config check; three narrow residuals remain — the fail-open GCS check, enabling versioning after mount (LOGICAL_ERROR), and skip_access_check.
   CAS-030	partly	P3	no	Mechanics correct (the probe is skipped entirely, decommission sets the flag unconditionally) but "removes every bucket-configuration defense" is overstated — the single-attempt gate and residual proof remain; the real residual is no operator signal at all plus a self-contradictory versioning message.
   CAS-031	confirmed	P2	no	Both write-once CREATE primitives (putIfAbsentStream, promoteStaged) move If-None-Match to CompleteMultipartUpload for large bodies while both probes exercise only the small single-operation path, so the battery certifies the wrong path; exploitable only on third-party S3-compatible stores.
   CAS-032	partly	P2	no	Code shape confirmed — pool identity is never tied to endpoint or bucket — but harm requires an operator error (writing into a CRR destination, bidirectional replication over the prefix) that violates the implied but undocumented "the prefix belongs to CAS alone" requirement.
   CAS-033	by design	P2	no	The gate really is pool-wide and unbounded — a deliberate fail-closed choice — but "no operator signal" is factually wrong (a WARNING with reasons, a ProfileEvent, suppressed in phase metrics, pending_reclaim/wedged_namespace_count); granularity is already tracked.
   CAS-034	partly	P2	no	Code shapes confirmed (5000 ref objects per round, one 1000-key janitor page) but the consequence is deferred reclamation and erasure latency, not data loss; the erasure SLA is settled by the author and the budget-vs-creation-rate arithmetic is real and already tracked.
   CAS-035	confirmed	P2	no	Confirmed throughout, and the memory peak is worse than described: a GC round fully lists cas/ns/stream/ holding every key in memory and materializes the entire new edge run as one in-memory string (pool-wide at the default gc_shards=1).
   CAS-036	partly	P3	no	Of three claimed mechanics one and a half survive: the control-object body is materialized before the object_cap check and JSON key dedup stays Θ(k²); the attacker-declared read buffer and pre-check zstd resize are factually wrong, the "attacker" is the fully trusted bucket-credential holder, and every refusal is loud.
   CAS-037	partly	P3	no	The unchecked readIntText wrapper is a real shape with two real residuals (std::stoull accepts -1 in three GC-key parses, one yielding max_gen + 1 == 0; location.offset + location.length can wrap and collapse the read window to EOF), but the central "wrapping defeats every decoder range gate" thesis is untenable.
   CAS-038	partly	P3	no	Five of seven sub-claims either do not match HEAD or have a safe default; only the optional oc in the GC outcome log and the missing write-side line-cap check on gc/state remain, and both are cosmetic.
   CAS-039	partly	P3	no	The shape is right (gc_shards has no upper bound anywhere and a local XML value is silently superseded by the pool value) but the consequences are a loud fail-closed allocation refusal and a missing warning, not corruption; the "no comparison" claim is false — a durable-pair mismatch is caught and throws.
   CAS-040	confirmed	P1	yes	Real and reachable through ordinary DDL (a projection with \n in its name), but the consequence is misstated: no unreadable part is committed (INSERT fails fail-closed, no data is lost) — instead an orphaned manifest wedges every GC round pool-wide forever.
   CAS-041	partly	P2	no	Mechanics (digest = canonical re-encode) match HEAD exactly, but the claimed CORRUPTED_DATA on a foreign field is unreachable today (the version gate fires earlier and louder); the residuals are a dead Tolerant policy for this format plus a measured 27-63% of decode time and two extra payload copies.
   CAS-042	by design	P2	no	Code shape correct (one global generation as the min-reader, changePoints unread at decode) but this is the deliberate pre-release recreate-only policy; the silent-field-erasure and Roster consequences are unreachable.
   CAS-043	partly	P3	no	The narrow catch is confirmed (CORRUPTED_DATA only, while the version gate and the critical key raise UNKNOWN_FORMAT_VERSION) but a generation skew inside one pool is impossible today — relink is offered only within one mounted pool — so only a one-line hardening remains.
   CAS-044	confirmed	P2	no	The aggregate 16 MiB inline-data limit per manifest is checked only in stageManifest and has no reclassification path to a blob, so INSERT/merge fails loudly and reproducibly — and it is reachable more widely than the finding describes.
   CAS-045	confirmed	P2	no	The part-folder cache entry weight is always 256 bytes because both producers hardcode Resolved::manifest_size to zero, making the part_folder_cache_bytes byte budget and the oversized-bypass threshold inoperative.
   CAS-046	confirmed	P2	no	All code descriptions hold (local scratch = the part's full bytes, no reservation, accounting, quota or limit, deleted only at transaction end, no startup cleanup) but this is an already twice-tracked DESIRABLE class, not a new defect: failures are loud (ENOSPC), with no silent corruption.
   CAS-047	by design	P3	no	Code shape correct (one process-wide pool, 16 threads, queue_size == max_threads, blocking enqueue) but the attached consequence is false: blocking on enqueue is ordinary backpressure, not serialization or deadlock; the size is a server setting and the tuning is already tracked.
   CAS-048	partly	P3	no	Shape confirmed — the empty covering part is published under DataPartsLock — but the path is rare (only DROP/REPLACE PARTITION), the work small and bounded, the failure loud, and an ordinary object-storage disk already holds the same lock while writing a part.
   CAS-049	partly	P2	no	The serialization is confirmed and mostly deliberate; the real residual is the absence of cooperative cancellation (a GC round and FSCK cannot be interrupted, SQL FSCK ignores KILL QUERY and passes no deadline), while "shutdown waits for FSCK" is wrong and "unbounded scans" is imprecise — a round is work-budgeted.
   CAS-050	partly	P2	no	The data race on thread objects in CasGcScheduler::stop is confirmed (join outside mutex, reachable via SYSTEM CAS DROP POOL MEMBER concurrent with GC STOP/shutdown), but the "joinable-but-dead scheduler that reports itself running" half is practically unreproducible and harmless.
   CAS-051	partly	P2	no	The pending_snapshot_publishes counter leak on dispatch failure was fixed back in 829ad698ef6, so the forever-wait in quiesceRefTablesForRemount/dropNamespaceImpl is unreachable; only the second half holds — background publish fan-out is not bounded pool-wide (one thread per namespace).
   CAS-052	not a bug	—	—	Code shape correct but the consequence is unreachable: Pool always lives under a shared_ptr, all synchronous calls come from PoolPtr owners, and the detached thread pins the pool with a copy of self.
   CAS-053	partly	P3	no	The single call site and the unguarded subtraction hold, but "once per namespace and never again" is imprecise (the pass runs on every cold admission) and the effects are only extra recovery I/O and a soft rather than hard memory ceiling; correctness is unaffected.
   CAS-054	partly	P3	no	The main charge (O(R) re-encode in admits) is stale — closed back in July; the snapshot every 256 transactions is a by-design background checkpoint; only a debug/sanitizer assert restoring O(K·N) on replay remains.
   CAS-055	confirmed	P2	no	Confirmed: the carry-forward branch of createHardLink does a ForceFresh resolve per file, which at the default part_folder_validate = always means a mandatory manifest HEAD per file; "full view rebuild" is overstated (the decode comes from cache) and the fix is transaction-level memoization as in unlinkFile.
   CAS-056	partly	P2	no	Real shape — a standalone write into a committed part does pay a second (draft) manifest PUT plus one adopt event per carried blob list — but "two full manifest encodes" and "inside the CAS retry closure" are inaccurate; correctness is unaffected.
   CAS-057	not a bug	P3	no	The LOGICAL_ERROR on a non-staged moveFile/replaceFile is confirmed and preserves the previously accepted fail-loud-stub position; the "fresh evidence" is false — DeleteBitmapFileOps::writeBitmapToStorage has no production caller.
   CAS-058	confirmed	P1	yes	freezeRemote really is the only one of three clone paths without a CAS transaction and cross-disk ATTACH PARTITION FROM fails on the very first part — issue #2173, already confirmed and reproduced on HEAD with a planned pre-release fix; only the REPLACE PARTITION FROM trigger and the silent-partial-state hint are wrong (the failure is loud and the tmp part is cleaned up).
   CAS-059	by design	P3	no	All code descriptions hold (DiskEncrypted takes any delegate, overrides neither isContentAddressed nor supportsAtomicFileWrites, use_fake_transaction defaults to true) but CAS plus encryption is a settled out-of-scope position and the combination fails loudly with NOT_IMPLEMENTED at the first part write; only a config fail-fast gate is missing.
   CAS-060	by design	P3	no	Code shape correct (a random IV per rewrite plus CAS hashing whatever it is handed means no dedup at all) but CAS × encryption is an accepted out-of-scope position; there is no silent corruption and the combination is never actually wired (DiskEncrypted does not forward isContentAddressed).
   CAS-061	partly	P2	no	Core confirmed (the only rebuild verb is gc/state; the catalog and _ckpt fail closed with no recovery path; all CA tools open via _pool_meta) but it is already tracked, the mount-lease part is wrong (cas-drop-member exists), and the absence of migration tooling is a deliberate pre-release decision.
   CAS-062	partly	P3	no	The SQL FSCK path is indeed counts-only with no deadline and no scope, but that is already tracked as CAS-049; "no repair path anywhere" is wrong and excluding the meta/body counters from clean() is by design — the real residual is that those two counters are rendered on no surface at all.
   CAS-063	partly	P3	no	The "drop namespaces then release the slot" ordering and the /mount filter in listMounts hold, but "a re-run cannot repair it" is factually wrong (a resume path exists in code and is pinned by tests); only the owner-only slot's invisibility in system.content_addressed_mounts remains.
   CAS-064	by design	P3	no	The absence of a fuzzer and property-based tests is factual but a settled position (S3 is trusted, decoders fail closed); the real residual is three live format classes not registered in the shared battery — a drift risk, not a coverage hole.
   CAS-065	partly	P2	no	The central claim is false — the native If-None-Match/If-Match path runs in six CAS-over-S3 CI lanes (RustFS, Mode::Native) and passes a fail-closed capability battery on every writable pool open; only a repeatable GCS-generation-dialect lane and a native row in the contract suite are missing, and both are already filed.
   CAS-066	by design	P2	no	Code shape confirmed (the mode is chosen by storage type, there is no knob, the log is at INFO) — a deliberate design decision with recorded rationale; the "two servers on one local path" risk is already tracked as doc debt, and the read-only-mount sub-claim has no consequence.
   CAS-067	partly	P3	no	The first half (coarse mtime granularity yields a valid stale token) was closed by an mtime-quantum disambiguator on 2026-07-18 and is covered by a test; the second (clock skew breaks token-state expiry) is real but only a memory leak in emulated mode with no correctness impact.
   CAS-068	by design	P3	no	The shape is real and deliberate (the divergence from four siblings is recorded in code and in commit 4f4f93c6bc6) but the claimed trigger (promoteStaged/resurrect → NOT_IMPLEMENTED) is unreachable in this lane and the outcome is a loud fail-closed wedge, not corruption; only lost diagnostics remain.
   CAS-069	partly	P3	no	The empty catches exist and do lose the cause, but "indistinguishable from corruption → full rebuild → permanent orphans per CAS-025" does not follow: rebuild is manual only (SYSTEM CAS GC REBUILD), and on the undecodable branch it does not drop holds but finds the seal by listing and refuses with CORRUPTED_DATA.
   CAS-070	partly	P2	no	Only one of three claimed mechanisms is real — remount_running is latched before the thread is created, so a failed spawn disables self-remount forever; "lost wakeup" and "no handler in the thread body" do not hold on HEAD.
   CAS-071	by design	P3	no	The prior verdict (CAS-090: by design, latent) still holds — mount_keeper changes only under Pool::remount_mutex and its only rival is cut off by a config gate in code; the claimed fence/deadline gap against mayMutate() is factually wrong, leaving one cosmetic residual (pool_uuid published outside pointer_mutex).
   CAS-072	partly	P3	no	Real shape (a single precommit slot, overwritten without a check) but a second precommitAdd on one PartWriteTxn is unreachable on every working path — a latent invariant with no executable guard.
   CAS-073	by design	P3	no	All three shapes are correct but the consequence is invented: the marker is by definition not the deletion authority — an exact-token delete using the token from the retired row does the deleting, and resurrect rotates the incarnation tag, so a previous incarnation's marker authorizes nothing.
   CAS-074	partly	P2	no	Code shape correct and long documented by the code itself (monotonic cursor, single-use hand-off) and the leak is tracked in three places, but an untracked residual is confirmed: the promised fsck backstop does not exist — runFsck never lists gc/, so an orphaned generation prefix is both unreclaimed and invisible.
   CAS-075	partly	P3	no	The "durable body then .meta" ordering and the absence of any blobs/ LIST outside fsck hold, but "reclaimed by no sweep at all and invisible" is half right: the class is already tracked as the by-design R4 residual and an orphaned body does show as unaccounted on all three fsck surfaces.
   CAS-076	not a bug	—	—	Code shape correct (the fold seal is written before the single gc/state CAS) but the consequence is false on HEAD: the wholesale gc/gen/<g>/ prefix prune is the one regular collector of all attempts, including unaccepted ones, and it is pinned by tests.
   CAS-077	partly	P3	no	The retention is real: only the dead node's own mount floor grants deletion rights, and an unreclaimed slot's min_active/writer_epoch stop moving — but the finding names the mechanism wrongly (the lease does not vanish when a node is lost), "forever" holds only for a slot with no restart, successor or decommission, and the conservatism is deliberate control #9.
   CAS-078	confirmed	P3	no	The namespace janitor cursor really does rewind on any LIST error and it is pinned by a test, but this is only delayed reclamation (loud, and counted by fsck), not data loss.
   CAS-079	confirmed	P2	no	GC ref-cleanup revalidation really does require token stillness of the whole pool-global catalog and bails out of the entire phase on failure — a live remnant of the class already removed for the ref writer in commit 684161d.
   CAS-080	not a bug	—	—	The finding's key claim (the publish driver is called only from the write path) is false on HEAD: the publish trigger also sits on every read path, and after recovery the tail counters are above the threshold again, so the very first read or write drives _ckpt to a new snapshot.
   CAS-081	partly	P2	no	Code shape correct and holding the staging object on abort is deliberate design (it is the resurrect source for promote), but the attributed consequences are partly false: SYSTEM CAS DROP POOL MEMBER drains a dead member and two disks with one srid are impossible thanks to the fail-closed mount claim; the residuals are no periodic cleanup during a mount and no introspection.
   CAS-082	partly	P3	no	Code shape correct (CAS keeps no multipart accounting of its own, _probe/ is excluded from the bootstrap scan) but the consequences are overstated: ordinary paths abort in the upstream buffer and probe debris arises only on a hard kill — a cosmetic, cost-only residual.
   CAS-083	by design	—	—	Code shape confirmed on HEAD (hardlink/repoint carry the same BlobRef) but that is CAS dedup itself and exactly what ordinary MergeTree does on a lightweight DELETE; the "won't fix" position stands.
   CAS-084	partly	P3	no	Real mechanism — CA deletions bypass the caching object storage, so file-cache entries for GC-reclaimed blobs are not invalidated — but "capacity held forever" and "deleted content is readable" are overstated: the cache is necessarily bounded and LRU-evicted, and content-addressed keys mean a stale entry physically cannot return foreign or wrong bytes.
   CAS-085	partly	P2	no	Core correct and reachable: with always_use_copy_instead_of_hardlinks=1 mutations and single-disk partition clones on a CA table fail with a loud NOT_IMPLEMENTED and the setting is never rejected; but FREEZE, the BACKUP/RESTORE clone and implicit zero-copy do not reach this path, and the refusal is fail-closed with no data corruption.
   CAS-086	partly	P3	no	All five shapes exist and nearly all are the deliberate, tested CA contract (a part dir is one atomic ref); the claimed consequences are either unreachable (getHardlinkCount — zero-copy is off for CAS) or already closed (unlinkFile no longer stays silent), leaving one narrow residual: a repeated FREEZE WITH NAME silently merges the shadow ref instead of raising DIRECTORY_ALREADY_EXISTS.
   CAS-087	by design	P3	no	Both parsing orders are real but this is an accepted, test-pinned limitation of a purely string-based parser, reachable only for a deprecated Ordinary database: in the Atomic layout the uuid anchor always wins and an unclassified part dir as a table-level file is unreachable; only a missing fail-closed refusal on a reserved name remains.
   CAS-088	partly	P3	no	The unconditional resurrect is deliberate design with three structural justifications; "fence-unchecked" is wrong (the check exists since 1fe585ea078 and is test-pinned) and "returns a token it never wrote" was closed today by 9b887ac8886; only stale putOverwrite comments and test names remain.
   CAS-089	by design	P3	no	Code shape correct (offset from pool meta, the envelope decoder only calls INSPECT) but an envelope-length divergence is structurally unreachable, identity is not stored in the envelope (only a diagnostic ref is truncated), and the version is gated pool-wide via min_reader_generation; the INTEGRITY consequence does not hold and the rest is a format-freeze decision.
   CAS-090	partly	P3	no	All four code shapes are real but none of the consequences is silent corruption: SSE-C breaks only the optional staging_backend=s3, which has a fail-closed mount probe falling back to local staging; the MAC/digest half duplicates settled CAS-008, plaintext manifest metadata and re-key impossibility are already tracked, and the plaintext-small-file-bodies claim is wrong.
   CAS-091	partly	P3	no	Real validator divergence — Layout::checkNamespace is the only one admitting ./.. segments, and the emulated-mode filesystem escape over local object storage is real — but there is no production reachability: every live namespace is prefixed by an already-validated server_root_id, UUIDs and names are escaped, and a FREEZE backup name goes through escapeForFileName.
   CAS-092	partly	P2	no	Only one of three sub-items is real — the non-portable CLOCK_BOOTTIME (Darwin build); "different clocks for fence and request" is refuted, and wall-clock in the decommission mint is by design and already documented.
   CAS-093	already fixed	—	—	Code shape correct (a nested scratch transaction under the part's tmp path plus a no-op removeRecursive) but the leak was closed by a targeted dropRefIfPresent in moveDirectory (B183, 2027de8346a) and covered by a separate gtest (3a32e0259f8), so "the temp ref and its blobs survive" is false on HEAD.
   CAS-094	partly	P3	no	Code shape correct — a refusal on a lost CAS does leave a full set of run objects and a valid fold seal — but "the next round will pick it up" and "leaks forever" are wrong; what remains is a false comment, unreported residue and attempt numbering that is not lease-derived.
   CAS-095	partly	P3	no	The silent preview_deletes=0 on a missing gc/state (and, more sharply, on a vanished adopted seal) is confirmed, but "unreadable gc/state" and "one bad run object" actually raise loud CORRUPTED_DATA rather than staying silent; this is a read-only diagnostic tool.
   CAS-096	partly	P3	no	Shape correct — REBUILD does not surface dropped plan-row counters — but "reports only performed=1" is false (11 fields, 13 columns) and droppedHolds is not counted in production at all: it is a test adapter.
   CAS-097	partly	P3	no	Format coverage and the loss of hold in the fold-seal dump are confirmed (10 of 17 live formats), but "raw keys are listed by nobody" and "the wedge is unnameable" are wrong, and a _files/ mis-decode raises a loud error rather than substituting data.
   CAS-098	partly	P2	no	Four of five claims hold on HEAD (0 means both "never led" and "just succeeded", is_leader=0 does not separate a stopped GC from a follower, pending_reclaim does not clear spared/replaced); the fifth (a clamp counter "firing every round") is stale in code and survives only in the ProfileEvent description text.
   CAS-099	partly	P3	no	The headline ("the only shutdown is stop-and-join", no ordered node-removal path) is factually wrong: ~Pool drains the ref lanes and writes a certified farewell, and SYSTEM CAS FORGET/GC STOP are the quiesce verbs; only a minor residual remains — the durable gc/state lease is not released on a clean stop, and it is already tracked.
   CAS-100	partly	P3	no	All four code shapes hold but the consequences are overstated: the skipped check families are declared cost decisions with compensating mechanisms (GC itself fail-closed verifies run checksums, stale_edge has a soak gate, --namespace warns in the help); the only real residual is that the report has no machine-readable coverage flag, so "0 because unchecked" is indistinguishable from "0 because clean".
   CAS-101	partly	P3	no	Confirmed that the round counters (objects_deleted/absent/replaced/spared) are computed from a budget-truncated outcome log and that GcFoldBegin/GcFoldEnd carry the previous round's number — but the deletions themselves are counted exactly (entries_redeleted) and round = 0 on phase rows is documented design (correlate by round_id), not a defect.
   CAS-102	partly	P3	no	Both facts hold (the CasNs::Server row is unreachable, server-root keys classify as Gc) but this is a deliberate, documented and test-pinned state; "mount/lease volume cannot be measured" is wrong — the residual is cosmetic: 11 always-zero counters in system.events.
   CAS-103	partly	P3	no	Code shape correct — CASBlobBodyPutAvoided/CASBlobDeduplicationCacheHit are incremented before observeAndAdmit and the condemned branch still uploads the body, so the request counter overcounts on a rare race — but nobody counts bytes and the branch is fail-closed for every code except ABORTED.
   CAS-104	partly	P3	no	Code shape correct (one mutex-guarded dispatcher per pool, the sink enabled by shipping config) but the claimed "every resolve and every manifest read serializes on one mutex" does not hold; the single residual is that with the <cas_log> section removed the sink is still installed, so every event is built and discarded.
   CAS-105	partly	P3	no	The missing config surface for the lease and request budget and some pool caps is confirmed, but "the validator is dead code that never fires" is wrong: validateCasRequestBudget runs on every writable mount and is covered by tests; ref_table_cache_bytes duplicates the already-tracked CAS-053.
   CAS-106	confirmed	P1	yes	Core confirmed and already backed by a field report: the enumerated non_cas_keys skip-set rejects almost every legal S3 disk key and the server fails to start with UNKNOWN_SETTING; the side claims (18 entries, unchecked numeric ranges) are inaccurate.
   CAS-107	confirmed	P2	no	Both halves hold on HEAD: ContentAddressedMetadataStorage does not override applyNewSettings, so no CAS setting is applied on SYSTEM RELOAD CONFIG and nothing is logged about it; removing a disk from the config yields only a generic warning while the mount lease keeps renewing until restart.
   CAS-108	partly	P3	no	The code shapes are real (a test-only backend links into dbms, ShardReducer/manifestCleanupShard have no production callers, a test seam drops a gate) but the racily-readable global hook is unreachable in production, "the gc_shards > 1 path has no callers at all" is wrong, and both real residuals are already tracked.
   CAS-109	partly	P3	no	The missing deterministic crash-at-step-N harness half is correct and already tracked; the "validation of 29 settings is practically uncovered" half is factually wrong — the coverage exists, it is just spread across three files.
   CAS-110	by design	P3	no	Code shape correct — allow_stale really is inert at both layers — but the consequence "callers get the strict path and lose the cheap best-effort one" is wrong: no strict/non-strict split exists, resolve is always fresh by contract, and only one stale comment remains.
   CAS-111	confirmed	P2	no	The 64 MiB ceiling on an encoded snapshot or removal transaction does impose a hard per-table ref limit (~0.6-0.9 M) with growing writes refused, but it is a loud fail-closed refusal before any object is created, and it is already tracked.
   CAS-112	partly	P2	no	Every committed positive ref chunk really does a fresh uncached GET plus a full decode of the pool-global cas/ref_catalog with a linear scan (≥2 GETs per part publish), but "deep copy and re-encode on every mutation" applies to catalog lifecycle mutations, not to ref append.
   CAS-113	partly	P3	no	The code shape holds only for the manifest (kMaxManifestEncodedBytes checked after encodePartManifest) and the consequence is overstated — pre-encode caps on entry count and inline bytes come first; the ref-lane claim is plainly wrong, since its admission budget is computed before encoding, incrementally and byte-exact.
   CAS-114	confirmed	P2	no	Confirmed: recovery closes every skipped writer epoch with its own durable pair (seal PUT + _ckpt CAS + an exact GET), the epoch is pool-wide and minted per mount while the seal chain is per namespace, so first touch of a long-idle table costs O(mount count) sequential writes with no upper bound; the per-epoch slot occupancy is forced by the protocol, so the step's cost must shrink rather than the step disappear.
   CAS-115	partly	P3	no	The DedupWeight half is right (a constant 64 against a real ~176 bytes per entry, plus NO_MAX_COUNT); the PartManifestWeight half is overstated — the dominant inline_bytes.size() term is accounted honestly and there is a separate 16384-entry limit.
   CAS-116	partly	P3	no	Shape confirmed — staging is a vector with no path index, every mutation rescans it, and moveDirectory scans the destination nestedly — but the consequence is overstated: a few million short-string comparisons against one PUT per file on the same path.
   CAS-117	partly	P3	no	Mechanics correct — every blob file really carries a body, a .meta sibling and a padded 256-byte envelope — but "wide part of small files" is wrong: small metadata is inlined into the manifest, and the packing/inline-by-size class is already tracked with measurements.
   CAS-118	partly	P3	no	The code shapes are real (a mandatory HEAD before probing the decode cache, a resolve before probing the view cache, every entry point parsing, routing and resolving itself) but the headline "no cache serves a read without a network round trip" is factually wrong — resolveRef is purely in-memory and a warm CachedForLoad hit never reaches readManifestShared; the real residual is duplicated parse+route+getView per file open.
   CAS-119	partly	P3	no	The single-attempt client for conditional writes and the 16 attempts are confirmed deliberate design with a fail-closed mount gate, but "fixed backoff" is wrong (capped exponential 200 ms→5 s), "SDK jittered" is wrong (default jitter_factor = 0) and "the client pushes harder than throttling asked" is inverted (16 CAS attempts against the SDK's default 500); the residuals are missing jitter on upload fan-out and the lane's exclusion from disk-wide pacing.
   CAS-120	partly	P3	no	Code shape confirmed (a CA destination always goes through a byte-by-byte sequential readFile/writeFile loop, with no relink fast path for a local move) but "re-writes every byte" is wrong — within one pool the publish is fully absorbed by the dedup gate, so the real cost is one-sided: a full source GET plus restaging.
   CAS-121	partly	P3	no	The BACKUP refusal via temporary hardlinks is confirmed but is a deliberate fail-closed gate reachable only for a deprecated Ordinary database; the second half ("every BACKUP re-reads all bytes because areBlobPathsRandom()==false") is factually wrong — part files have a precalculated checksum and it is verified before the FromRemotePath path.
   CAS-122	duplicate of CAS-042	P3	no	Both quotes hold on HEAD, but "an additive field reads as CORRUPTED_DATA" is unreachable (the checkCompatibility gate on v fires before the body and yields UNKNOWN_FORMAT_VERSION) and the producerless ! key is a deliberately dormant compat plane of the pre-release recreate-only policy — exactly the ground already adjudicated as CAS-042.
   CAS-123	partly	P3	no	Byte accounting really is limited to blob bodies (plus one off-topic namespace_janitor_pending_bytes counter) and there is no reclaim-forecast surface; previewDeletes does mix physical and logical size in one column, but "counts entries it will not delete" is overstated — they are flagged in the reason column and there is no double counting.
   CAS-124	partly	P3	no	Real shape (empty content under cityhash128 yields a zero digest that collides with BlobRef{}) but the consequence is confined to one classification line in fsck and does not affect data.
   CAS-125	not a bug	—	—	False claim: XXH3_128bits_reset checks for NULL and returns XXH_ERROR, there is no dereference, and the valid() guard is live and working.
   CAS-126	by design	—	—	The asymmetry is real but it is not a pre-check: the callback gates a durable effect before sink->finalize(), and the local path has no such effect (only a private scratch file) whose real write is fenced anyway.
   CAS-127	partly	P3	no	The code shapes are real but the consequences are overstated: "three copies per byte" holds only for xxh3/sha256 (the default cityhash128 does two) and five substring searches per request are negligible beside the network RTT; the only open residual is a byte-wise readLine allocating per record on decode paths.
   CAS-128	partly	P3	no	A real shape of the same class as the already-fixed f7539af045c (the Inline branches of createHardLink/moveFile create no destination PartWriteTxn), but there are no production callers and the outcome is a loud fail-closed LOGICAL_ERROR before any publish, not data loss.
   CAS-129	not a bug	—	—	The finding's key facts are wrong on HEAD: requireAlive (with its epoch check) is called from seven call sites, not one, and the durable append itself fails closed on admitted_fence_generation/checkFenceOrThrow plus an in-band EpochSeal; the "90 s window" misreads operation_deadline_ms.
   CAS-130	partly	P3	no	All three code shapes are real but none of the attributed consequences occurs: the mount lease seq is nowhere authoritative (liveness is judged by token stability) and writer_epoch = 0 requires a corrupted cas_epoch object and fails loudly at the first ref-transaction encode; two cheap hardening residuals remain.
   CAS-131	partly	P3	no	The first claim (timestamp, thread_id and query_id taken on the draining thread) is true on HEAD and was untracked — a real observability defect; skipping RefResolve on a warm cache hit is a documented volume decision (the hit shows in CASPartFolderViewHits), and the double-count/mixed-units claim about view-cache counters is factually wrong.
   CAS-132	not a bug	—	—	An unprivileged SQL user never reaches these messages (the only throw site is the mount path, every SYSTEM CAS * verb is behind its own GLOBAL privilege, and inline disk() requires the user to bring their own bucket and credentials); literal object paths are a publicly documented layout, and system.cas_mounts exposes no more than system.clusters/system.disks.
   CAS-133	partly	P3	no	Real shape (a LIST failure collapses into a synthetic row) but the consequence is inverted: the lifecycle column is exactly what distinguishes a live pool from one not started or vanished; only a minor "LIST failed" versus "live pool with no slots" ambiguity remains.
   CAS-134	partly	P3	no	The code shape holds only for the "no disk passed" branch (which takes the policy's first CA disk), but "silently ... with no diagnostic" is factually wrong — a pool mismatch after reservation is logged at LOG_INFO with both UUIDs, and it degrades to an ordinary byte fetch onto the CA disk with no loss of correctness.
   CAS-135	partly	P3	no	The code shapes are confirmed but the consequence is inverted: emu_mutex is the very mechanism that emulates conditional operations (without it exact-token semantics break) and serializing resurrects deliberately caps peak memory at a maximum rather than a sum; EmulatedSingleProcess is documented as tests and local development only, and the resurrect half is already tracked.
9. filimonov

   added 3 commits that reference this issue on Aug 21, 2026

   ca: 2031-triage — skeleton document for per-finding adjudication of i…

   ...

   e811b3d

   cas: one undecodable manifest no longer stops reclamation for the who…

   ...

   2649bce

   cas: give the CAS disk settings their own `cas_` config-key namespace

   ...

   917600b
10. alsugiliazova commented on Aug 31, 2026

    

    alsugiliazova

    on Aug 31, 2026

    Member

    More actions

    Issue body was updated with new audit rerun after the merge of #2159.

    👍React with 👍1Reacted by filimonov
11. filimonov

    added a commit that references this issue on Oct 7, 2026

    docs(cas): Backlog.md, real creation dates, GitHub CAS reconciliation…

    ...

    66087be
12. filimonov commented on Oct 14, 2026

    

    filimonov

    on Oct 14, 2026

    Last edited by filimonov

    Member

    More actions

    CAS-007 Nested server_root_id (a/b) - fix in #2566
13. filimonov commented on Oct 14, 2026

    

    filimonov

    on Oct 14, 2026

    Last edited by filimonov

    Member

    More actions

    CAS-022 Orphan-manifest sweep still nominates a manifest whose namespace has no catalog row - fix in a923533