Skip to content

Leading anchor: one-substitution matching, per-file detection, and an infer-design fix - #6

Merged
alex-sandercock merged 6 commits into
developmentfrom
anchor-mismatch
Nov 18, 2026
Merged

alex-sandercock merged 6 commits into
developmentfrom
anchor-mismatch

Conversation

@alex-sandercock

Copy link
Copy Markdown
Contributor

Summary

Three changes to how generate and infer-design handle the leading anchor of raw DArT deliveries, plus help and docs.

  • generate --leading-anchor-mismatches {0,1} (default 1). A read with no exact copy of the anchor is cut after its only one-substitution match, and the QC counts it as reads_leading_anchor_mismatch. Anchors under 12 bases must match exactly. 0 gives the 0.13.0 behaviour. The setting joins the cache key only when an anchor is matched with a substitution, so other caches stay valid.
  • Per-file anchor detection. --leading-anchor auto and the raw-delivery stop message take one prefix per sampled FASTQ and use the suffix all of them share, sampling 24 files instead of 8. Inline barcodes of 10 to 14 bases no longer leak into the anchor.
  • Fix: infer-design fuzzy pass. _infer_fuzzy_init stored the anchor and then cleared it, so on a raw delivery the fuzzy pass scanned uncut reads and they did not vote.
  • Help and docs. The --leading-anchor help goes from 99 words to 44 and says detection samples up to 24 FASTQs.

_infer_fuzzy_init stored the leading anchor and then cleared the state it
had stored it in, so the fuzzy pass scanned raw reads uncut. On a raw
delivery its reads did not vote. The anchor is now stored with the rest
of the state, after the clear, as the exact pass does.

The new test fails without the fix: five raw reads with an exact anchor
gave no fuzzy votes.
…g anchor

A read whose common sequence carries a sequencing error has no exact
leading anchor and was counted as reads_no_leading_anchor. With
--leading-anchor-mismatches 1 (the default), such a read is cut after the
one placement within a single substitution in the search window, and the
QC counts it as reads_leading_anchor_mismatch.

- An exact copy is tried first, so a read that has one is cut as before.
- No placement, or more than one, still counts as no anchor. Indels are
  not matched, because they would move the cut.
- An anchor shorter than 12 bases is always matched exactly.
- 0 matches every anchor exactly, as 0.13.0 did.
- The setting joins the cache key only when an anchor is matched with a
  substitution, so other caches stay valid.
- infer-design still matches the anchor exactly; its reads only vote.

Measured on the salmon 3K DArTag plate (1,104 raw FASTQs): with 0 the
table is byte for byte the 0.13.0 table after the provenance cell. With 1,
reads without an anchor fall from 13,505,332 to 2,048,910, 1.06% more
reads are assigned, and no row changes category. The numbers are in
docs/DEVELOPMENT.md.
… lengths do not leak into the anchor

detect_leading_anchor pooled the sampled reads and built the anchor from
the prefixes at the modal tag offset.  DArT inline barcodes have 10 to 14
bases, so of 8 sampled files only two or three sit at that offset, and the
anchor grew into the barcode bases those few shared.  An anchor one base
too long fits about a third of the samples, and the rest lose every read.

Each sampled file now gives one prefix, the most common among its shifted
reads, and the anchor is the longest suffix that every file's prefix
shares.  The check samples 24 files instead of 8.  A file that mixes
barcodes has its prefix shortened until 90% of its shifted reads would be
cut by it.  Files holding 90% of the shifted reads must end in the same 8
bases, and the rest are set aside, so a failed sample does not hide the
anchor.  The gates in resolve_leading_anchor are unchanged.

With one or two FASTQs the anchor may keep barcode bases.  It still cuts
those files where the amplicon starts, so auto uses it and warns not to
reuse it for other samples, and the stop message for an undeclared prefix
says the same.  The messages give the tag start as a range when barcodes
differ in length.

On the salmon plate DSalm21-6350 (common sequence CGATGGATCCAGTGC), random
subsets of 1, 2, 3 and 4 files (60 draws each) and of 8 files (150 draws)
returned the exact sequence in 0, 9, 19, 39 and 133 draws before, and in
0, 39, 57, 58 and 150 now.  Draws of 16 and of 24 files: 150 of 150 each.

The help text and the command pages still say auto reads the first FASTQ,
and the test counts in docs/DEVELOPMENT.md are not updated: both sit next
to lines the anchor-mismatch branch changes.
Per-file leading-anchor detection joins the one-substitution anchor matching. The two changes touch different functions and merge without conflicts.
The help, the stop message and the docs still said that anchor
detection reads the first FASTQ. Since the per-file detection, it
samples up to 24 FASTQs spread across the run.

- --leading-anchor help: says auto samples up to 24 FASTQs. It shrinks
  from 99 words, the longest of the 37 generate options, to 44, in the
  active voice. The search-window detail stays on the docs page.
- --leading-anchor-mismatches help: 56 words to 35, active voice.
- The raw-delivery stop message counts "sampled reads", not "the first"
  reads.
- generate.md, the DArTag tutorial and DEVELOPMENT.md match, with the
  warning for runs of one or two FASTQs. The test count is 264.
@alex-sandercock alex-sandercock self-assigned this Nov 18, 2026
Copilot AI lite review requested due to automatic review settings November 18, 2026 22:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An active cache-fingerprint issue remains: including the setting for short anchors causes needless cache rebuilds despite having no effect on matching behavior.

1 open issue
What changed in this PR

This pull request improves raw DArT delivery handling across generate and infer-design by supporting one-substitution leading-anchor matches, detecting anchors per FASTQ, and preserving anchor trimming during fuzzy inference.

Changes:

  • Adds configurable leading-anchor mismatch matching with QC and cache/provenance integration.
  • Reworks automatic detection to sample up to 24 FASTQs and account for per-file barcode variation.
  • Fixes fuzzy inference on raw reads and updates tests, help text, and documentation.
File Description
tests/​test_madcats.py Adds coverage for anchor mismatch matching, cache behavior, fuzzy inference, barcode variation, and detection edge cases.
madcats.py Implements anchor mismatch recovery, per-file detection, fuzzy-pass anchor preservation, and CLI, QC, caching, and provenance integration.
docs/​src/​tutorials/​dartag.md Documents raw-delivery anchor detection, barcode limitations, and mismatch QC behavior.
docs/​src/​development/​changelog.md Records the new anchor detection, inference, and mismatch behavior.
docs/​src/​commands/​generate.md Documents the updated leading-anchor options, defaults, and automatic detection behavior.
docs/​DEVELOPMENT.md Adds implementation details and validation measurements for the anchor-handling changes.

Review effort: Lite

Comment thread madcats.py Resolved
Copilot AI lite review requested due to automatic review settings November 18, 2026 23:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported, and the change is supported by targeted tests covering anchor matching, caching, fuzzy inference, and multi-file detection.

1 open issue

Review effort: Lite

…smatches 1

An anchor shorter than 12 bases is always matched exactly, so the
mismatch setting cannot change its read pass. It still entered the cache
key, so a default run rebuilt short-anchor caches for nothing. The key
now needs the anchor to be long enough for a substitution, as the
matcher does. Found by Copilot review on PR #6.
Copilot AI lite review requested due to automatic review settings November 18, 2026 23:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported, and the changes include targeted test coverage and documentation for the updated anchor matching, detection, caching, QC, and fuzzy-inference behavior.

No open issues 🎉

1 resolved since last review

Review effort: Lite

@alex-sandercock
alex-sandercock merged commit 4241cf2 into development Nov 18, 2026
17 checks passed
@alex-sandercock
alex-sandercock deleted the anchor-mismatch branch November 18, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants