[RobbinBouwmeester]

On a large pool the classifier is the rescore's cost, and most of what it scores is never identified. rescore.prefilter_keep_decoys (default 0.35; 0 turns it off) removes that part before the full feature set is read.

How it works

• A logistic regression on 32 feature columns (rescore.prefilter_features replaces the list) is fitted per fold on at most 400,000 rows of the other folds: the targets at train_fdr against the decoys, refitted three times. Every PSM is scored by the model of the fold it is held out of, on the rescore's own fold keys.
• The PSMs above the decoy that completes the given fraction of the decoys go to the classifier, targets and decoys alike. The share of rows kept therefore follows the pool.
• The other per-row columns are cut to the kept rows, and the feature stream skips the dropped rows (an arrow filter per batch). The report records the cut under prefilter.
• If the model cannot be fitted (fewer than 10 confident targets in a fold, a missing or non-finite column), every PSM goes to the classifier with a warning.

Why decoys, not rows. A first version cut a fixed share of rows. Keeping 35% lost 0.27% of the peptides on diaPASEF but 0.75% on Astral, where the same 35% kept only 19% of the decoys. The loss followed the share of decoys kept on every pool, so only the decoy-share cut ships.

Measured at 0.35 (pooled rescores of stored competed tables, NN seeds 0-2 except where stated; docs/35 section 16):

pool	rows kept	peptides at 1%, all -> prefilter	wall	peak RSS
six HYE diaPASEF runs (seed 0)	36.4%	98,524 -> 98,412	14:56 -> 8:39	51.0 -> 20.6 GB
six Astral HYE runs	48.7%	116,571 -> 116,660	5:49 -> 3:33	7.2 -> 4.1 GB
one HYE Orbitrap run	39.0%	62,960 -> 63,053		3.9 -> 2.2 GB

The model sees the target/decoy labels, so the FDR was checked against nulls it does not see:

• 22 HeLa runs on a human + yeast + E. coli library: 352,838 against 352,096 human precursors per run, species-entrapment FDP 1.81% against 1.79%. At precursor level 23,734 against 23,781, FDP 1.75% either way. Rescore 6:04 against 10:35, 13.8 against 33.6 GB.
• 24 non-alkylated single cells (carbamidomethylated targets are false): pooled 169,007 against 168,249 precursors with 379 against 374 carbamidomethylated targets, 2:59 against 5:06. Per file 161,055 against 161,371 with 144 against 147. On 3 of the 24 files the prefilter stood aside, as designed.

Default. 0.35, chosen by the user after the measurements above (fd7fbeb). 0 restores the previous behaviour. CLAUDE.md, docs/35 and the CHANGELOG record it.

Under native_tda (one seed each; it is deterministic) the prefilter is also a sensitivity gain at an unchanged entrapment FDP, because the linear model fits the boundary far better without the weak bulk:

• one HYE Orbitrap run: 51,118 -> 55,461 peptides at 1%, 5:33 -> 2:04;
• six Astral runs: 96,840 -> 105,400, 8:06 -> 4:19;
• 22 HeLa runs: 197,636 -> 254,382 human precursors per run at a species-entrapment FDP of 2.01% against 2.08% (precursors 9,955 -> 14,887 at 2.17% against 2.38%), 39:02 -> 14:36, 55.8 -> 20.8 GB.

Few-decoy guard (0e9ca88): a pool with fewer decoys than rescore.min_train_decoys (100 at least) is not cut. On the smoke fixture's tag-prefilter arm, 1 decoy in 316 PSMs put the cut at that decoy and kept 96 targets. The q floor (decoys + 1) / targets then sat at 1.04%, so nothing passed. ci/smoke.sh passes again.

Still running: the E. coli diaPASEF entrapment run (three seeds) with the prefilter.

The PR also records the diaPASEF gate's validation (from #206) in docs/35 and the CHANGELOG. On the 5-min HYE runs, gate 0.845 gives -1.75% peptides at -48% wall. On the E. coli entrapment run, the FDP is 0.32-0.35% against 0.30-0.38%.

Checked: cargo fmt, clippy -D warnings, workspace tests, cargo doc -D warnings, config reference regenerated.

🤖 Generated with Claude Code