[johnml1135]

Start here: the final-template prune (#491) stays on in grammars that have partial rules, which is what Mbugwe chaanjiyɛ needs to finish (#643). A prune that fires only where no partial word can be involved gives the same analyses as before and a fraction of the work.

On master, Morpher.HasPartialRules turns the prune off grammar-wide when any rule is partial. The change has four parts:

1. Static reachability. A partial rule leaves the analysis state at None, so what it licenses after a final template survives. FinalTemplatePartialReachability computes at load which final templates a partial word can reach (a part-of-speech closure over every rule, across strata). Any other final template is pruned as before.
2. An owed-partial flag. Word.NeedsPartial is set when a final template is unapplied right after a non-template rule, and cleared when a partial rule is unapplied. It is part of Word equality, its hash and AnalysisMergeKey. An analysis that still owes one when it reaches lexical lookup or root guessing is dropped.
3. A trace event. An analysis dropped because it still owes a partial rule leaves a TemplateAnalysisOutput trace node with the new FailureReason.OwedPartialCannotUnapply, the owing final template as source and the grammar's partial rules as Trace.FailureObject, so Try a Word shows the drop. It uses the EndUnapplyTemplate(..., object failureObj) signature that Prune affix template partials whose required slots cannot find their material, in order and at the word edges #575 adds. State tracking and the look-ahead are on only when the grammar has a final template that can be pruned, and reachability seeds only from the partial rules that synthesis marks partial (not slot or clitic rules).
4. A shape look-ahead. PartialRuleLookAhead drops an owing analysis early when its shape shows no partial rule can still be unapplied. It is on only for a stratum whose rules use nothing but copy and insert-segments actions; otherwise it says "possible".

Result on the reference grammars (this branch against master, one thread, rule-selector calls): zero changed analyses on every compared word, and Aweti, Sena and Mbugwe do 60 to 92 % less work. In the unit tests, 31 adversarial grammars give the same analyses as master at degree 1 and at the default degree.

Caveats.

• Lexical families. CheckBlocking and ChooseInflectionalStem return a fresh Word for a family member that drops the pending rule list. In a grammar with both partial rules and lexical families, master never prunes, so it returns duplicate analyses (same morphemes and shape) that this branch no longer returns. Only a duplicate is lost, never a distinct analysis: grammar cx3 returns kamdtr and kamdtrt twice each on master and once each here. PartialRulePruneTests pins that. FieldWorks never writes Family. I did not add a "no lexical families" condition.
• Work can rise. Where the prune rarely fires, state tracking splits equivalent analyses by template state: rule-selector calls on g7 rise 31 % and on cx2 25 % (degree 1). I found no cheap way to keep the split only where the prune can fire. The reference grammars do not show it.

Next: review. Landing order: the suite PR sillsdev/fieldworks-parser-conformance#42 (fixtures, rows identical on master and this branch) can land first; nothing here pins it, because master has no conformance runner until #615 lands.

Closes #643. Related: #606, #590, #553, #551, #491. #607 holds the remaining Mbugwe cost.

Results: master against this branch

hc harness with a rule-selector counter, degree 1, pure master (511fecc) dlls against this branch's SIL.Machine.Morphology.HermitCrab.dll (rebased on it). "Compared" excludes words the loader skips (letters not in the table) and words that hit the cap on a side. Work is the sum of selector calls over the compared words; allocated bytes are GC.GetTotalAllocatedBytes, the repeatable measure on this loaded machine. Both sides were rerun for every row at head 2db0929.

grammar	words	compared	with parses	changed analyses	work	allocated
Indonesian	70	70	54	0	13,290 to 13,290 (0.0 %)	1.71 to 1.72 GB (-0.2 %)
Aweti	206	194	157	0	2,918,023 to 930,220 (68.1 %)	23.8 to 8.3 GB (65.1 %)
Amharic	673	669	190	0	363,240 to 359,432 (1.0 %)	31.93 to 31.62 GB (1.0 %)
Sena (full)	6,146	5,134	2,500	0	12,064,988 to 4,767,294 (60.5 %)	178.4 to 102.6 GB (42.5 %)
Mbugwe, cap 10 M calls (sample)	173 of 1,645	168	155	0	39,652,360 to 3,183,939 (92.0 %)	550.7 to 106.1 GB (80.7 %)

The Mbugwe sample is the first 100 words of the earlier list, 13 later ones and a random 60 from index 400 on. master needs about 90 s per capped word. Five of them hit the cap on master and finish on this branch (nyaanjiyɛ, nyanjiyɛ, wāanjiyɛ, woomiyɛ, yaanja); they are not in the compared column. Before the rebase, 377 Mbugwe words were compared (369 with both answers, 345 with parses, 88.0 % less work) with 0 changes.

All 31 test grammars (g* and cx*; 62 runs, degree 1 and default, 758 words) were parsed through the hc harness on both dlls at this head: 0 differences in the distinct analysis sets. Raw multiplicities differ only on cx3 (documented) and on g13b and g21, where master itself varies between runs. Work on the test grammars (degree 1): cx11 17.6 % and cx12 23.5 % less, cx10 equal (nothing to prune), g7 30.7 % more (see caveats).

Conformance suite (rerun at this head) at origin/main plus the five new fixtures (sillsdev/fieldworks-parser-conformance#42), run with the #615 runner on master + #615 and on this branch + #615: every signature of every word dumped and diffed (with --include-pathological): 1,494 rows for the conformance run and 791 for the stress run, 0 differences. Both engines pass 152 of 153 runs and fail the same one (infix-split-root-allomorph-environment, which needs an engine fix not on master). Stress fixtures: 16 of 16 on both.

Tests

PartialRulePruneTests (77 tests) in tests/SIL.Machine.Morphology.HermitCrab.Tests:

• 31 grammars (g6, g6b, g7, g8, g10 to g21, cx1b, cx2 to cx12) with their words, each parsed at degree 1 and at the default degree. The .expected files hold the distinct allomorph sets that master returns, so a lost, extra or changed analysis fails.
• The cx grammars cover how the look-ahead reads each rule shape: cx1b a compound whose output is head, non-head, head; cx4 ModifyFromInput on a bounded part and cx9b on an unbounded one; cx5 InsertSimpleContext; cx6 an infix plus a copied part; cx7 a non-head-first compound; cx8 a head part copied twice around the non-head. cx2 makes a flagged and an unflagged word differ only in NeedsPartial. cx3 has a lexical family and a partial rule. cx10 has partial rules and no final template, cx11 a partial rule only in a template slot and cx12 only a clitic partial rule.
• Two tests pin the cx3 duplicate loss (each word once here, twice on master).
• Three work-bound tests count selector calls at degree 1: a template no partial word can reach (g8), and a shape with no room for partial material, by the contiguous test (g6) and by the existence test (g12).
• ParseWord_OwedPartialDropped_IsTraced (g6, g6b, cx1b, both degrees) checks the trace node's type, source, null output and non-empty rule list. TracksFinalTemplateState_... pins that cx10 tracks no state and that cx1b, cx11 and cx12 do. Two work-bound tests (cx11, cx12) count selector calls to show the prune stays on.
• A guard test fails if a new MorphologicalOutputAction subtype appears, because the look-ahead only knows the four that exist.

Does each test fail without the mechanism? Mutations of the branch, each run against the whole class (new mutations run on the final head):

mutation	result
a non-head copy no longer separates head copies (the cx1b fix removed)	cx1b fails at both degrees
NeedsPartial out of Word equality and hash	cx2 fails at both degrees
NeedsPartial out of AnalysisMergeKey only	cx2 fails at both degrees
wrong look-ahead variant: treat every shape as contiguous	cx1b, cx6, g12, g15 fail
ModifyFromInput let into the look-ahead (its switch-off removed)	cx9b fails at both degrees: a modify of an unbounded part widens a node without rebuilding it, so the switch-off is required
InsertSimpleContext let into the look-ahead	not detected: its analysis side is a matched constraint that GenerateShape drops, like InsertSegments, so the look-ahead stays sound (by reading)
root guessing no longer skips flagged analyses	not detected: no test grammar produces a guessed root
trace call removed from TraceOwedPartialDropped	the 3 OwedPartialDropped_IsTraced cases fail
final-template state tracked whenever the look-ahead is active, final template or not	the cx10 tracking case fails
reachability seeds from every partial rule again, slot and clitic included	the cx11 and cx12 work-bound tests and the cx11 tracking case fail
earlier rows, from before the last two commits and not rerun: ignore nodes a rule rebuilds (g13c fails); every final template unreachable (32 tests fail); a partial rule leaves the state at NonTemplate (26 fail); state never tracked (the 3 work-bound tests fail); master (the 3 work-bound tests fail in 6 of 6 runs, the results tests pass, as they must)

Two defects the first round of tests found, both fixed here: with state tracking off, a compounding rule set NonTemplate and the template rule then owed a partial nobody cleared (g21); and the look-ahead unified against unfrozen feature structures from several threads and lost parses at the default degree (g13b, g18). A review then found a third, fixed in the last two commits: a compounding rule that puts the non-head between two head parts was read as a contiguous run, so lopkmootr (cx1b) was lost.

The whole HermitCrab test project (rebased on master 511fecc): 261 passed. The class (77 tests, both degrees) passes 10 of 10 runs. master itself returns a varying number of identical analyses, and a varying morph order under truncation, in the test host, so the comparison uses distinct, ordered allomorph sets.

Not done or not run

• ./local_check.sh stops at the solution build: SIL.Machine.Tokenization.SentencePiece runs cmake and fails in this environment. The HermitCrab project and its tests build and pass, dotnet csharpier check on both is clean, and ./local_check.sh --agent-strict comment hygiene is clean after two shortenings.
• FieldWorks: FwXmlTraceManager, HcXmlTraceManager and the two trace XSLs need a case for the new reason, shaped like FieldWorks #1247 for Prune affix template partials whose required slots cannot find their material, in order and at the word edges #575; not written.
• Only cx3 has lexical families, and it pins the lost duplicate; no FieldWorks project, no PanGloss run.
• Realizational rules, and rules with several subrules or allomorphs, were read against the look-ahead but no grammar here exercises a realizational rule with partial material.
• Mbugwe is compared on a 173-word sample of 1,645 words under a 10 M call cap, because master does not finish its heavy words: chaanjiyɛ hits the cap on master and finishes on this branch (issue HermitCrab: one partial rule switches off the final-template prune for the whole grammar; Mbugwe chaanjiyɛ never finishes #643).
• A root gate, skipping repeated analysis states, a non-final-template-last check and finer reachability were measured and left out (issue HermitCrab: one partial rule switches off the final-template prune for the whole grammar; Mbugwe chaanjiyɛ never finishes #643).

🤖 Generated with Claude Code

---

This change is 

[codecov-commenter]

Codecov Report

❌ Patch coverage is 93.48958% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.58%. Comparing base (511fecc) to head (2db0929).

Files with missing lines	Patch %	Lines
...ogy.HermitCrab/FinalTemplatePartialReachability.cs	82.14%	8 Missing and 7 partials ⚠️
...hine.Morphology.HermitCrab/PartialRuleLookAhead.cs	95.21%	4 Missing and 5 partials ⚠️
...chine.Morphology.HermitCrab/AnalysisStratumRule.cs	90.90%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #644      +/-   ##
==========================================
+ Coverage   75.30%   75.58%   +0.27%     
==========================================
  Files         454      456       +2     
  Lines       38527    38892     +365     
  Branches     5339     5423      +84     
==========================================
+ Hits        29013    29395     +382     
+ Misses       8376     8360      -16     
+ Partials     1138     1137       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[claude]

FYI: F10. A flagged analysis that reaches a stratum deeper than every partial rule is still fully unapplied there. That stratum has no _material entry, so this returns true and OwesImpossiblePartial never fires. Only a non-template partial rule clears NeedsPartial, so every such analysis is dropped at synthesis anyway. Return false for a stratum with no partial rule in it or below it (lang.Strata[0] is the deepest). Unverified: I did not measure the cost.

[claude]

Major: F11. This breaks FieldWorks: a fifth parameter on the public ITraceManager.EndUnapplyTemplate means FieldWorks' FwXmlTraceManager (Src/LexText/ParserCore/FwXmlTraceManager.cs:72), which implements the four-parameter version, no longer compiles against this package. Keep the four-parameter signature and trace the drop with FailureReason.OwedPartialCannotUnapply alone, or add the failure object without changing the interface.

[claude]

Verdict: request changes.

Most important: F11 (ITraceManager.cs:49). A new fifth parameter on the public ITraceManager.EndUnapplyTemplate breaks FieldWorks' FwXmlTraceManager, which implements the four-parameter version. The rest of this round fixes F2, F3 and F4 cleanly.

Counts: Critical 1 (F11), Important 0, Low 1 open (F10).

Ran: gh pr checks 644: the comment-hygiene report and changes pass. Both builds and claude-review are pending. Locally, dotnet test with a PartialRulePruneTests filter: 77 passed, including the new trace, cx10, cx11 and cx12 tests. I read FwXmlTraceManager.cs and HcXmlTraceManager.cs in sillsdev/FieldWorks (gh api, default branch). Both use the four-parameter EndUnapplyTemplate. FwXmlTraceManager implements ITraceManager directly.

Not verified: I did not build FieldWorks against this branch. The break comes from reading its source. F10's cost is still not measured.

API and parity: public API changed. ITraceManager.EndUnapplyTemplate gains object failureObj (breaking, F11). FailureReason.OwedPartialCannotUnapply and Trace.FailureObject are new (additive). FieldWorks' trace manager handles only NonPartialRuleProhibitedAfterFinalTemplate there, so it will show the new reason as a bare template failure. No target-framework, package or sillsdev/machine.py parity change.

Findings: F1 addressed (c65a0b1). F2 addressed (2db0929). F3 addressed (2db0929). F4 addressed (2db0929). F5 addressed (f7e1e88). F6 addressed (71ce0fe). F7 addressed (f6c0b84). F8 addressed (f6c0b84). F9 addressed (f6c0b84). F10 open, unverified. F11 new.

Reviewed at 2db0929