Skip to content

[Test] Check GLM-5.3 end-to-end logits alongside loss - #2139

Closed
xs1997zju wants to merge 2 commits into
feat/glm53flash-f6-text-moefrom
feat/glm53flash-f7-logits-parity
Closed

xs1997zju wants to merge 2 commits into
feat/glm53flash-f6-text-moefrom
feat/glm53flash-f7-logits-parity

Conversation

@xs1997zju

@xs1997zju xs1997zju commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Stack created with GitHub Stacks CLI • Give Feedback 💬

Stack #2112 (bottom to top):

  1. [Feature] Add GLM-5.3-Flash F0: 25B cropped reference checkpoint builder #2105 — F0: cropped reference checkpoint
  2. [Feature] Add GLM-5.3-Flash F3: Kimi Delta Attention (KDA) #2106 — F3: KDA
  3. [Feature] Add GLM-5.3-Flash F4: mHC four-stream residual #2107 — F4: mHC
  4. [Feature] Add GLM-5.3-Flash F5: NoPE DSA + KPool indexer + clamped SwiGLU #2108 — F5: NoPE DSA
  5. [Feature] Add GLM-5.3-Flash F1: VL data preprocessing pipeline #2109 — F1: VL data preprocessing
  6. [Feature] Add GLM-5.3-Flash F2: vision tower + projector (eager) #2110 — F2: vision tower and projector
  7. [Feature] Add GLM-5.3-Flash F6 core: text model + MTP + compose model #2111 — F6: text and compose model
  8. [Test] Check GLM-5.3 end-to-end logits alongside loss #2139 — F7: end-to-end loss and logits parity ← you are here

Base is #2111's branch (feat/glm53flash-f6-text-moe). Review this PR's own diff; the seven lower layers are dependencies.


The existing TestGlm53TextMoEAccuracy.test_fsdp_accuracy compares native HF and XTuner forward loss on the reduced GLM-5.3 checkpoint. Retain that test and its loss assertion, and additionally compare full-vocabulary logits at the final eight next-token positions for each existing text sample.

The test still uses the same checkpoint, sentences and EP1/4/8 parameterization. Capture HF logits during its existing forward; request XT logits with loss_ctx=None in an additional no-grad forward so the public training-loss path remains covered. Both models run in eval mode. There is no backward or parameter update.

The small metric helper checks shape/finiteness and computes relative L2, cosine and maximum absolute error with CPU FP64 reductions. Each sample must satisfy relative L2 <5% and cosine >0.998, independently of the unchanged loss-curve thresholds. Six CPU checks cover identity, shape/nonfinite failures, position permutations, and a common logits offset that leaves CE unchanged but must fail the logits check.

Validation:

  • Existing real-checkpoint accuracy test with the added logits assertions: 3 passed, EP1/4/8 on 8×H200. These retained files are unchanged from that GPU-validated version.
  • Metric regressions rerun after scope reduction: 6 passed.
  • Diff whitespace check passed.

This PR contains only the existing text test change, the metric helper and its regression tests. The separate multimodal replay/data-preparation scripts and documentation are excluded. No model implementation, preprocessing or training recipe is changed. Base is #2111; the vision RoPE fix is already in the underlying stack.

@xs1997zju
xs1997zju added this pull request to stack #2112 October 8, 2026 12:57
@xs1997zju
xs1997zju marked this pull request as ready for review October 8, 2026 12:58
@xs1997zju

Copy link
Copy Markdown
Collaborator Author

Superseded by #2141, an ordinary PR targeting the existing F6 layer (#2111), combining the logits and image parity tests without adding a stack layer.

@xs1997zju xs1997zju closed this Oct 9, 2026
@jayhenry
jayhenry deleted the feat/glm53flash-f7-logits-parity branch October 9, 2026 06:18
@jayhenry
jayhenry removed this pull request from stack #2112 October 15, 2026 03:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant