Skip to content

tcu/dpi: apply MX block scales with a float multiply - #492

Open
youzi27 wants to merge 3 commits into
vortexgpgpu:masterfrom
youzi27:fix/dpi-mx-scale-multiply
Open

youzi27 wants to merge 3 commits into
vortexgpgpu:masterfrom
youzi27:fix/dpi-mx-scale-multiply

Conversation

@youzi27

@youzi27 youzi27 commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Fixes #491.

Summary

The DPI FEDP added the E8M0 block scales into the exponent field of each group's fp32 sum. That wrapped when the scaled sum left the fp32 range and made NaN and Inf products finite. Each MXFP8, MXBF8 and MXFP4 element is now multiplied by its scale as the fp32 value 2^(scale-127), as SimX and the test references decode MX elements. The summation order is unchanged, so in-range results stay bit for bit the same. The first commit makes those decodes (sw/common/softfloat_ext.cpp) return NaN for a 0xFF scale, as TFR and docs/designs/tensor_core_wgmma_engine.md have it; they gave Inf when every scaled element was nonzero. The last commit adds sgemm_tcu_mx -u and three -k64 -u cases to tensor_mx.yaml.

Impact and evidence

Impact class: R2

Contracts touched: none

evidence status detail
functional tests run (simx, rtlsim) new rtlsim cases fail on master (1024, 1024, 1022 of 1024 wrong) and pass here; every DPI MX rtlsim case and every MX simx case of tensor_mx, tensor_sp_mx, tensor_wg passes, cycles equal to master
model_parity run model_parity-mxfp8 (gap 0.26 %), -lsu-lanes (3.41 %), same as master
perf_gate run perf_gate-mxfp8: cycles equal to master; not compared with the baseline (toolchain v3.0.1 here)
Beacon logic_gate n/a the DPI FEDP is simulation only
synthesis n/a same
XLEN 64 build run the new cases

Run through ci/blackbox.sh, not pytest. Not run: xrt, the full ci/regression.sh.

  • No test silenced (known_issue:, wider tolerance, smaller args, removed case) without a root-cause plan.
  • Golden data changed only through its update command, named in the commit message.
  • Cycle-moving RTL changes carry the matching SimX change, or the parity case that shows none moved.
  • A toolchain pin moves only after the prebuilt tag exists.

An E8M0 scale of 0xFF is NaN (docs/designs/tensor_core_wgmma_engine.md),
and the TFR datapath and its TfrFedp model make every element it scales
NaN. mxfp8_to_f32, mxbf8_to_f32 and mxfp4_to_f32 instead multiplied by
ldexp(1.0f, 128), which is +Inf, so a nonzero element became Inf and
only a zero one NaN. These decodes are SimX's MX datapath for the
non-TFR tensor-core types and the reference of the sgemm_tcu_mx,
sgemm_tcu_sp_mx and sgemm_tcu_wg_mx tests. They now return NaN for that
scale, as TFR does. Other scales are unchanged.
The DPI FEDP applied the two E8M0 block scales by adding their exponents
into bits [30:23] of each group's fp32 product sum with an 8-bit add. When
the scaled group left the fp32 exponent range the field wrapped: a group
below the normal range became a huge value and an overflowing group a
tiny one. A NaN or Inf product also became a finite number whenever the
scale was not 1, and a 0xFF scale gave Inf instead of NaN.

Each element is now multiplied by its scale as the fp32 value
2^(scale-127), the way SimX and the test references decode MX elements
(rv_mxfp8tof_s and friends). Overflow goes to Inf, underflow is gradual,
NaN propagates, and a 0xFF scale is NaN, as in SimX and the TFR datapath.
While the scaled values stay in the normal range the result is the same
as before, since scaling by a power of two is exact there.
-u scales the first K block of A and B by 2^-70 before quantization.
That block then gets E8M0 scales near 2^-78 and its products fall below
the fp32 normal range, so with -k64 every output is the sum of the
second block plus a negligible term. The DPI backend used to wrap the
exponent of such a block into a huge value.

tensor_mx runs it for mxfp8, mxbf8 and mxfp4 at NT=8 on simx and on
rtlsim with the DPI backend.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DPI tensor core: MX block scales wrap the fp32 exponent

1 participant