Skip to content

Add bfloat16 and float8e4m3 to the spec - #956

Open
mklimenko-nv wants to merge 2 commits into
webmachinelearning:mainfrom
mklimenko-nv:low-precision-fp-spec
Open

mklimenko-nv wants to merge 2 commits into
webmachinelearning:mainfrom
mklimenko-nv:low-precision-fp-spec

Conversation

@mklimenko-nv

@mklimenko-nv mklimenko-nv commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

This PR adds WebNN spec wording to support bfloat16 and float8e4m3, according to #938

Detailed changes:

  • MLOperandDataType: new values "bfloat16" and "float8e4m3", plus a definition list describing every value. float8e4m3 is the OCP OFP8 E4M3 encoding (1/4/3 bits, bias 7, no infinities, max ±448, NaN = S.1111.111), mirroring the PyTorch float8_e4m3fn and ONNX FLOAT8E4M3FN. Neither new type is a required data type for any operation, support is discoverable through opSupportLimits().
  • Per-operator tables: bfloat16 is added to the allowed data types wherever float16 is allowed. The required columns are untouched. float8e4m3 is allowed on dequantizeLinear (input, zeroPoint), quantizeLinear (zeroPoint, output) and cast, and is implicitly allowed on operands whose allowed types are any. It is intentionally not listed on compute operators: fp8 compute is expressed through the Q/DQ pattern and realized by backend fusion, which a Note next to the enum spells out.
  • cast(): the behavior table gains bfloat16 and a float8e4m3 target column. Conversions to float8e4m3 saturate (out-of-range and infinite values become ±448, NaN is preserved), matching OFP8's saturating mode and the ONNX Cast default. NaN handling is now stated in every float-source cell.
  • ConvertToFloat (used for MLNumber casting) takes the data type instead of a bit length and handles the two new formats, RNE and saturation match OFP8 1.1 Appendix A.
  • quantizeLinear / dequantizeLinear: states the type promotion (arithmetic happens in the scale/input float type) and the float8 formula cast(input / scale + cast(zeroPoint, input.dataType), zeroPoint.dataType). The quantizeLinear emulation branches on integer vs. floating point output and clamps to the output type's range instead of a hardcoded 0...255. A note points out that fp8 quantization is typically symmetric (zeroPoint = zeros).
  • Buffer compatibility appendix: bfloat16 <-> Uint16Array, float8e4m3 <-> Uint8Array (native-endian bit patterns; Uint8Array remains the generic view for any type). validate buffer with descriptor needed no change since it is table-driven. The stale "Float16Array is Stage 3" note is refreshed.
  • Bibliography: adds OCP OFP8 Revision 1.1 (January 2026).

Also fixed while here: the isNaN/isInfinite tensor-limits table said any while the method steps restricted to floating point types; the table now matches the steps.


Preview | Diff

@fdwr fdwr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✌️ Looks good Mikhail. Thanks. Minor comments...

Comment thread index.bs Outdated
Comment thread index.bs
: <dfn>bfloat16</dfn>
:: 16-bit "brain" floating point, with 1 sign bit, 8 exponent bits (exponent bias 127) and 7 trailing significand bits. It has the same exponent range as {{MLOperandDataType/"float32"}}, and infinities, NaNs, signed zeros and subnormal values are represented as in the binary interchange formats of [[IEEE-754-2019]]. The bit layout of a value in this format is identical to the 16 most significant bits of the binary32 encoding of the same value.
: <dfn>float8e4m3</dfn>
:: 8-bit floating point in the E4M3 format of the OCP 8-bit Floating Point Specification [[OCP-OFP8]], with 1 sign bit, 4 exponent bits (exponent bias 7) and 3 trailing significand bits. Unlike the other floating point data types, this format has no representation for infinities; the largest finite magnitude is 448, and NaN is encoded with all exponent and significand bits set to 1. Signed zeros and subnormal values are supported. Conversions to this format are saturating: values whose magnitude exceeds 448, including infinities, convert to the largest finite value with the same sign, see {{MLGraphBuilder/cast()}}.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this format has no representation for infinities

Oh yeah, I thought this was the one with infinity for a moment (float8m3e4s1_t), but it's actually (float8m3e4s1fn_t) from the earlier table.

  • float8m3e4s1_t - mantissa: 3 bits, exponent: 4 bits, sign: 1 bit, infinities, NaNs
  • ⭐ float8m3e4s1fn_t - mantissa: 3 bits, exponent: 4 bits, sign: 1 bit, infinities, NaNs
  • float8m3e4s1fnuz_t - mantissa: 3 bits, exponent: 4 bits, sign: 1 bit, no infinities, NaN as -0

Double checking D3D12_LINEAR_ALGEBRA_DATATYPE_FLOAT8_E4M3FN and ONNX TensorProto.DataType.FLOAT8E4M3FN and PyTorch, indeed there is no infinity. Additionally verified here:

(resolve me)

@mklimenko-nv mklimenko-nv Oct 8, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's the fn flavor. To be on the safe side, let's pin the encoding down (no infinities, NaN = S.1111.111, max 448), so the enum value is unambiguous whatever we call it.
Personally, I'd prefer to keep the unsuffixed version to match the approach in OCP spec itself, TensorRT, OpenVINO, etc. There is a camp that prefers explicit suffixes (PyTorch, ONNX, CoreML), so both approaches have industry backing. Since we specify explicitly what the data type means, there shouldn't be misreadings, but we can adjust it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to keep the unsuffixed version to match the approach in OCP spec itself, FP8 (E4M3)
...
There is a camp that prefers explicit suffixes (PyTorch, ONNX, CoreML {and Direct3D})

Hmm, I wager a reason why people appended the "fn" is because the spec nomenclature was ambiguous and surprising, given that same spec with "E4M3" also has "E5M2", with no visible distinction that they differ not only it bit allocation (which is apparent), but also infinity support and NaN representations. The type is an odd outlier in that regard compared to every other WebNN float type. Anyway, a minor naming change can be a separate issue.

Comment thread index.bs
Comment thread index.bs

{{MLOperandDataType/"bfloat16"}} and {{MLOperandDataType/"float8e4m3"}} are not among the [=/required data types=] of any operation. Support can be queried per operation using {{MLContext/opSupportLimits()}}.

Note: The {{MLOperandDataType/"float8e4m3"}} format corresponds to the `float8_e4m3fn` type of PyTorch and the `FLOAT8E4M3FN` type of ONNX. It is distinct from E4M3 encodings that reserve codes for infinities and from the "FNUZ" variants without negative zero; neither is represented by this enum.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You think we should make that more apparent in the naming, that this type is finite and doesn't support the usual norms expected of a floating-point type?

float8e4m3 -> float8e4m3fn

(just seeing float8e4m3 by itself, I would have expected infinity too)

Comment thread index.bs Outdated
1. If |a|'s [=MLOperand/dataType=] is not {{MLOperandDataType/"uint8"}}, then [=exception/throw=] a {{TypeError}}.
1. If |op| is one of "isNaN", "isInfinite", then:
1. If |a|'s [=MLOperand/dataType=] is not one of « {{MLOperandDataType/"float32"}}, {{MLOperandDataType/"float16"}} », then [=exception/throw=] a {{TypeError}}.
1. If |a|'s [=MLOperand/dataType=] is not one of « {{MLOperandDataType/"float32"}}, {{MLOperandDataType/"float16"}}, {{MLOperandDataType/"bfloat16"}} », then [=exception/throw=] a {{TypeError}}.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't float8 types also be present for isNaN(), since float8e4m3fn supports NaN's?

We could also add a helper function (like with blockwise broadcastable https://webmachinelearning.github.io/webnn/#blockwise-broadcastable) that returns true for floating point types, rather than listing them all out 🤷‍♂️.

Now, isInfinite is more interesting, since that particular float8 format doesn't support infinities, but maybe the function should still succeed (since isNaN and isInfinite should generically work with any floating point data type) but just return a false tensor in that case 🤔.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sounds fair, fp8 is now allowed in both isNaN() and isInfinite().

On the topic of helper function I'm not sure, since the scope of fp8 is somewhat limited. But I don't have a strong opinion on that and it should be straightforward to add.

@fdwr fdwr left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@huningxin huningxin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants