[Feature] add sm120 hidden datatypes support (float8e3m4 / float4e0m3)

[jinzhen-lin]

This PR add sm120 hidden datatypes support (float8e3m4 / float4e0m3).

The sm120 devices supports float8e3m4 and float4e0m3 but PTX doesn't expose. We can patch the cubin to use those formats.

Technical Details

• The E3M4 format here is an FN (finite number) format, meaning there is no infinity. The maximum value is 30.
• The E0M3 format has a value range (-7, -6, -5, ..., 5, 6, 7). The value range is equivalent to that of INT4 but without -8. Note that floating-point numbers with an exponent field of 0 are not actually defined by the specification. Humming simply chooses to make the data type that falls within this value range an E0M3.
• Each SASS instruction is 128 bit, little-endian.
• opcode = the low 12 bits (byte0 | ((byte1 & 0x0f) << 8)), can be used to identify the instruction class.

OMMA (FP4 MMA)

• opcode: 0x47f
• SASS (NVFP4): OMMA.SF.16864.F32.E2M1.E2M1.UE4M3.4X
• SASS (MXFP4 with groupsize=16): OMMA.SF.16864.F32.E2M1.E2M1.E8.4X

bit	meaning	0	1
78	A element format	E2M1	E0M3 (hidden)
79	B element format	E2M1	E0M3 (hidden)

MXFP4 with groupsize=32 doesn't support E0M3.

QMMA (FP8 MMA)

• opcode: 0x47a
• SASS (scaled): QMMA.SF.16832.F32.E4M3.E4M3.E8
• SASS (plain): QMMA.16832.F32.E4M3.E4M3

Two independent 3-bit type fields:

• A type field = {bit78, bit82, bit83}
• B type field = {bit79, bit84, bit85}

code	format
000	E4M3
001	E5M2
010	E3M4 (hidden)
011	E3M2
100	E2M3
101	E2M1
110	ERROR (illegal)
111	ERROR (illegal)

CVT (F2FP)

• opcode: 0x23e
• SASS (f32 → narrow): F2FP.SATFINITE.<fmt>.F32.PACK_AB_MERGE_C
• SASS (narrow → f16(up)): F2FP.<f16>.<fmt>.UNPACK_B

Down-convert format subfield = (bit87, bit86, bit85, bit76).

(b87,b86,b85,b76)	format
1,0,0,1	E4M3
1,0,0,0	E5M2
1,0,1,0	E3M4 (hidden)
0,1,1,1	E3M2
0,1,1,0	E2M3
0,0,1,1	E2M1
0,0,1,0	E0M3 (hidden)
other	illegal / garbage

Humming Implement

Humming developed kernels that use these two formats. Since these formats are not exposed in PTX, we use E5M2 and E2M1 as substitutes during compilation and then patch the resulting cubin file.

The current limitation is that mixed-precision multiplication between different formats with the same bit width is not supported (for example, A cannot be E4M3 while B is E3M4, nor can A be E2M1 while B is E0M3). This is supported in hardware, but has not yet been implemented in Humming.

performance

NVFP4-E0M3

python benchmarks/bench_humming.py --shape_n 8192 --shape_k 8192 --a_dtype float4e0m3 --b_dtype float4e0m3 --bs_dtype float8e4m3 --c_dtype bfloat16 --input_scale_group_size 16 --weight_scale_group_size 16 --use_m_major_input_scale --shape_m_list 8192

+-----------+--------+---------------+----------------+
|   shape_m |   time |   memory_gbps |   compute_tops |
+===========+========+===============+================+
|      8192 | 0.7068 |      296.7141 |      1555.6058 |
+-----------+--------+---------------+----------------+

MXFP8-E3M4

python benchmarks/bench_humming.py --shape_n 8192 --shape_k 8192 --a_dtype float8e3m4 --b_dtype float8e3m4 --bs_dtype float8e8m0 --c_dty
pe bfloat16 --input_scale_group_size 32 --weight_scale_group_size 32 --use_m_major_input_scale --shape_m_list 8192

+-----------+--------+---------------+----------------+
|   shape_m |   time |   memory_gbps |   compute_tops |
+===========+========+===============+================+
|      8192 | 1.4191 |      192.1109 |       774.7685 |
+-----------+--------+---------------+----------------+

FP8-E3M4 (no-blocked, channelwise)

python benchmarks/bench_humming.py --shape_n 8192 --shape_k 8192 --a_dtype float8e3m4 --b_dtype float8e3m4 --bs_dtype bfloat16 --c_dtype
 bfloat16 --shape_m_list 8192

+-----------+--------+---------------+----------------+
|   shape_m |   time |   memory_gbps |   compute_tops |
+===========+========+===============+================+
|      8192 | 1.3505 |      198.8095 |       814.1620 |
+-----------+--------+---------------+----------------+

The patched cubin can utilize the device's peak TFLOPS.

[gemini-code-assist]

Code Review

This pull request introduces support for hardware narrow-float formats (float8e3m4 and float4e0m3) not natively exposed by PTX, implementing a C++ cubin patcher (patch_cubin.cpp) and integrating it into the JIT compiler, runtime, and quantization kernels. The review feedback identifies several critical and medium-severity issues, including a copy-paste bug in humming/kernel/humming.py that selects the wrong patching mode, potential out-of-bounds reads during ELF parsing, unhandled failures for ftell and fwrite in the C++ patcher, a lack of error handling when g++ is missing during compilation, and potential undefined behavior in cubin_patch_buffer with null or empty inputs.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[huangzhilin-hzl]

This path has a PTX ISA version requirement. The combination kind::mxf4nvf4 + scale_vec::4X + ue8m0 is not accepted by PTX ISA 8.8 . Support for this combination was added in PTX ISA 9.1

[jinzhen-lin]

Thank you for pointing that out. However, this issue also exists under the original e2m1, and the compatibility problem with different CUDA versions is not the target of this PR. We will address it in future work.

[huangzhilin-hzl]

https://docs.nvidia.com/cuda/archive/12.9.2/parallel-thread-execution/index.html

https://docs.nvidia.com/cuda/archive/13.1.1/parallel-thread-execution/index.html#ptx-isa-version-9-1

[huangzhilin-hzl]

Credit to @im0qianqian for the original idea shared in https://zhuanlan.zhihu.com/p/2059376150565089368, which inspired this PR.