“This diagram cost us three weeks of life that we will not get back, hence the public shaming.”
An H100 has 989 TFLOPs of half-precision matrix multiply and about 60 TFLOPs of everything else. Any cycle the tensor cores sit idle, you use at most 6% of the chip. Skip the new wgmma instructions and you top out around 63% of peak, and NVIDIA’s documentation for their swizzled layouts is wrong. NVIDIA sells the hardest chip in AI to feed and hands out a wrong map to it.