“Its Transformer Engine acceleration “delivers up to 12x faster inference performance at GPT-3 compared to the prior generation A100 at data center scale,” the company said.”
Nvidia already owns training. The H100 NVL, with 94 gigabytes of memory, goes after inference, the work Intel’s Xeon chips have long dominated. Jensen Huang hand-delivered OpenAI its first DGX system in 2016 and now calls ChatGPT an “iPhone moment.” Nvidia sells the shovels and gets to name the gold rush.