“Cerebras inference delivers 1,800 tokens per second for Llama3.1 8B and 450 tokens per second for Llama3.1 70B, which is 20x faster than NVIDIA GPU-based hyperscale clouds.”
Cerebras gets the speed by keeping the whole model in 44GB of SRAM on one wafer-sized chip. A 70B model at 16 bits needs 140GB, so it runs across as few as four CS-3 systems. The launch offers only Llama3.1 8B and 70B and a vague promise that pricing is “a fraction” of GPU clouds. Fast is easy to claim when nobody can see the bill for four wafers.