“Real numbers can’t be perfectly represented in hardware simply because there are infinitely many of them.”
A Madrid team built a processor core that does posit math in hardware, and accuracy improved by up to four orders of magnitude on a basic task. Training GPT-3 cost about $5 million in compute. Meanwhile Nvidia, Arm and Intel just agreed on an 8-bit float standard. The industry picks whatever its existing silicon already does.