“A BitNet model with 3 billion parameters performed just as well on various language tasks as a full-precision LLaMA model with the same number of parameters and amount of training, but it was 2.71 times as fast, used 72 percent less GPU memory, and used 94 percent less GPU energy.”
Microsoft’s BitNet 1.58b stores every weight as -1, 0 or 1. At 3 billion parameters it matched LLaMA and cut GPU energy by 94 percent. The industry answer to AI’s power bill has been more data centers and more GPUs. Rounding the numbers off beats building another power plant.