“This was accomplished by changing the file format so we can mmap() weights directly into memory without having to read() or copy them”
The big win is not clever math. It is letting the kernel’s file cache hold the weights instead of copying gigabytes through standard I/O. Loads run up to 100x faster, models up to 2x larger fit, and many processes share one copy. The price is a breaking change to the file format, and everyone gets to convert their models again.