Apple Silicon and macOS VMs: 11-16x Faster LLM Inference with Llama.cpp

“On an M1 Ultra, TinyLlama 1.1B running through llama.cpp processed prompts 11.08x faster and generated tokens 16.36x faster than the same workload in the same stock VM.”

The speedup is not new hardware. Apple’s virtualization layer understates GPU capability to the guest, so llama.cpp picks the slow Metal kernels and everyone assumes VMs are just bad at this. A shim reports the real GPU family and the numbers jump an order of magnitude. It works by leaning on private Metal internals, was tested on one machine, and Apple can break it in any point release. Good benchmark. Bad foundation.