“The implementation of the MoE layer in this repository is not efficient.”
xAI dropped 314 billion parameters of Grok-1 weights under Apache 2.0, delivered by magnet link. The repo is example JAX code that xAI itself calls inefficient, and running it needs a machine with enough GPU memory for a 314B model. There is no training data, no training code and an 8,192 token context. Open weights that almost nobody can run are a press release with a torrent attached.