“Mistral NeMo was trained with quantisation awareness, enabling FP8 inference without any performance loss.”
Mistral and NVIDIA released a 12B model with a 128k context window under Apache 2.0. It compares itself to Gemma 2 9B and Llama 3 8B, two smaller models. The instruction-tuned scores come with a footnote: “Evals done with GPT4o as judge.” It is packaged as an NVIDIA NIM container, which tells you who the partnership is really for.