Gemma 3 QAT Models: Bringing state-of-the-Art AI to consumer GPUs

April 20, 2025 · Edouard Yvinec and Phil Culliton

URL: https://developers.googleblog.com/en/gemma-3-quantized-aware-trained-state-of-the-art-ai-to-consumer-gpus/

“Using int4 means each number is represented using only 4 bits – a 4x reduction in data size compared to BF16.”

Google’s quantized Gemma 3 27B fits in about 14GB, enough for one RTX 3090. Quantization-aware training cuts the quality loss roughly in half. The memory figures cover weights only, and long contexts need more. Running a capable model at home keeps getting cheaper.