Gemma 4 Technical Report

Gemma 4 is Google DeepMind's fourth open-weight family, released under Apache 2.0. It has five models. E2B and E4B are dense models with 2.3B and 4.5B effective parameters (5B and 8B including per-layer embeddings), built for phones and laptops, with vision and audio encoders. 12B is dense and encoder-free: image patches pass through a single 35M-parameter projection and raw audio is fed in 40 ms chunks. 26B-A4B is a mixture-of-experts with 3.8B parameters active per token. 31B is dense.

All five have a thinking mode, switched on by putting <|think|> in the system turn, that writes a reasoning trace before answering. The architecture is tuned for memory: five sliding-window layers per global attention layer (four for E2B), keys reused as values in global layers of the larger models, p-RoPE on global layers, and KV-cache sharing across layers in E2B and E4B. Each model ships with a small multi-token-prediction drafter (76M to 500M parameters) for speculative decoding and with quantization-aware-trained checkpoints.

In thinking mode, the 31B scores 89.2 on AIME 2026, 80.0 on LiveCodeBench v6 and 84.3 on GPQA Diamond; Gemma 3 27B, without thinking, scored 20.8, 29.1 and 42.4. On the Arena text leaderboard the 31B reached 1451 Elo, the top dense open model, against 1508 for the top closed model. The report's Table 3 gives memory for weights and a 32k-token KV cache for every size.

Most open-model reports give benchmark tables. This one also gives the numbers you need to size a deployment, which makes it a working document for a model-selection conversation.

Memory, per model. Quantized weights take 0.8 GB (E2B), 2.3 GB (E4B), 7.65 GB (12B), 16.2 GB (26B-A4B) and 19.2 GB (31B). An int8 KV cache for one 32k-token session adds 0.05 to 1.10 GB. From those two numbers you can estimate how many concurrent long sessions fit on a 24 GB or 80 GB card before testing anything.

The MoE trade. 26B-A4B computes like a 4B model per token but holds 26B of weights, so it saves compute and latency, not memory. Its KV cache is a quarter of the 31B's at the same context, which matters more than its weights once many users share a GPU.

Images cost context. The vision budget is set per image from 70 to 1,120 soft tokens. Documents need the high end: the 12B's InfographicVQA score falls from 88.4 to 58.7 at 280 tokens. A 40-page PDF at full resolution is over 40,000 tokens of context.

Thinking costs tokens. The large gains on math and code are in thinking mode. Each answer then carries a reasoning trace, so measure latency and token spend with it on and off for the customer's tasks.