Google Deep MindConditional Open

Gemma 4 120B

Name: Gemma 4 120B
Author: Google Deep Mind

Compare this model

A high-performance open-source model developed by Google DeepMind. It excels in balancing efficiency and performance.

Share:X Hatena LINE

Parameters

1200

Context Window

128K

License

Gemma License

Release Date

2026-05-19

Japanese Language Capability

✅High-Quality JP

Multilingual model with strong Japanese language processing capabilities.

API Pricing

API pricing for this model is not yet available

Strengths

Weaknesses

Use Cases

Deep Analysis

Parameters

11.95B

Dense, encoder-free unified architecture

Context Window

256K tokens

Matches Gemma 4 31B and 26B-A4B

GPQA Diamond

78.8%

vs Llama 4 Scout 74.3%

AIME 2026

77.5%

3.8x improvement over Gemma 3 27B (20.8%)

Codeforces ELO

1659

Strong competitive coding

Modalities

Text + Image + Audio

First medium Gemma with native audio; encoder-free

VRAM (Q4)

~7.65 GB

Runs on 16GB laptop with quantization

License

Apache 2.0

First Gemma generation with full commercial freedom

Strengths

・Encoder-free unified architecture eliminates separate vision/audio encoders, reducing latency and enabling single-pass fine-tuning
・Native audio input (ASR + translation) in a 12B model — first medium-sized Gemma with this capability
・256K context window with hybrid local/global attention and KV cache optimizations at a laptop-deployable size

Weaknesses

・No dedicated Arena Elo ranking yet — lacks the human-preference validation data that the 31B and 26B-A4B have received
・Encoder-free vision projects raw patches with a single matmul (35M params), sacrificing fine-grained visual detail vs the 550M dedicated encoder in the 31B
・Lags significantly behind the 31B Dense on complex multi-step reasoning (HLE 5.2% vs 19.5%, BigBench Extra Hard 53.0% vs 74.4%)

Competitor Comparison

Model	Arena	SWE	GPQA	Price
Gemma 4 31B Dense	1451	N/A	84.3%	Free (Apache 2.0)
Gemma 4 26B-A4B MoE	1438	N/A	82.3%	Free (Apache 2.0)
Llama 4 Scout (17B active)	~1400	N/A	74.3%	Free (Llama License)

Overview

Gemma 4 12B Unified is Google DeepMind's first encoder-free, medium-sized open model, released in June 2026 as part of the broader Gemma 4 family. Its defining architectural innovation is the elimination of separate vision and audio encoders: raw 48×48 image patches and 40ms audio chunks are projected directly into the LLM's embedding space via lightweight linear layers. This unified design reduces multimodal latency, lowers the memory footprint (24 GB at BF16, ~7.65 GB with Q4_0 quantization for weights alone), and allows the entire model to be fine-tuned in a single pass. The model handles text, image, and audio inputs with a 256K token context window — matching the largest Gemma 4 variants. Benchmarks position the 12B Unified as a strong mid-tier model that punches above its weight. On GPQA Diamond (78.8%), it outperforms Llama 4 Scout's 74.3% despite having fewer active parameters. Its AIME 2026 score of 77.5% and LiveCodeBench v6 score of 72.0% demonstrate robust mathematical and coding reasoning. However, there is a meaningful quality gap to the 31B Dense (GPQA 84.3%, AIME 89.2%) and even the 26B-A4B MoE (GPQA 82.3%, AIME 88.3%), particularly on complex multi-step reasoning tasks like HLE (5.2% vs 19.5% for 31B) and BigBench Extra Hard (53.0% vs 74.4%). The model supports native function calling for agentic workflows, configurable thinking modes for step-by-step reasoning, and multilingual support across 35+ languages with pre-training on 140+. The 12B Unified targets developers who need a laptop-deployable multimodal model with audio capability — a niche no other medium-sized Gemma fills. It runs on consumer hardware with 16GB VRAM (8-bit) or 8–10GB (4-bit), making it accessible for local prototyping, privacy-sensitive document processing, and agentic coding workflows. Released under Apache 2.0 with day-one support across Hugging Face Transformers, llama.cpp, Ollama, MLX, SGLang, and vLLM, it arrives as the most deployability-optimized entry in the Gemma 4 lineup.

Benchmarks & Performance

### Official Benchmark Results (Gemma 4 12B Unified, instruction-tuned, thinking mode) | Benchmark | Gemma 4 12B | Gemma 4 31B | Gemma 4 26B-A4B | Gemma 3 27B | |---|---|---|---|---| | MMLU Pro | 77.2% | 85.2% | 82.6% | 67.6% | | AIME 2026 (no tools) | 77.5% | 89.2% | 88.3% | 20.8% | | LiveCodeBench v6 | 72.0% | 80.0% | 77.1% | 29.1% | | Codeforces ELO | 1659 | 2150 | 1718 | 110 | | GPQA Diamond | 78.8% | 84.3% | 82.3% | 42.4% | | Tau2 (avg) | 69.0% | 76.9% | 68.2% | 16.2% | | HLE (no tools) | 5.2% | 19.5% | 8.7% | — | | BigBench Extra Hard | 53.0% | 74.4% | 64.8% | 19.3% | | IFEval | 97.2% | 98.9% | 98.5% | 90.4% | | IFBench | 74.0% | 76.0% | 72.0% | 32.0% | | MMMLU | 83.4% | 88.4% | 86.3% | 70.7% | | Terminal Bench Hard | 18.0% | 36.0% | 14.0% | 4.0% | **Vision Benchmarks:** | Benchmark | Gemma 4 12B | Gemma 4 31B | Gemma 4 26B-A4B | |---|---|---|---| | MMMU Pro | 69.1% | 76.9% | 73.8% | | MATH-Vision | 79.7% | 85.6% | 82.4% | | MedXPertQA MM | 48.7% | 61.3% | 58.1% | | OmniDocBench 1.5 (↓) | 0.164 | 0.131 | 0.149 | **Audio Benchmarks (12B only among medium models):** | Benchmark | Gemma 4 12B | Gemma 4 E4B | Gemma 4 E2B | |---|---|---|---| | CoVoST (avg BLEU) | 38.5* | 35.54 | 33.47 | | FLEURS ASR (WER ↓) | 0.069* | 0.08 | 0.09 | *Excluding Chinese language. The 12B achieves competitive audio performance despite having no dedicated audio encoder — a key validation of the unified architecture. **Long Context:** | Benchmark | Gemma 4 12B | Gemma 4 31B | Gemma 3 27B | |---|---|---|---| | MRCR v2 8-needle 128k | 43.4% | 66.4% | 13.5% | | RULER 32k | 96.4% | 96.8% | 91.1% | | RULER 128k | 91.2% | 96.4% | 66.0% | **Memory & Hardware:** | Configuration | Weight Memory | + KV Cache (32k ctx) | |---|---|---| | BF16 | 24.0 GB | +0.28 GB | | Q4_0 Quantized | 7.65 GB | +0.28 GB | *Source: Google DeepMind Gemma 4 Technical Report (arXiv:2607.02770)*

Detailed Comparison

### Gemma 4 12B Unified vs Gemma 4 31B Dense | Dimension | 12B Unified | 31B Dense | |---|---|---| | Parameters | 11.95B | 30.7B | | VRAM (Q4) | ~7.65 GB | ~19.2 GB | | Context | 256K | 256K | | GPQA Diamond | 78.8% | 84.3% | | AIME 2026 | 77.5% | 89.2% | | Codeforces ELO | 1659 | 2150 | | HLE | 5.2% | 19.5% | | Arena Elo | N/A | 1451 (#43) | | Encoder-free | Yes | No (550M vision encoder) | | Audio | Yes | No | | Function calling | Yes | Yes (more reliable) | **Summary:** The 31B Dense is the clear quality leader, outperforming the 12B by 5–14 points on reasoning benchmarks and showing a 3.75x advantage on the hardest benchmark (HLE). However, it requires 2.5x more VRAM, lacks audio support, and uses separate encoders. The 12B is the pragmatic choice for laptop deployment and audio-inclusive workflows. ### Gemma 4 12B Unified vs Gemma 4 26B-A4B MoE | Dimension | 12B Unified | 26B-A4B MoE | |---|---|---| | Total Params | 11.95B | 25.2B (3.8B active) | | VRAM (Q4) | ~7.65 GB | ~16.2 GB | | GPQA Diamond | 78.8% | 82.3% | | AIME 2026 | 77.5% | 88.3% | | BigBench Extra Hard | 53.0% | 64.8% | | Encoder-free | Yes | No | | Audio | Yes | No | | Tau2 (avg) | 69.0% | 68.2% | **Summary:** The 26B-A4B MoE delivers meaningfully better reasoning (AIME +10.8pp, BigBench +11.8pp) but at 2x the VRAM. Notably, the 12B actually edges out the MoE on Tau2 agentic benchmarks (69.0% vs 68.2%), and it uniquely supports audio. The MoE also has community-reported latency issues on consumer hardware (~2 tok/s on 24GB MacBooks), making the 12B the better local deployment option. ### Gemma 4 12B Unified vs Llama 4 Scout (17B active) | Dimension | 12B Unified | Llama 4 Scout | |---|---|---| | Active Params | 11.95B | 17B | | GPQA Diamond | 78.8% | 74.3% | | Context | 256K | 10M | | Audio | Yes | No | | License | Apache 2.0 | Llama License | **Summary:** Gemma 4 12B outperforms Llama 4 Scout on GPQA Diamond by 4.5pp despite fewer active parameters and has the advantage of native audio. Llama 4 Scout's massive 10M context window is its differentiator for extreme long-context workloads. Gemma 4's Apache 2.0 license is more permissive than the Llama License.

Community Feedback

The Gemma 4 family has crossed 150 million cumulative downloads as of the 12B's June 2026 launch, according to Google. The developer community has responded positively to several aspects: **Encoder-free architecture excitement:** The unified design has been highlighted as a genuine innovation. Google's developer blog demonstrates the 12B running as a fully local agent that codes Gradio apps powered by the same model, and processing 5 minutes of video with audio in a single inference call. The fact that all modalities share weights and can be fine-tuned in one pass has been noted as a significant practical advantage. **Local deployment enthusiasm:** Community analysis (e.g., the detailed DEV Community breakdown) identifies the 12B as a 'sweet spot' model — strong enough for serious tasks yet deployable on consumer hardware. Multiple guides have emerged showing it running on Mac M-series with 16GB unified memory at ~25-40 tok/s. Ollama, LM Studio, and Google AI Edge Gallery all supported day-one launch. **Controversy over stealth updates:** In July 2026, Google shipped a silent update that improved Flash Attention 4 performance (25-70% prompt processing speedup, up to 31% TTFT reduction), fixed tool calling bugs (10.1% improvement in telecom agentic scenarios), and addressed truncated responses. The community pushed back on Google shipping this under the same 'Gemma 4' name rather than 'Gemma 4.1' — raising reproducibility concerns for researchers citing specific benchmark numbers. **Developer ecosystem support:** The model launched with support across Hugging Face Transformers, llama.cpp, MLX, SGLang, vLLM, Unsloth (fine-tuning), Ollama, and was integrated into Android Studio Agent Mode. Google also released a Skills Repository specifically for building agentic applications with Gemma models. **Critical reception:** Some researchers have noted that while the encoder-free architecture is elegant, the 35M-parameter vision projection may sacrifice fine-grained visual detail compared to the 550M dedicated encoder in the 31B — evident in OmniDocBench scores (0.164 vs 0.131). Others praise the audio capability as a genuine differentiator that no other 10B+ class open model currently offers.

Use Cases

**1. Privacy-Sensitive Document Processing (Legal, Healthcare, Finance):** The 12B runs entirely on-device with no data leaving the machine. For legal document review, medical record summarization, or financial compliance queries, the encoder-free architecture processes PDFs, invoices, and forms — including charts and tables — at the 16GB VRAM tier. Pair with RAG for factual accuracy. Choose this over the 31B when hardware is constrained and data sovereignty is non-negotiable. **2. Local Multimodal Agent Development:** The 12B is the only medium-sized Gemma with native audio, making it uniquely suited for voice-controlled agentic workflows. Google demonstrated it coding a complete Gradio app (an image processing tool powered by itself) via the OpenCode harness. For developers building local AI assistants that can listen, see, and act, this is the model to start with. The native function calling enables structured tool invocation without prompt engineering workarounds. **3. Multilingual Audio Transcription and Translation:** With CoVoST BLEU scores of 38.5 and FLEURS WER of 0.069, the 12B delivers competitive ASR and speech-to-text translation across multiple languages — without any dedicated audio encoder. For teams building offline meeting transcription, voice-controlled interfaces, or multilingual audio pipelines, the 12B eliminates the need for a separate ASR system. Audio inputs support a maximum of 30 seconds per clip. **4. Fine-Tuning Target for Domain-Specific Applications:** The encoder-free design means all modalities share the same weights, so downstream fine-tuning (via LoRA or full fine-tuning with Unsloth) naturally updates the entire multimodal loop in a single pass. At ~24GB BF16, it fits on a single A100 for training. For specialized applications — contract analysis, medical QA, low-resource language adaptation — a fine-tuned 12B can outperform a base 26B MoE on domain-specific benchmarks while being dramatically easier to train. **When NOT to choose the 12B:** If you need the highest possible reasoning quality (AIME 89.2%, HLE 19.5%), choose the 31B Dense. If you need massive context windows (10M tokens), choose Llama 4 Scout. If you need the strongest open-source multi-step mathematical reasoning, choose Qwen 3.5 or DeepSeek V4 families.

Latest News

**June 3, 2026 — Gemma 4 12B Launch:** Google DeepMind released the 12B Unified as a new addition to the Gemma 4 family. Key highlights: encoder-free architecture, native audio input, 256K context, Apache 2.0 license. Google also launched macOS desktop apps (AI Edge Gallery, Eloquent) for local spoken and visual interaction, a LiteRT-LM CLI for OpenAI-compatible local API serving, and an official Skills Repository for agentic development. **July 2026 — Stealth Performance Update:** Google shipped a silent update across all Gemma 4 sizes (including 12B) that introduced Flash Attention 4 support (25-70% faster prompt processing on Hopper GPUs), fixed tool calling bugs with up to 10.1% improvement in agentic scenarios, addressed truncated response issues, and raised the max_soft_tokens parameter to 1,120 for higher-resolution OCR. The update was published under the same 'Gemma 4' name, drawing criticism from the research community about reproducibility. **Ongoing — 150M+ Downloads:** The Gemma 4 family has surpassed 150 million cumulative downloads, with community-built applications ranging from wearable robotic arms to enterprise AI security tools. The 12B has 292K+ downloads on Hugging Face alone. **Ecosystem Expansions:** Day-one integration with Android Studio Agent Mode, LM Studio, Ollama, Google AI Edge Gallery, and LiteRT-LM serve. Google Cloud deployment supported via Gemini Enterprise Agent Platform Model Garden, Cloud Run, and GKE.

Benchmarks position the 12B Unified as a strong mid-tier model that punches above its weight. On GPQA Diamond (78.8%), it outperforms Llama 4 Scout's 74.3% despite having fewer active parameters. Its AIME 2026 score of 77.5% and LiveCodeBench v6 score of 72.0% demonstrate robust mathematical and coding reasoning. However, there is a meaningful quality gap to the 31B Dense (GPQA 84.3%, AIME 89.2%) and even the 26B-A4B MoE (GPQA 82.3%, AIME 88.3%), particularly on complex multi-step reasoning tasks like HLE (5.2% vs 19.5% for 31B) and BigBench Extra Hard (53.0% vs 74.4%). The model supports native function calling for agentic workflows, configurable thinking modes for step-by-step reasoning, and multilingual support across 35+ languages with pre-training on 140+.

The 12B Unified targets developers who need a laptop-deployable multimodal model with audio capability — a niche no other medium-sized Gemma fills. It runs on consumer hardware with 16GB VRAM (8-bit) or 8–10GB (4-bit), making it accessible for local prototyping, privacy-sensitive document processing, and agentic coding workflows. Released under Apache 2.0 with day-one support across Hugging Face Transformers, llama.cpp, Ollama, MLX, SGLang, and vLLM, it arrives as the most deployability-optimized entry in the Gemma 4 lineup.

Sources

Analysis generated: 2026-07-17