DeepSeek V4 Pro Base

DeepSeek-V4-Pro-Base is the pre-trained foundation checkpoint of DeepSeek's flagship V4 series, released April 24, 2026 under the MIT license. As a 1.6-trillion-parameter Mixture-of-Experts model with 49 billion parameters activated per token, it represents the largest open-weight base model available and sets new performance ceilings across knowledge, reasoning, coding, and long-context benchmarks among DeepSeek foundation models. The model was pre-trained on over 33 trillion tokens using a novel architecture that combines Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA), achieving dramatic efficiency gains: at 1M-token context, it requires only 27% of the single-token inference FLOPs and 10% of the KV cache compared to its predecessor, DeepSeek-V3.2.

Key architectural innovations include Manifold-Constrained Hyper-Connections (mHC) for stable deep-layer signal propagation, the Muon optimizer for faster convergence, and FP4+FP8 mixed-precision training for MoE expert weights. The base model demonstrates comprehensive superiority over V3.2-Base across virtually all evaluated domains — achieving 90.1 on MMLU, 73.5 on MMLU-Pro, 76.8 on HumanEval, 92.6 on GSM8K, and 51.5 on LongBench-V2 — while operating with a significantly larger expert pool and supporting native 1M-token contexts.

As a base model, DeepSeek-V4-Pro-Base is designed for researchers and developers building custom post-training pipelines. It is not intended for direct conversational use; that role falls to the instruct variants (DeepSeek-V4-Pro with Non-think, Think High, and Think Max reasoning modes). The companion DeepSeek-V4-Flash-Base offers a more deployable alternative at 284B total parameters (13B active), achieving most of Pro's quality at roughly one-fifth the parameter count.