Qwen3 Technical Report

Qwen3 is a family of eight open-weight models under Apache 2.0: dense models of 0.6B, 1.7B, 4B, 8B, 14B and 32B, and two mixture-of-experts models, 30B-A3B and 235B-A22B (235B total, 22B active per token, 128 experts with 8 active and no shared experts). All were pre-trained on 36 trillion tokens covering 119 languages and dialects, in three stages: about 30T general tokens at 4K length, about 5T reasoning-heavy tokens, then long-context data at 32K, extended to 128K at inference with YaRN.

The main product idea is one model with two modes. Adding /think or /no_think to a message (or a template flag) turns the reasoning block on or off; with thinking off the model emits an empty <think></think> and answers directly. A thinking budget stops reasoning after N tokens, inserts a fixed sentence telling the model to answer from what it has, and lets it finish.

The two flagships went through four post-training stages: a small long chain-of-thought cold start, reasoning RL with GRPO on 3,995 verifiable problems, a fusion stage that teaches both modes, and general RL across more than 20 task types. The six smaller models were distilled from them instead, first from teacher outputs and then by matching the teacher's token probabilities on the student's own samples.

The flagship scores 85.7 on AIME'24, 81.5 on AIME'25 and 70.7 on LiveCodeBench v5 with thinking, beating DeepSeek-R1 on 17 of 23 benchmarks. Without thinking it beats GPT-4o (November 2024) on 18 of 23.

The thinking switch turns a model-routing problem into a per-request setting, and this report has the data to set it.

Thinking pays on hard reasoning only. On the flagship, turning thinking on adds 56.8 points on AIME'25 and 35.4 on LiveCodeBench, but 0.2 on IFEval, 2.8 on BFCL tool calling and -0.5 on Arena-Hard. For a support chatbot or a tool-calling agent, thinking mostly adds latency and tokens. For SQL generation, debugging or calculation, it is the difference between usable and not.

Long-context retrieval gets worse with thinking. On RULER, thinking mode scores lower than non-thinking at every size (8B: 84.4 against 89.1). For RAG over long documents, the authors suspect the reasoning interferes with retrieval. Default to non-thinking there.

Budgets give a cost dial. The flagship's accuracy rises steeply between 4K and 16K thinking tokens on AIME and flattens after; on GPQA most of the gain is in by 8K. A product can cap thinking at the point where the next doubling stops paying.

Small models are distilled. The 4B and 8B models inherit both modes from the flagships. Distillation was ten times cheaper than RL on the 8B and scored higher, which is also a recipe for customers training their own small model from a larger one.