DeepSeek-R1’s paper was updated 2 days ago, expanding from 22 pages to 86 pages and adding a substantial amount of detail.

I did quickly throw the two papers into Gemini. It's really fun to live in the future. #Gemini summary: The paper **"DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning"** (arXiv:2501.12948) marks a significant milestone in open-source AI by demonstrating that advanced reasoning (similar to OpenAI’s o1) can be achieved through large-scale Reinforcement Learning (RL) with minimal human-annotated data. The two versions you provided represent the initial release (**v1**, Jan 22, 2025) and the latest updated version (which has been significantly expanded to **86+ pages** as of January 2026). ### 1. Overall Paper Summary The paper introduces two primary models: * **DeepSeek-R1-Zero:** A model trained via "pure RL" (using the GRPO algorithm) starting directly from a base model without any Supervised Fine-Tuning (SFT). It demonstrates that reasoning behaviors like self-correction and reflection can emerge purely from reward signals. * **DeepSeek-R1:** A more "user-friendly" version that uses a multi-stage pipeline (Cold-start SFT → Reasoning RL → Rejection Sampling/SFT → General RL) to fix the "readability" and "language mixing" issues of R1-Zero while maintaining state-of-the-art reasoning performance. * **Distillation:** The authors show that the reasoning patterns discovered by the 671B model can be distilled into smaller models (1.5B to 70B), allowing a 14B model to outperform much larger ones on math and coding benchmarks. --- ### 2. Comparison: Extensions in the New Version The newer version is a massive technical expansion (growing from roughly 22 pages to over 85 pages). The key additions and extensions include: #### A. The "Aha Moment" Expansion (Section 2.2.1) The new version provides a much deeper analysis of the **"Aha Moment"**—the point during RL training where the model unexpectedly learns to "re-think" its approach. The extension includes more qualitative examples and internal data showing the model's transition from linear solving to iterative self-correction without being prompted to do so. #### B. Detailed 4-Stage Training Pipeline While v1 outlined the stages, the new version details the specific composition of the **800k total training samples**: * **Stage 1 (Cold Start):** Expanded details on the ~5,000-10,000 long CoT (Chain of Thought) samples used to "prime" the model. * **Stage 3 (Rejection Sampling):** A deeper dive into how 600k reasoning-related and 200k non-reasoning samples were filtered and used to improve the model's general chat capabilities and prevent "forgetting" during the reasoning-heavy RL stages. #### C. Comprehensive Ablation Studies The new version adds extensive "What if?" scenarios that were absent or brief in v1: * **Distillation vs. RL:** New evidence explaining *why* distilling a large model's reasoning traces into a small model is more effective than training that small model directly with its own RL. * **Base Model Impact:** Analysis of how different base models (DeepSeek-V3 vs. Qwen vs. Llama) respond to the R1 training recipe. #### D. Expanded "Unsuccessful Attempts" (Section 4.2) One of the most valuable additions for researchers is the expanded section on what **did not work**. The new version elaborates on their failures with: * **Process Reward Models (PRM):** Detailed reasons why step-level rewards were difficult to scale or prone to "reward hacking" compared to the outcome-based rewards used in R1. * **Monte Carlo Tree Search (MCTS):** Technical explanation of why MCTS didn't provide the expected gains over simple RL in the context of LLM reasoning. #### E. New Benchmarks & Technical Specs * **Updated Results:** Includes more recent evaluations on benchmarks like **AIME 2025**, **LiveCodeBench**, and specialized medical/legal reasoning tests. * **Hyperparameters:** The new version includes exhaustive tables of training hyperparameters (learning rates, GRPO group sizes, KL divergence coefficients) which were previously withheld or summarized. ### Summary Table | Feature | v1 (Original) | Latest Version (Extension) | | :--- | :--- | :--- | | **Page Count** | ~22 Pages | **86+ Pages** | | **Methodology** | High-level 4-stage overview | Granular detail on each stage (SFT, RL, Rejection Sampling) | | **Behaviors** | Mentions "self-correction" | Deep dive into "Aha Moment" with case studies | | **Failed Paths** | Brief mention of PRM/MCTS | Exhaustive analysis of why PRM and MCTS underperformed | | **Distillation** | Introduced 1.5B to 70B models | Added deep ablation on distillation efficiency and data filtering | | **Hyperparameters** | Partial/Summary | **Complete Technical Specs** for reproducibility |