ReasonFlux: why reasoning templates beat token-by-token chain-of-thought
TL;DR: ReasonFlux (arXiv:2502.06772, Princeton/PKU, February 2025) introduces thought templates: compact, metadata-rich reasoning strategies agents select and compose rather than generating reasoning token-by-token. A library of ~500 templates extracted by Gemini-2.0 from math competition data provides structured exploration of the reasoning space. Hierarchical RL optimizes template sequences, not individual tokens. On MATH: 91.2%, 5.7 points above o1-preview. On AIME 2024: 56.7% vs. 44.6% for o1-preview, tying o1-mini. Follow-up ReasonFlux-F1 models are on Hugging Face; no training required.

Chain-of-thought reasoning generates thinking as a sequence of tokens. The model produces reasoning the same way it produces text: one token at a time, without any structural commitment to what kind of reasoning it’s doing. The result is verbose, inconsistent, and expensive: a model might “think” about the same type of problem in completely different ways on different runs, with no mechanism for learning from successful strategies. (Whether visible chain-of-thought reflects real reasoning at all is a separate debate; see is chain-of-thought a mirage?.)
ReasonFlux asks a different question: what if the agent picked reasoning strategies from a library rather than improvising them from tokens?
The template library
ReasonFlux (Ling Yang, Zhaochen Yu, Bin Cui, Mengdi Wang, Princeton/PKU, arXiv:2502.06772, February 2025) builds a library of ~500 thought templates extracted from mathematical problem-solving data.
Each template is not a worked example. It’s a structured reasoning pattern. An illustrative template in the paper’s format:
Template: "Algebraic Substitution for Irrational Functions"
Tags: [algebra, substitution, optimization, calculus]
Description: Transform irrational expressions by variable substitution to reduce to polynomial form
Scope: Problems involving sqrt, fractional powers, or transcendental functions in optimization
Application steps:
1. Identify the irrational expression as the target for substitution
2. Select substitution variable (t = sqrt(f(x)) or t = x^(1/n))
3. Express original variables in terms of t
4. Solve the transformed problem in t-space
5. Back-substitute to recover x
Examples: [worked example 1], [worked example 2]
The library of ~500 templates was curated by Gemini-2.0 analyzing 7.5K MATH problems and 2K Chinese math competition problems. Gemini-2.0 didn’t generate templates from scratch. It extracted the underlying reasoning patterns from existing solutions, abstracting them into reusable strategies.
The metadata is the key. Names and tags enable keyword-based retrieval: the model names the template it wants for each step, and the library returns it. The paper deliberately avoids plain embedding-similarity search, which it argues works poorly for math templates. Application steps enable instantiation: the abstract template gets filled with the specific problem’s variables and constraints.
How reasoning happens
At inference time, ReasonFlux doesn’t generate a reasoning chain from scratch. It follows a three-phase process:
graph TD
P[Problem] --> S[Plan template trajectory\nRL-trained navigator]
S --> R[Retrieve named templates\nby name and tags]
R --> I[Instantiate templates\nadapt to specific problem]
I --> T[Execute template sequence\nstep-by-step reasoning]
T --> A[Answer]
style R fill:#e8f4f8,stroke:#4a9eca
style S fill:#fff3cd,stroke:#e0a800
style I fill:#d4edda,stroke:#28a745
Planning: the RL-trained navigator analyzes the problem and outputs a trajectory of high-level steps, each tied to a template name. It generalizes from training problems to new ones by learning which sequence types work for which problem structure.
Retrieval: each named template is looked up in the library by name and tags. With about 500 structured entries, this lookup is cheap.
Instantiation and execution: an inference model fills the selected templates with the problem’s specific variables, constraints, and unknowns and solves each step. The navigator can revise the trajectory as intermediate results come back, which is where the paper’s inference-time scaling comes from.
Why hierarchical RL on template sequences matters
Most preference and RL methods for reasoning reward individual responses or individual steps. The policy optimizes locally but has no mechanism for learning that this kind of step works best when preceded by that kind of step.
ReasonFlux’s hierarchical RL trains on template trajectories: complete sequences of templates applied to a problem. The reward for a trajectory is the average accuracy an inference model reaches on a set of similar problems when guided by it. Better and worse trajectories for the same problem then form preference pairs that train the navigator with a DPO-style loss. The policy learns which template sequences lead to correct answers on which problem types, a genuinely long-horizon learning signal.
This is related to the Buffer of Thoughts approach (NeurIPS 2024 Spotlight, arXiv:2406.04271), which also proposes thought templates and a meta-buffer for storage. ReasonFlux extends it with hierarchical RL on template sequences rather than static template selection, and demonstrates significantly stronger benchmark performance.
The numbers
| Benchmark | ReasonFlux-32B | o1-preview | o1-mini | DeepSeek-V3 |
|---|---|---|---|---|
| MATH | 91.2% | 85.5% | 90.0% | 90.2% |
| AIME 2024 | 56.7% | 44.6% | 56.7% | 39.2% |
The AIME numbers are the interesting ones. AIME (American Invitational Mathematics Examination) is a set of hard multi-step problems with integer answers; non-reasoning models such as GPT-4o scored 9.3% in the paper’s table. ReasonFlux-32B at 56.7% is about 12 points above o1-preview and 17 above DeepSeek-V3, and it ties o1-mini, from a 32B model the authors trained on 8 GPUs.
MATH is a broader benchmark and closer to saturation. The paper reports the gain over o1-preview as 6.7%, which is a relative figure: the absolute gap is 5.7 points, and the margin over DeepSeek-V3 and o1-mini is only about 1 point.
Comparing to ReAct, ToT, and CoT
| Approach | Mechanism | Key limitation |
|---|---|---|
| Chain-of-Thought | Token-by-token reasoning, unstructured | No strategy reuse, high verbosity |
| ReAct | Thought → Action → Observation loop | Structures tool use, not reasoning itself |
| Tree-of-Thought | BFS/DFS over multiple paths | No persistent learning; re-derives strategies per problem |
| Buffer of Thoughts | Templates + meta-buffer | Static selection without RL-optimized sequencing |
| ReasonFlux | Template library + hierarchical RL on sequences | Requires template curation; math-domain primary validation |
ReasonFlux’s limitation is important: the template library is domain-specific. The current library of ~500 templates covers mathematical reasoning, extracted from competition math data. Extending to software engineering reasoning, scientific reasoning, or legal reasoning requires domain-specific template curation, which is non-trivial, and the paper evaluates only math benchmarks.
For practitioners building tool-using agents where the reasoning structure is sequential steps applied to retrieved context, the template approach is more naturally applicable. For open-ended creative or strategy tasks, the template library’s coverage limitations matter more.
How to use it today
No training required for standard use. Note that, as of November 2026, the template-RL ReasonFlux-32B from the paper is not published on Hugging Face. The Gen-Verse checkpoints are follow-up models built on its ideas. Three paths:
Pre-trained models (recommended for most):
Gen-Verse/ReasonFlux-F1(32B), plusReasonFlux-F1-14BandReasonFlux-F1-7B: long chain-of-thought math models trained on template-augmented reasoning tracesGen-Verse/ReasonFlux-Coder-14B,-7B,-4B: code reasoning focus
Process Reward Model variants:
Gen-Verse/ReasonFlux-PRM-1.5B: smaller and cheaper to runGen-Verse/ReasonFlux-PRM-7B: higher quality- PRMs score reasoning trajectories, so you can use them for Best-of-N selection at test time, for filtering fine-tuning data, or as an RL reward, without fine-tuning the reasoner
Fine-tuning on your domain:
- Start from a ReasonFlux checkpoint
Gen-Verse/ReasonFlux_SFT_15kdataset on Hugging Face for supervised template training- Domain-specific template curation is the bottleneck; template extraction quality determines the ceiling
For production agents that currently use ReAct or chain-of-thought for math-heavy reasoning steps, trying a ReasonFlux-F1-7B backbone is a low-effort experiment; for code, try ReasonFlux-Coder. See tool calling fundamentals for the integration patterns and tool design principles for how structured reasoning interacts with tool dispatch.
Key takeaways
- ReasonFlux replaces token-by-token reasoning with template selection and composition. ~500 templates cover math reasoning strategies; RL learns optimal sequences.
- MATH: 91.2% (+5.7 points vs. o1-preview, about +1 vs. DeepSeek-V3). AIME 2024: 56.7% vs. 44.6% for o1-preview (+12 points) and 39.2% for DeepSeek-V3 (+17 points), tying o1-mini.
- Hierarchical RL trains on complete template trajectories, not individual steps. That’s a long-horizon learning signal that improves cross-problem generalization.
- Follow-up models (ReasonFlux-F1 32B/14B/7B, Coder and PRM variants) are on Hugging Face under Gen-Verse; the paper’s own 32B model is not. No training required for standard deployment.
- The primary limitation: template library coverage is math-domain. Extension to other reasoning domains requires domain-specific template curation.
FAQ
What is ReasonFlux and what problem does it solve? ReasonFlux (arXiv:2502.06772, Princeton/PKU) replaces token-by-token chain-of-thought with structured thought templates: compact, metadata-rich reasoning strategies a model selects and composes. Hierarchical RL learns optimal template sequences rather than individual token generation. Follow-up models are on Hugging Face.
What are thought templates? Structured reasoning patterns with metadata: name, domain tags, application steps, scope conditions, worked examples. An “Algebraic Substitution” template specifies exactly how to transform irrational expressions: abstract enough to apply across problems, concrete enough to instantiate for a specific one.
How does it compare to chain-of-thought and ReAct? CoT generates reasoning as unstructured tokens with no strategy reuse and high verbosity. ReAct structures tool use, not reasoning. Tree-of-Thought branches without persistent learning. ReasonFlux encodes domain strategies into templates and uses RL to learn which sequences work, combining structure with long-horizon learning.
What are the benchmark results? MATH: 91.2% (+5.7 points vs. o1-preview’s 85.5%). AIME 2024: 56.7% vs. 44.6% for o1-preview and 39.2% for DeepSeek-V3; o1-mini also scored 56.7%.
How to use it without training? Download follow-up checkpoints from Gen-Verse on Hugging Face: ReasonFlux-F1 (32B, 14B, 7B) for math, ReasonFlux-Coder (14B, 7B, 4B) for code. Process Reward Model variants (1.5B, 7B) score trajectories for Best-of-N selection without fine-tuning. Fine-tuning data: ReasonFlux_SFT_15k on Hugging Face.
Further reading
- ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates: the paper
- Gen-Verse/ReasonFlux on GitHub: code and model cards
- Buffer of Thoughts (NeurIPS 2024): predecessor work on thought templates
- Tool calling fundamentals: how structured reasoning integrates with tool dispatch
- Tool design principles: interaction patterns for reasoning-heavy agents