RL for LLM Agents and Reasoning Models
-
- $9.99
-
- $9.99
Publisher Description
Reinforcement learning is how modern language models are taught to reason, use tools and act over many steps. This book builds the subject from first principles, with the mathematics worked out and the code written from scratch.
It starts with Markov decision processes and the policy gradient theorem, then develops advantage estimation, trust regions, PPO, reward models, DPO and its variants, and group-relative methods such as GRPO. It goes on to verifiable rewards, outcome versus process supervision, test-time compute, agents as partially observable decision processes, multi-turn and long-horizon training, tool use and environment design, self-play, reward hacking and collapse, evaluation and statistics, and the systems problems of RL post-training. The last chapter maps open problems.
Every chapter derives the results, proves the ones that matter, and checks them with original Python programs (NumPy, SciPy, PyTorch) on synthetic data with fixed seeds. Experiments are reported as they came out, including negative and null results. The book includes 21 chapters, five appendices (notation, a tested utility module, deferred proofs, a reproducibility checklist and worked solutions) and solutions to 167 exercises.
Written for readers who know calculus, probability and basic machine learning and want to understand why these methods work and when they fail. All data are synthetic; the listings run on a laptop CPU.
Use the worked examples to trace the choices behind a training run. The book connects reward design, policy updates, and evaluation to concrete questions such as what behavior a reward actually encourages and how to tell whether an improvement survives a change in task. The derivations and small experiments make it possible to inspect those choices step by step.
Product details and companion source code: https://www.choralsoft.com/books/rl-for-llm-agents-and-reasoning-models/
This book was generated using AI.