High-Performance LLM Inference with TensorRT-LLM

"High-Performance LLM Inference with TensorRT-LLM: Optimizing and Serving Models on NVIDIA GPUs"
Large language model deployment lives or dies by inference efficiency, and this book is written for engineers who need more than surface-level guidance. Aimed at experienced practitioners working with NVIDIA GPUs, TensorRT, Triton, and production AI systems, it provides a rigorous path to understanding how TensorRT-LLM turns model checkpoints into high-throughput, low-latency services. The focus is not on introductory machine learning, but on the hard engineering decisions that determine real-world performance.
Readers will learn how to move from model assets to runnable engines, choose among FP8, FP4-family, AWQ, and GPTQ quantization paths, and reason clearly about hardware compatibility, build-time constraints, runtime scheduling, KV cache behavior, paged attention, and in-flight batching. The book also covers native OpenAI-compatible serving, Triton deployment with the TensorRT-LLM backend, executor-oriented execution, benchmarking discipline, and performance attribution. By the end, readers will be able to design, optimize, and operate LLM inference stacks with far greater precision and confidence.
Structured for advanced readers, the book emphasizes system boundaries, trade-offs, reproducibility, and production validation rather than cookbook shortcuts. It assumes familiarity with GPU computing, Python-based ML tooling, and modern model-serving concepts, and distinguishes itself by connecting low-level inference mechanics directly to operational outcomes in real deploym