Accelerated AI Inference
An Independent Guide to Serving Models on NVIDIA GPUs with NIM, vLLM and SGLang
-
- $17.99
-
- $17.99
Publisher Description
Accelerated AI Inference is an independent, practical guide for engineers who serve language models on NVIDIA GPUs in production. It starts with the mechanics that decide everything else: prefill and decode, time to first token and inter-token latency, queueing, GPU memory and the KV cache, continuous batching, chunked prefill, prefix caching and quantization. It then teaches the engines in depth: vLLM, SGLang and TensorRT-LLM, plus NIM as a packaged way to run them, including an accurate account of NIM licensing for development and production. On Kubernetes, it covers the GPU Operator, MIG and time-slicing, weight distribution, health checks, rollouts, autoscaling on queue metrics, multi-node serving, disaggregated prefill and decode and cache-aware routing. It closes with multi-tenant serving, observability, cost per token and failure runbooks. One fictional running example, the model platform at a freight company, connects every chapter with worked numbers, configuration listings, checklists and a load-testing lab you can run without a GPU.
This book is an independent publication and is not authorized, sponsored or endorsed by any company whose products it mentions. All trademarks belong to their respective owners and are used only for identification. This book was drafted with generative AI tools and checked with automated tools. Tools, features and prices change often; confirm them in current documentation.