Building LLMs from Scratch

Most books treat large language models as something you call through an API. This one shows you how to build one.

Building LLMs from Scratch is a single, continuous engineering case study that constructs a domain-specific "Engineering Copilot" from the ground up. Instead of disconnected snippets, it follows one coherent path through every layer of the modern LLM stack:

• Define the problem and lock the scope of a precision-critical assistant
• Curate trusted technical data and build a domain-aware tokenizer that preserves numbers, units, and formulas
• Implement a decoder-only transformer in PyTorch, step by step
• Pretrain and fine-tune for structured, repeatable reasoning
• Ground answers with retrieval-augmented generation (RAG) so the model cites real references instead of guessing
• Delegate calculations to deterministic tools for exact, auditable math
• Enforce evaluation, hallucination control, safety, and guardrails
• Optimize inference and deploy, then maintain the system as standards change

Every chapter adds working code and concrete artifacts, and every design decision is judged by one standard: would this be acceptable in a real engineering environment?

Written for software developers, engineers, and serious learners who can read code and think in systems, this book replaces hype with disciplined engineering. You do not need to be a machine learning researcher—if you understand how software systems are built, you can follow along and finish knowing how LLMs are actually engineered, not just used.