Numerical Analysis and Scientific Computing

Numerical analysis is the course where the mathematics a student has spent two years learning collides with the machine that has to carry it out — and the machine loses. A computer has no infinity, no continuum, and no exact arithmetic; it has a finite grid of representable numbers and it rounds every single operation. Everything in this book follows from that collision: how to turn a derivative into a difference, an integral into a sum, a differential equation into a recurrence, and a continuous curve into a finite list of numbers, without the errors quietly eating the answer. This is a complete treatment of that subject, built to do three jobs at once. It is a standalone textbook for the gateway course, opening with the bits of a floating-point number and the meaning of a rounding error before it asks anyone to analyze one. It is a field guide for working engineers, because every chapter carries a cost-and-accuracy table, a failure-mode index organized by symptom rather than by cause, and a survey of what LAPACK, SciPy, PETSc, SUNDIALS, and the GPU libraries actually run. And it is a preparation manual for technical interviews in quantitative development, machine-learning infrastructure, high-performance computing, and simulation, where the floating-point question and the which-factorization question come up almost every time. The difficulty ramp is deliberate and always explicitly marked. Early chapters assume calculus and linear algebra and nothing else: IEEE 754 from the bits up, the standard model of rounding, catastrophic cancellation, Taylor series and truncation error, and the distinction that organizes the whole field — that conditioning belongs to the problem and stability belongs to the algorithm. The middle is the canonical curriculum: root-finding from bisection through Newton to Brent; twelve chapters of direct linear algebra covering LU with pivoting and its growth factor, Cholesky, least squares and why the normal equations destroy accuracy, QR by Householder and Givens, the singular value decomposition, and the QR algorithm for eigenvalues as LAPACK actually implements it; iterative methods with Krylov subspaces, conjugate gradients, GMRES, preconditioning, and multigrid; interpolation, the Runge phenomenon and the Chebyshev cure, splines, the FFT, and how a library really computes exponentials and sines; differentiation, automatic differentiation in both modes, and quadrature from the trapezoid rule to Gauss and adaptive schemes; random number generation, Monte Carlo, and quasi-Monte Carlo; eight chapters of ordinary differential equations including Runge-Kutta order conditions, multistep methods, absolute stability, stiffness and the Dahlquist barriers, and symplectic integrators that stay right over a million steps; partial differential equations with finite differences, finite elements, the CFL condition, and spectral methods; and optimization from gradient descent through quasi-Newton, trust regions, and interior points. The later chapters go where graduate courses go, and every one of them is marked so it can be skipped without losing the thread. The final part is about practice rather than theory: the memory hierarchy and why the BLAS levels exist, GPUs and the reproducibility problem that parallel reductions create, how to test numerical software using manufactured solutions and observed convergence order, six famous numerical disasters and the specific engineering lesson each one carries, the interview, and a closing field guide of reference tables and decision procedures. Mathematics is written in plain ASCII throughout so it reads correctly on any device, and the code is Python with NumPy and SciPy, with MATLAB, Julia, C, and Fortran noted wherever the language changes the answer. Every method is given four things rather than three: the idea in plain language before any formula appears, the algorithm, the error analysis covering both truncation and rounding, and the failure mode. The fourth is what most textbooks leave to the exercises, and it is what a practitioner needs first.



Al-generated content disclosure: This work was created with the help and usage of artificial intelligence. All sources are cited in the bibliography section.