A First Look at Stochastic Processes

This textbook introduces the theory of stochastic processes, that is, randomness which proceeds in time. Using concrete examples like repeated gambling and jumping frogs, it presents fundamental mathematical results through simple, clear, logical theorems and examples. It covers in detail such essential material as Markov chain recurrence criteria, the Markov chain convergence theorem, and optional stopping theorems for martingales. The final chapter provides a brief introduction to Brownian motion, Markov processes in continuous time and space, Poisson processes, and renewal theory.

Interspersed throughout are applications to such topics as gambler's ruin probabilities, random walks on graphs, sequence waiting times, branching processes, stock option pricing, and Markov Chain Monte Carlo (MCMC) algorithms.

The focus is always on making the theory as well-motivated and accessible as possible, to allow students and readers to learn this fascinating subject as easily and painlessly as possible.
Contents: PrefaceAbout the AuthorMarkov Chain ProbabilitiesMarkov Chain ConvergenceMartingalesContinuous ProcessesAppendices:BackgroundBibliography for Further ReadingSolutions to Problems Marked [sol]Index
Readership: Senior undergraduate and graduate students in Mathematics, Statistics, Economics, Finance, Computer Science, Engineering, Physics, Actuarial Science, and other fields, who already know some basic probability theory, and who want to learn the foundations of stochastic processes — including Markov chains, martingales, continuous processes, and a variety of applications.Stochastic Process;Markov Chain;Martingale;MCMC;Probability0Key Features:The material is presented directly and succinctly, without excessive detours or diversionsTheorems are stated clearly and provedThe book provides the mathematical foundations of Markov Chain Monte Carlo (MCMC) algorithms, which are the author's main research area