Computational Methods In Optimization

Algorithms are inventions which very often appear to have little or nothing in common with one another. As a result, it was held for a long time that a coherent theory of algorithms could not be constructed. The last few years have shown that this belief was incorrect, that most convergent algorithms share certain basic properties, and hence that a unified approach to algorithms is possible.



This book presents a large number of optimization, boundary-value and root-solving algorithms in the light of a theory developed by the author, which deals with algorithm convergence and implementation. The theory of algorithms presented in this book rests on two pillars. One of these consists of a few, very simple, algorithm models (prototypes) with corresponding convergence theorems. The great importance of these algorithm models lies in the fact that they fit almost all existing algorithms for solving problems in optimal control and in nonlinear programming. Consequently, they provide us with a highly systematic approach to the study of convergence properties of algorithms. This systematic approach is valuable to us in three ways: It guides our inventive process toward algorithms whose convergence is assured, it simplifies considerably our work in showing that a specific algorithm is convergent, and it makes the teaching of algorithms considerably less time-consuming.