Approaches to the Ex-ante Evaluation of Information Systems: A Critical Review

This paper critically reviews approaches for the evaluation of investments in information
systems prior to their implementation. First, the ground for the review is prepared by
examining characteristics of evaluation, information systems and value. A classification of 54
evaluation approaches identified in English and German literature is then presented. Examples
of each class are reviewed and their advantages and drawbacks are discussed. Their use in
evaluation practice is analysed through the examination of empirical studies and directions for
future research are given. Today, a significant share of corporate funds is spent on the implementation, upgrading and
maintenance of an information system (IS). Recent studies show that in 2001 the IS budget of
companies worldwide accounted for an averaged 8.8% of total corporate revenues (cf. CSC,
2001). Consequently, a thorough evaluation of investments in information systems before,
during and after the implementation of a project is important. However, the normative
literature reports a great deal of difficulty in the appraisal of these investments (cf. Irani,
2002:11). Although IS evaluation has been an issue for both academics and managers for
more than three decades now, there are still serious concerns about how to select projects for
investments, how to control the development and how to measure benefits after the
implementation (cf. Farbey, 1999:189). This concern has been matched by increased research
activity which prevailed through two broad streams. The first stream aimed to directly
measure the payoff of IS investments for companies and came to mixed conclusions (cf.
Dehning and Richardson, 2002:8). The second stream addressed the question of how IS
investments can actually be assessed by decision-makers and particularly focussed on the
research of evaluation criteria, evaluation methods and the very nature of the evaluation
process (cf. Avgerou, 2000:570). Of late, several deficiencies in the field of evaluation methods have induced calls for in-depth
research. Academics have criticised the current state of the field as being immature and
fragmented (cf. Mahmood and Szewczak, 1999:491) and have thus demanded “an overview
of the whole panoply of evaluation methods, together with … the assumptions they depend on
...[in order to enable]... the identification of gaps.” (Farbey, Land and Targett, 1999: 205). [...]