Home > Doc > The measurement of coherence in the evaluation of criteria > Introduction

The measurement of coherence in the evaluation of criteria


The problem of ranking a set of entities – projects, territorial units, price indexes, economic policy measures, and the like – while simultaneously taking account of several assessment criteria associable with the individual units, and of the importance assigned to each of these criteria, has been subject to a large number of studies which adopt highly diverse theoretical positions and analytical approaches. In order to provide a reference scheme for the exposition in the rest of this paper, it is useful to distinguish among situations in which:

- information is available regarding both the evaluation of the criteria and their importance. In this regard, here I do no more that refer to the large body of knowledge on the construction of index numbers of prices and quantities;

- information on the evaluation of the criteria and their importance is implicitly expressed in the preferences between pairs of units. In this case, I refer to one of the broadest strands of analysis, which comprises the studies and applications of the Analytic Hierarchy Process (AHP);

- situations intermediate to the former are treated, and problems are addressed using multicriteria analysis methods. My analysis is conducted within this last ambit, and it will have two purposes:

- the first is to furnish a theoretical justification to an empirical technique by which a partial pre-order, obtained using a multicriteria analysis method, is transformed into a total pre-order;[1]

- the second, which is of greater practical interest, consists in the possibility of measuring statistically the amount of modification implicitly made to the evaluations concretely expressed in the binary comparisons in order to make them coherent.

This is a set of information useful both to the decision-maker who may wish to redefine certain evaluation criteria, and to the user of the ranking obtained with this multicriteria method, in order to determine its reliability. In the next section I shall briefly outline the additive comparison scheme implicit in the multicriteria method analysed in the rest of paper, considering it in analogy with the better known and more widely used multiplicative method. The third section deals succinctly with those aspects of the PROMETHEE method necessary to define the notation and concepts used in the fourth and fifth sections, which set out the results of the paper.

By Dr Elvio Mattioli

Next: The multiplicative and additive comparison schemes

Summary: Index