Seeing is believing: A graphical reference framework for multi-criteria evaluation

  • Tânia F.G.G. Cova, Susana Jarmelo, Sandra C.C. Nunes, Sebastião J. Formosinho, J. Sérgio Seixas de Melo, Alberto Pais
  • Evaluation, October 2017, SAGE Publications
  • DOI: 10.1177/1356389017733336

Multi-criteria evaluation of Research and Development institutions

What is it about?

A method for assessment of evaluation processes, applicable to systems in which both measurable and policy-based performance criteria coexist is proposed. The main idea is that the principal aspects of the system, as extracted from the objective indicators (built from measurable observations), allow the mapping of the subjects under evaluation, without using weights, and then less objective criteria are subsequently introduced. Performance maps are presented and inspected for a set of Research and Development institutions, to illustrate examples of application of the methodology proposed.

Why is it important?

The present work deals with an important topic on the use of multiple indicators in evaluation systems: how can ‘objective’ measurable and so-called ‘policy’ criteria be combined in an evaluation system? A straightforward method is proposed for performance assessment, relying on a dynamic three-step process involving both quantitative and subjective or policy-making components. It is extremely relevant, when trying to characterize productivity, to establish a clear distinction between objective and policy dimensions. The proposed procedure underlines the necessity of setting up an efficient system, in order to assess the impact of the different criteria.


Dr Tânia F. G. G. Cova
Universidade de Coimbra

The method can have a broader application than that described in this article. In fact, it can be used in any situation in which relevant numerical indicators provide a basis for the mapping, and other criteria, of any type (even also based on metrics), are subsequently introduced.

Read Publication

The following have contributed to this page: Dr Tânia F. G. G. Cova