Sorting and performance evaluation with QUalitative/ORdinal information Using Multiple criteria (QUORUM)
Title: Sorting and performance evaluation with QUalitative/ORdinal information Using Multiple criteria (QUORUM)
Coordinator: Luis Dias
Dates start/end: 23.05.2005 / 30.06.2008
Entity: FCT (POCI/EGE/58371/2004)],
Synopsis: Much of the research on how to aid evaluators / Decision–Makers (DMs) in organisations addresses the important problems of seeking to satisfy multiple objectives at the same time (multi-criteria decision analysis). Some of the research addresses the same problem in group decisions and negotiations, which becomes even more complex due to the need of conciliating multiple interests. Decision support models are often difficult to put into practice, since they require that DMs provide precise values for many parameters (e.g., criteria weights). Working with models that demand less effort from the DMs is a manner of contributing to increase the use and success of formal decision-aiding methods. In particular, some methods are able to work with qualitative/ordinal information, expressed as order relations (e.g., stating that one criterion weighs more than some other, rather than providing precise numerical weights).
The QUORUM project intends to develop the state-of-the-art in decision-aiding methods working with qualitative/ordinal information applied to sorting and performance evaluation problems. Sorting (assignment, classification) problems consist in separating the evaluated alternatives (objects, persons, projects,...) into different categories. If the categories are ordered (e.g., Poor, Fair, ..., Excellent) then this can be seen as measuring performance through a rating or a qualitative note. Ordinal information multi-criteria methods will be proposed, being for instance able to reproduce sorting examples. Sorting problems with multiple DMs, who may not entirely agree but wish to reach consensus, will also be studied. There exist other methods, which measure performance using a quantitative note, e.g., using a utility function or a value scale. Such methods are usually intended for selection and ranking problems, where the alternatives are compared against each other, contrarily to sorting methods. Ordinal information methods based on aggregating multiple utility/value functions will also be studied, considering ordinal information about the alternatives’ performance scores on each criterion, as well as the weights associated with the utility functions. The extension to cases of group decisions and negotiations is also planned.
In the spirit of comparing each alternative to the best ones, efficiency measurement will also be considered in the context of evaluating performance, namely using Data Envelopment Analysis and other productivity analysis tools. An important output of these inter-related research streams will be the development of software to facilitate the usage of the methods, which will play a major role in the dissemination of the project’s results among potential users. Case studies will be performed in different contexts, to assess the usability of the methods and software. Of particular interest, due to the potential to bring new theoretical developments, is the problem of assigning tasks to agents.