Evaluation criteria

Evaluation criteria are some kind of standards by which a given project is evaluated. The evaluation criteria are directly related to the key questions and should therefore be formulated clearly and precisely. They are a kind of value system to which the evaluator refers at every stage of his research (A. Rowe,2019, pp. 496). Unlike the key questions, which are not evaluators, the evaluation criteria have a clear evaluation formula. They are a kind of prism through which the evaluator will look at the project under evaluation, indicating what is most relevant to its objectives and effects (A. Rowe,2019, pp. 496). Project evaluation is a complex and repetitive process, so the most important criteria for project evaluation are cost and time. Gathering knowledge about the status of a project is a very important element of control in terms of management paradigms (A. Rowe,2019, pp. 498).

Subjectivity of evaluation criteria

One and the same event may be assessed extremely differently if another value is used as the assessment criterion (M. Larsson, Hanberger A. , 2016pp. 190). For example, when assessing the training provided by a vocational reorientation project using the 'effectiveness' criterion, we find that the project has delivered the expected number of training courses and is therefore 'good’ (M. Larsson, Hanberger A. , 2016pp. 190-). However, when evaluating the same training using the criterion of "usefulness", we state that the project did not meet the expectations of its participants, and therefore is "bad". Depending on the criterion adopted, the assessment changes (S. C. Funnell, P. J. Rogers, 2011pp. 106-108). Therefore, the stage of selection of evaluation criteria requires close cooperation between the evaluator and the contracting authority in order to identify and select such criteria that will become the basis for evaluation of the evaluated project (A. Rowe,2012 pp.384)

The most frequently used evaluation criteria

The most popular evaluation criteria are (Højlund S., 2014, pp.428-446):

1. Relevance (Conformity),
2. Efficiences (Efficiency, Productivity, Economics)
3. Effectiveness
4. Utility ( Usability)
5. Sustainability (Durability)

In theory, there are many translations of the meanings of the evaluation criteria. In simple terms, the definition of individual criteria can be interpreted as follows (J.L. Fitzpatrick, J.R. Sanders, B.R. Worthen, 2010 pp.22-25):

• Relevance: to what extent do the objectives of the programme correspond to the needs of its addressees?
• Efficiency: to what extent were the inputs proportional to the outputs, results?
• Effectiveness: what was the impact of the project's outputs on its results? To what extent did the project results meet its objectives?
• Cost-effectiveness - the lowest possible investment with established products,
• Productivity - the highest possible products at a fixed cost,
• Cost-effectiveness - best value for money (impact),
• Utility: to what extent do the results and impacts of the programme meet the needs of its addressees?
• Sustainability: how long will the positive impacts of the programme last beyond its end?

Hierarchy of importance of evaluation criteria

Effectiveness - first, the programme should be effective (achieving impact in line with objectives, needs). Effective are projects where the results achieved (results obtained from products) are in line with the intended (objectives).

Efficiency - The programmes with the best ratio of outputs (results) to inputs are effective (J. Gysen, H. Bruyninckx, K Bachus.,2006, pp. 95).

Examples of Evaluation criteria

• Effectiveness: How well does the project meet its intended goals and objectives?
• Efficiency: How efficiently does the project use resources, such as time and money?
• Quality: How well does the project measure up to established standards?
• Satisfaction: How satisfied are stakeholders with the project's outputs, outcomes, and impact?
• Innovation: How innovative is the project in terms of its approach, methods, and use of technology?
• Collaboration: How well does the project demonstrate collaboration between stakeholders?
• Sustainability: How sustainable is the project and its outcomes?
• Impact: What is the project's impact on the people it is intended to serve?

Advantages of Evaluation criteria

• Evaluation criteria provide a basis to measure the success of a project. They provide a way to measure progress towards stated goals and objectives, and they act as a yardstick to judge the overall success of the project.
• Evaluation criteria provide an objective way to measure the quality of the project. They are not based on any subjective opinion, but instead on quantitative measures. This ensures that the evaluation is fair and unbiased.
• Evaluation criteria provide a consistent way to compare different projects. By using the same set of criteria for different projects, it is possible to compare the results objectively and accurately.
• Evaluation criteria provide feedback to the project team. By evaluating the project against the criteria, the team can assess their performance and identify areas where improvements can be made.
• Evaluation criteria can also be used to identify potential risks and issues that could affect the project. By understanding the criteria, the project team can take steps to mitigate any potential risks.

Limitations of Evaluation criteria

• Evaluation criteria are limited by the evaluator’s own biases and experiences and may be inaccurate or incomplete.
• Evaluation criteria are often subjective and open to interpretation and therefore can be difficult to apply in a consistent and fair manner.
• Evaluation criteria can be difficult to develop and maintain, since they rely on a constant review and updating process to remain relevant and useful.
• Evaluation criteria can be difficult to measure and track, since they require expertise and knowledge to properly interpret and apply.
• Evaluation criteria can be costly to implement and maintain, depending on the complexity and scope of the criteria.
• Evaluation criteria may not be applicable or applicable only in certain contexts, such as different countries or industries.
• Evaluation criteria can be difficult to generalize across different types of projects, as criteria may need to be tailored to the specific project.

Other approaches related to Evaluation criteria

• Goal-oriented approach: This approach involves setting objectives for the evaluation and then assessing the degree to which those objectives have been met.
• Contextual approach: This approach looks at the external factors surrounding the project and assesses how they influence the results.
• Systematic approach: This approach involves a structured evaluation process that takes into account all aspects of the project.
• Cost-benefit approach: This approach looks at the monetary costs and benefits associated with the project and evaluates its overall value.
• Quality assurance approach: This approach looks at the quality of the project’s outputs and evaluates them against predetermined standards.

In summary, evaluation criteria involve setting objectives, examining external factors, following a structured evaluation process, looking at costs and benefits, and assessing quality.

Evaluation criteria — recommended articles

Evaluation of the project — Training objective — PMBOK framework — Managerial controlling — Process performance — Quality plan — Preparation of project — Project evaluation — MoSCoW technique

References

• Fitzpatrick, J.L., Sanders, J.R., Worthen, B.R., (2010), Program Evaluation: Alternative Approaches and Practical Guidelines. 4th Edition, Pearson, New York,
• Funnell, S. C., Rogers, P. J., (2011), Purposeful Program Theory, The Canadian Journal of Program Evaluation Vol. 27 No. 2
• Gysen J., Bruyninckx H. and Bachus K ., (2006), The modus narrandi: A methodology for evaluating, effects of environmental policy. Evaluation 12(1)
• Højlund S., (2014), Evaluation use in evaluation systems - the case of the European Commission, Copenhagen Business School,
• Larsson M., Hanberger A. , (2016), Evaluation in management by objectives: A critical analysis of Sweden’s national environmental quality objectives system, Evaluation 22(2)
• Rowe A., (2012), Evaluation of natural resource interventions. American Journal of Evaluation 33(3)
• Rowe A., (2019), Rapid impact evaluation , ARCeconomics, Canada; University of Victoria, Canada,

Author: Marzena Rusin