Introduction
'Lessons Learned' and 'Best Practices'
Doing Evaluation Well
Politics of Evaluation

Introduction

As an abundant source of lessons learned and as a means to the end of best practices, evaluation helps improve the performance of conflict management interventions. The practice of conflict management evaluation can therefore serve the interests of elite, middle-range and grass roots communities. However, evaluation is an inevitably political act - it is always for someone and for some purpose. Politics may undermine the effectiveness of conflict management evaluations.

'Lessons Learned' and 'Best Practices'

'Lessons learned' is a fashionable term in the field of conflict management. It implies 1) the gathering together of knowledge about the way things are done, and 2) the drawing of conclusions from this knowledge. Evaluation is therefore an abundant source of lessons learned. For example, an evaluation can show which aspects of a project or program worked well or badly. Conclusions can then be drawn as to the effectiveness of the strategies used to implement the project or program.

Evaluation is also an important means to the end of best practices. A recent short paper by UNESCO defines the term ‘best practices’ as the extraction of excellent examples from case studies, and the use of these excellent examples in the selection and development of new projects and programs. By selecting, studying and circulating ‘best practices’ a bridge is constructed between empirical evidence, research and practice. Evaluation is both a case study of a project or program and a source of excellent examples. For example, an evaluation might show how a project or program achieved a certain goal or objective. Evaluation therefore facilitates the development of best practices.

Both ‘lessons learned’ and ‘best practices’ help improve the performance of conflict management projects and programs. The practice of conflict management evaluation therefore serves the interests of elite, middle-range and grass roots communities. For example, the improved performance of a conflict management project or program provides funders with a bigger bang for their buck; helps practitioners achieve their goals and objectives; and provides greater benefits to target groups. Both ‘top-down’ and ‘bottom-up’ evaluation may therefore be carried out to serve the purpose of improving the performance of projects and programs.

Doing Evaluation Well

It is useful to distinguish between ‘top-down’ and ‘bottom-up’ evaluation in order to examine how the different actors and forces driving an evaluation affect the functions it serves. However, in order to be effective evaluation must, at least to some degree, serve the interests of all three of Lederdach’s communities. This is because a successful evaluation demands that information and resources flow vertically and horizontally between and within groups or communities.

An evaluation may serve the interests of all three of Lederdach’s communities by coincidence. For example, elite, middle-range and grass roots actors all share an interest in improving the performance of projects and programs. It is more likely however, to happen by necessity. For instance, a Donor Institution may want to evaluate a project in order to determine its cost-effectiveness. However, if the evaluation is perceived as threatening the financial future of the project, middle-range and grass roots actors are unlikely to engage honestly in the process.

In order to avoid this, the Donor Institution would likely try to take account of at least some of the middle-range and grass roots actors’ interests in the evaluation’s terms of reference. For example, they might specify that the evaluation will provide some useful feedback to the project or program. This would encourage the honest engagement of middle-range and grass roots actors in the evaluation process.

Similarly, a ‘bottom-up’ evaluation may be launched by middle-range and grass roots actors to build up their evaluative capacity. However, every evaluation entails costs. These range from the cost of hiring an evaluator, to the cost of practitioners committing their time and resources to an evaluation. It is therefore likely that the cost of a ‘bottom-up’ evaluation will have to be justified to the elite actor who funds the project or program. This means funders’ interests must be taken into account. Thus, even ‘bottom-up’ evaluations are likely to serve elite interests.

Politics of Evaluation

In the conflict context, doing evaluation well matters in pragmatic terms because poor interventions cost lives. Doing evaluation well also matters in ethical terms because it helps weed out poor interventions before they exact such a cost. Ultimately, therefore the practice of conflict management evaluation should be driven by the field’s sense of ethical responsibility to grass roots communities.

However, evaluation is inevitably a political act – evaluation is always carried out for someone and for some purpose. At times this may undermine the integrity of the evaluation. For instance, the expectations of an elite actor driving a ‘top-down’ evaluation may influence the findings of an external evaluator hired to evaluate a project or program. The evaluator may fear the funder who initiated the evaluation will ‘shoot the messenger’ if they deliver a highly critical and negative evaluation to them.

Evaluators may fear this, because elite actors do not always want to hear bad news. Funders may, for example, have insufficient resources to implement the changes a critical evaluation might recommend. Or, the project or program may simply have fallen down the funder’s list of priorities. There is therefore an incentive for external evaluators to tell the elite actors who pay their wages what they want to hear.

Politics may also undermine the integrity of an evaluation because evaluations are an important source of validation for those who implement and administer conflict management interventions; those who fund conflict management interventions; and for the field of conflict management as a whole. Positive evaluations provide the ‘success stories’ that practitioners need to secure the financial future of their projects and programs; the ‘success stories’ funders need to show they are fulfilling their mandates; and the ‘success stories’ the field of conflict management needs to demonstrate its worth. However, the ethical dimension of conflict management work demands that safeguards be put into place to prevent the practice of evaluation being used for PR purposes.