 | ||||
 |  |  |  |  |
 |  |  |  | |
|
| Evaluation - Stakeholders | ||
Introduction The honest engagement of all stakeholders in the practice of conflict management evaluation is essential to its effectiveness. However, current practice of conflict management evaluation does not always promote this. This section examines how six aspects of conflict management evaluation influence the engagement of stakeholders in the evaluation process. Most conflict management evaluations are carried out by external evaluators. This is because most conflict management evaluations are ‘top-down’ evaluations driven by elite actors. Funders generally feel that external evaluators safeguard the objectivity and transparency of evaluations. Yet the external evaluator’s professional livelihood depends on those who hire them to carry out the evaluation. Most often, this means the elite community of Donor Institutions, Private Foundations, Trusts, Specialized Agencies and NGO’s. Ultimately therefore, evaluations are likely to serve the ends, and meet the expectations of the elite community. This makes it difficult to persuade members of middle-range and grass roots communities to commit their scarce time and resources to the process of evaluation. DANIDA’s (Danish International Development Assistance) Evaluation Guidelines describe an evaluation’s terms of reference as specifying:
Importantly, because most conflict management evaluations are ‘top-down’ in nature it is generally funders who set the terms of reference for evaluations. These terms of reference may constrain the evaluator from providing feedback and recommendations of use to middle-range and grass roots communities. As a result, these communities are likely to prove uninterested in committing their scarce time and resources to evaluation. The evaluator must therefore strike a balance between ‘top-down’ and ‘bottom-up’ interests. That is, try to provide feedback and recommendations of use to the project or program being evaluated while operating within the terms of reference set by funders. In order to engage all stakeholders in the process of an evaluation, evaluators must strike a delicate balance between ‘top-down’ and ‘bottom-up’ interests. Yet, in contrast to evaluators working in the fields of development and humanitarian assistance, there is no specific training available for evaluators working in the field of conflict management. This hinders the good practice of conflict management evaluation. Recently therefore, it has been suggested that a code of ethics be developed for conflict management evaluators. It is argued that such a code would help evaluators better position themselves between the competing demands of elite, middle-range and grass roots communities. A code of ethics would also render evaluators more accountable to all of these communities thereby encouraging their engagement in evaluation. The content of such a code has yet to be developed. However, it would likely be based on the premise that it is ethically questionable for an evaluator to use people during the course of an evaluation without due regard for their subjectivity and the impact of the evaluation on their situation. Rather than distance themselves from interventions as ‘objective’ and ‘neutral’ observers, evaluators should therefore perhaps play more of a family doctor role. That is, where possible, evaluators should engage in ongoing and supportive relationships with members of middle-range and grass roots communities. The conflict context often makes it extremely difficult for evaluators to interact with middle-range and grass roots communities. For example, it may be impossible for evaluators to interview participants in projects and programs because it is simply too dangerous to travel to the areas in which they live. Moreover, evaluators are ‘outsiders.’ The concept of ‘outsiders’ has emerged from the Collaborative for Development Action’s Reflecting on Peace Practice Project. This project has defined an ‘outsider’ as an individual or agency from outside the conflict area that chooses to become involved in a given conflict. ‘Insiders’ are therefore all those who have no choice but to be involved in a given conflict. It is difficult for external evaluators to overcome the deep mistrust locals often feel for ‘outsiders’ in the short space of time they generally have available.
Even if the evaluator builds up a trusting relationship with middle-range and grass roots communities, ethical dilemmas may result. The evaluator may, for example, learn something they simply didn’t want to know. Further, the evaluator may not be able to act upon such information because it has been supplied on a confidential basis. However, rather than knowing too much it is more likely that evaluators will continue to know too little about conflict management interventions, especially those that involve highly secretive activities, for example, mediation efforts.
Evaluations often conclude with a judgement about the intervention’s overall success or failure. However, in the conflict context, different stakeholders may have competing visions as to what success and failure means. For example, for the funder ‘success’ might mean that the intervention achieved its stated goals and objectives; but for those who benefit from the intervention ‘success’ might simply mean the continued operation of the intervention. The concepts of success and failure employed by the evaluator are crucial. These concepts encourage or discourage the engagement of middle-range and grass roots actors in the evaluation. A number of innovative approaches to evaluation, including Participatory Evaluation and Action Evaluation, take all stakeholders’ concepts of success and failure into account. However, because most evaluations are ‘top-down’ in nature, evaluators most often judge projects and programs according to elite actors’ ideas of what success and failure means. Thus, middle-range and grass roots actors are often reluctant to admit any degree or instance of ‘failure’ to evaluators. These actors fear, somewhat justifiably, that to admit to ‘failure’ could be to jeopardize future funding for their project or program. ‘Top-down’ evaluations are therefore unlikely to gauge the true ‘value’ of interventions. As a result, learning opportunities may be lost and resources less effectively deployed. Elite actors therefore need to provide middle-range and grass roots actors with the freedom to acknowledge failure. They could do so by using the concept of ‘good enough’ rather than the absolute concepts of success and failure. In his article, ‘“Good enough” isn’t so bad: thinking about success and failure in ethnic conflict management,’ Marc Howard Ross explores definitions of success and failure and the concept of ‘good enough’ conflict management. For the purposes of conflict management evaluation, ‘good enough’ means recognising degrees of success and failure. Evaluations are often designed and conducted without an explicit understanding of who is going to see or make use of the evaluation. After an evaluation has been completed information may be drawn from an evaluation and manipulated by elite communities for political purposes. Elite actors may also claim exclusive ownership of evaluations, to the extent that those who participated in an evaluation are unable to see it. It is therefore not surprising that middle-range and grass roots communities fear and resist evaluation. Despite the competitiveness within the field of conflict management there is evidence to suggest that evaluations are beginning to be circulated vertically and horizontally between and within elite, middle-range and grass roots communities. For example, two international initiatives have recently been launched to collect ‘lesson learned’ from conflict management interventions: the Lessons Learned in Conflict Interventions and Peacebuilding by the European Platform for Conflict Prevention; and the Reflecting on Peace Practice Project by the Collaborative for Development Action. Such initiatives may encourage further sharing of evaluations in the future.
|