Tag Archives: objectivity

Importance of Triangulating Evaluators

The value of including stakeholders in the evaluation team has various dimensions.

  • It can increase the usefulness of evaluations if their views and expertise are considered and integrated whenever appropriate. This requires a skilled evaluation facilitator and stakeholder commitment to substantial participation, particularly in analysis and interpretation activities.
  • Participatory evaluation methods can be used to create consensus and ownership in relation to the development activities.
  • Dialogue with stakeholders can help improve understanding and responsiveness to their needs and priorities.

In evaluation work “triangulation” is a fancy word that stands for using multiple methods to collect data, data sources, perspectives and evaluators to develop a more in-depth understanding of whatever is being studied or evaluated. Independent corroboration of a result strengthens its utility for decision making as well as extending our knowledge.

See post on triangulation … Introduction to triangulation

The triangulation dimension is not given the same degree of attention in the participatory community development evaluation literature. Participation by stakeholders can be a critical way of revealing and dealing with bias, and uncovering complexity in how the evaluated program is affecting participants and others.

Triangulation is not evaluation magic. Two common assumptions about the value of triangulation need to be examined closely.

  1. Does it eliminate bias?

The first assumption is that bias will be eliminated in a multimethod design. Although different methods can yield different understandings of the object of investigation it is difficult to conclude that those different understandings somehow neutralize any biases present. Each may not compensate for the limitations.

  1. Does it reveal true propositions?

The second common assumption is that use of triangulation will lead to convergence upon true propositions. Conflicting findings is a typical outcome of using different methods for collecting information especially if there is both quantitative and qualitative information. The evaluator must be prepared to wrestle with ambiguity creatively and to encourage others to do so. Exploration of possible explanations for differences in findings may lead to valuable conclusions that otherwise would not be included. Patton (Qualitative Evaluation Methods, 1980, pp. 329-332) recommends triangulation during analysis of the information, where different teams of evaluators or different members of the same evaluation team use different analysis approaches. Exploration of differences in conclusions may lead to additional insights about the object of evaluation.

Triangulation is not magic, but it can lead to better informed conclusions and evaluation advice.

See post on evaluation advice…  Evaluation Advice

Demonstrating Accountability for TD Programs

Different modes of inquiry are needed for demonstrating accountability for Transformational Development programs. In addition to program implementation monitoring. program results evaluation, and impact evaluation there are disciplined reflection, worldview discernment and internal audit. The attached file describes three principles for internal and external accountability and how these modes of inquiry can be applied to answer six accountability questions.  Click 88 MnE for accountability.

Note added September 2017: Programs  that are funded by a grant may be required by the funding agency to host a monitoring evaluation. This is an inquiry exercise conducted by an external evaluator or auditor to determine if the program is in compliance with the conditions agreed by the program agency by accepting the grant and using resources provided by the grant. This way of demonstrating external accountability is not discussed further in this post.

Role of Worldview in Evaluation Work

Clarifying one’s worldview is essential for the transformative evaluator who believes in being Christian first and evaluator second. The attached file emerged as I clarified my worldview. Currently I am reading several texts that may lead to further clarification, perhaps revision. I encourage you to spend a significant amount of time doing something similar.

The focus of the document is an examination of major scientific concepts related to evaluation work from a Christian perspective. I welcome suggestions and challenges.

Click LINK Worldview in Evaluation

Click category Worldview at the right for other worldview posts.