Tag Archives: evidence

Fundamental Logic of Causal Connections

I have argued that in general impact evaluation is not appropriate for community development programs. The primary purpose for impact evaluation is to judge if the evaluated program caused the documented outcomes. But collecting sufficient rigorous evidence to support that claim is complex and unreasonably expensive. I advocate using available evaluation resources for rigorous monitoring and other types of evaluation that provide valid and useful evaluation at reasonable cost.

LINK to Evaluating transformational development outcomes; includes discussion of difficulties in using impact evaluation

LINK to Creating a project monitoring system

Recently I have done more study on the nature of causation across disciplines. I believe now that there are ways we can examination causation in community development programs while practicing stewardship of meager resources for monitoring and evaluation.

Scriven’s (2008) description of General Elimination Methodology (GEM) includes the basic logic. There are two features:

  • Causation is directly and reliably observable. A causal connection is a complex pattern of perceptions (observations) that have been registered in the brain.
  • The benchmark for certainty regarding a causal connection in all disciplines is “beyond a reasonable doubt.”

Employing the GEM approach is a process of systematically ruling out plausible causes by:

  • Creating a wide ranging list of plausible causes for the effect of interest. For TE this requires deep understanding of the nature of transformation and knowledge of settings in which transformation has occurred.
  • Seeking rigorous relevant triangulated evidence for the validity of each cause on the list. Those causes that are judged invalid beyond a reasonable doubt based on rigorous evidence are eliminated. Criteria for “beyond a reasonable doubt” need to be agreed by the evaluation team and stakeholder representatives as a component of the evaluation design.

Note that experimentation is not logically required to establish a causal connection; random controlled trials (RCTs) are not required. Future posts will explore Scriven’s views on RCTs.


Scriven, Michael, A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research, Journal of MultiDisciplinary Evaluation, Volume 5, Number 9, ISSN 1556-8180, March 2008, pp. 11-24. Retrievable from http://www.jmde.com. (search for RCT)

Evaluation Rubrics

How good is this program, really?

An evaluation rubric “paints a picture of what the evidence should look like at different levels of performance” (Davidson, 2013, p. 24). It is a set of descriptions, ideally created by key stakeholders working together, of what particular conditions will look like when a program is performing excellently, satisfactorily, or poorly.

Click link for information and an example for transformation development → Evaluation Rubrics


What is different about evidence in TE

Typically a program evaluation involves searching for knowledge that we can use to change the world so that it is more like our vision of the good life.  An alternative approach to evaluation involves understanding how a program helps and hinders transformation of individuals and society. To understand transformation we need to seek faith-based knowledge as well as empirical knowledge.

I share two illustrations of how topics in TE can be organized to evaluate a transformational development program.

Continue reading