Monthly Archives: May 2015

Fundamental Logic of Causal Connections

I have argued that in general impact evaluation is not appropriate for community development programs. The primary purpose for impact evaluation is to judge if the evaluated program caused the documented outcomes. But collecting sufficient rigorous evidence to support that claim is complex and unreasonably expensive. I advocate using available evaluation resources for rigorous monitoring and other types of evaluation that provide valid and useful evaluation at reasonable cost.

LINK to Evaluating transformational development outcomes; includes discussion of difficulties in using impact evaluation

LINK to Creating a project monitoring system

Recently I have done more study on the nature of causation across disciplines. I believe now that there are ways we can examination causation in community development programs while practicing stewardship of meager resources for monitoring and evaluation.

Scriven’s (2008) description of General Elimination Methodology (GEM) includes the basic logic. There are two features:

  • Causation is directly and reliably observable. A causal connection is a complex pattern of perceptions (observations) that have been registered in the brain.
  • The benchmark for certainty regarding a causal connection in all disciplines is “beyond a reasonable doubt.”

Employing the GEM approach is a process of systematically ruling out plausible causes by:

  • Creating a wide ranging list of plausible causes for the effect of interest. For TE this requires deep understanding of the nature of transformation and knowledge of settings in which transformation has occurred.
  • Seeking rigorous relevant triangulated evidence for the validity of each cause on the list. Those causes that are judged invalid beyond a reasonable doubt based on rigorous evidence are eliminated. Criteria for “beyond a reasonable doubt” need to be agreed by the evaluation team and stakeholder representatives as a component of the evaluation design.

Note that experimentation is not logically required to establish a causal connection; random controlled trials (RCTs) are not required. Future posts will explore Scriven’s views on RCTs.

Reference

Scriven, Michael, A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research, Journal of MultiDisciplinary Evaluation, Volume 5, Number 9, ISSN 1556-8180, March 2008, pp. 11-24. Retrievable from http://www.jmde.com. (search for RCT)

Ten Seed Technique (TST)

The ten seed technique is a participatory monitoring and evaluation tool that documents a group’s perceptions about a wide range of topics and issues. The attached file describes and illustrates the technique. Click the link → Ten Seed Technique

Search the site using keyword “ten seed” to see examples of TE indicators based on TST.

Review the post on evaluating TD outcomes to keep in mind the essential characteristics of TE.  Click the link → TE