Getting Program Evaluation Right

This post is an outline of the article: “Getting evaluation right: a five point plan,” by Dr. Jyotsna Puri, Deputy Executive Director and Head of Evaluation of the International Initiative for Impact Evaluation (3ie), October 25, 2012. Retrieve from https://oxfamblogs.org/fp2p/getting-evaluation-right-a-five-point-plan/

The 5-point plan is a prototype for an organization’s evaluation policy that embeds evaluation exercises throughout the program planning, implementation, and follow-up processes.

This is an outline of the five points; go to the article for details. The audience is an International Non-governmental Organization.

Point 1: Have a good theory of change/causal pathway/impact pathway or whatever you want to call it.

Theories of change are good for understanding the program, for schematics and great communication tools too. Additionally an evidence-based theory of change can help you decide where you need most investigation, where a process evaluation is sufficient, where a counterfactual analysis of outcomes is required and where a simple tracking of indicators is useful.

Point 2: Put in place monitoring and information systems. Track process and process/output and some outcome indicators across program areas.

Put together a set of detailed standard operating procedures for collecting information on process indicators. Train persons in doing the procedures and periodically verify that they are doing them correctly. At least one full-time skilled person should manage data collection and analysis.

Point 3: Think about measuring attributable change.

[FGC note. Consult with an expert about the costs and benefits of doing this right. Don’t agree to include this in an evaluation design unless the benefit is worth the cost.]

Point 4: Undertaking cost and cost effectiveness studies.

What are the priced and non-priced inputs in the project? Think about whether you want to use these projects in other places? Scale them up?

Put together a standardized template with cost categories and measurement methods. (E.g. how will you measure the cost of using good seeds for the farmer? It’s not just the cost of procurement or transportation but also the cost of additional manure, the cost of storage for seed and post-harvest produce.)

Point 5: Focus on implementation research as an important part of your program design.

Systematically document implementation factors, and put together a protocol which contains questions that are relevant to informing all stages of the evaluation. This is where participatory methods, focus groups, observational scrutiny, process research should come in, and also inform your theory of change.

Leave a Reply

Your email address will not be published. Required fields are marked *