Reflect on James 4:13-17
© World Vision International 2000, All Rights Reserved. No part of this document or any of its contents may be reproduced, copied, modified or adapted, without the prior written consent of the author.
This passage reminds me that ambitious planning may become an obstacle to getting closer to God. When my attention is focused on achieving greater things I forget to seek what almighty God, creator of all things large and small, wants me to do.
Evaluation of transformational development should examine motives behind development planning. Why did agency staff set particular goals for their work? What is the relationship between their personal goals and the way they work with community members?
Why did community leaders and others set the development goals in the program that is being evaluated? Is the primary motive to satisfy worldly ambitions or is it to bring members of the community in closer relationship with our merciful God?
Lord, help me focus my evaluation work on values related to your kingdom on earth. Amen.
The essence of program evaluation involves sound reasoning about the linkages between good evidence and good conclusions. Internalizing the material in these books will provide a basic evaluative mindset grounded in sound reasoning regarding transformational development. Then you can read any text on evaluation theory and methods to increase your knowledge about evaluation within a Christian worldview. Or you can review evaluation texts you have read from a more holistic perspective.
There are two different emphases in program evaluation. One is to demonstrate accountability for achieving goals on time within budget. The other is to document learning about program theory and program implementation.
This paper describes a focus diagram that can be used with stakeholders to negotiate the relative emphasis on accountability and learning with the resources available for the evaluation. The diagram can also be used to show similarities and differences between different approaches to evaluation.
Click this link → Focusing an Evaluation
A transformative evaluator must understand the nature of impact evaluation very well. The fundamentals of impact evaluation should be applied to transformational development programs when assessment of evaluation feasibility justifies the cost.
The transformative evaluator must also understand technical and ethical limitations of impact evaluation applied to TD programs. Go to page 8 in Evaluating Transformational Development Outcomes for some thoughts about this.
Future posts will examine impact evaluation in more detail along with possible alternative approaches that may be more illuminating. Your comments about points to explore are welcome.
“Lessons learned” is a worthy phrase that has lost its core meaning. This article, based on Michael Quinn Patton’s insightful review in 2001, may lead to learning about lessons learned. May it be so.
Click on this link → learning lessons about lessons learned
The story in Acts 6 about feeding the widows provokes reflection on the relationship between project objectives and broader project aims. May this meditation be a blessing for you.
Click this link → Meditation2
Typically a program evaluation involves searching for knowledge that we can use to change the world so that it is more like our vision of the good life. An alternative approach to evaluation involves understanding how a program helps and hinders transformation of individuals and society. To understand transformation we need to seek faith-based knowledge as well as empirical knowledge.
I share two illustrations of how topics in TE can be organized to evaluate a transformational development program.
In addition to the content of a basic monitoring and evaluation curriculum I believe that the following research methodology topics should be learned thoroughly by anyone engaged in Transformational Development monitoring and evaluation.
Technical competence needs to be nuanced by insights that come as he or she engages in spiritual disciplines such as Bible study and reflection, prayer, fasting, etc.
- Field interview techniques for sensitive topics and in different cultures.
- Participatory approaches to inquiry.
- Focus group principles and methods. link to focus group guide
- Ordinal measures and common statistics tests for ordinal measures.
- Analysis and interpretation of Likert scale results from multiple perspectives.
- Characteristics of trustworthy qualitative data and procedures to increase trustworthiness.
- Logic of statistical inference and interpretation of statistical findings. Although statistics is advanced mathematics the logic of inference and interpretation can be understood without necessarily understanding the complex math formulas.
I know this will take time! A good evaluator continually updates his or her knowledge and adds skill sets.
The attached file is an overview of my approach to Transformative Evaluation (TE) as it has evolved in the past 15 years. The 28 pages are the foundation for posts on this site. The major sections:
- My understanding of transformational development
- Evolution of the definition of TE
- Illustrations of content that may be examined in an evaluation
- Implications for implementing this approach
May those of you who choose to dive in be provoked to see yourself as a Christian that does program evaluation in professional ways to bring glory to God rather than an evaluator that happens to be Christian.
Click link: Evaluating Transformational Development Outcomes
Some time ago I became concerned that “participatory evaluation” was being used to refer to vastly different evaluation approaches. One evaluation report called a participatory evaluation merely surveyed a sample of people, but there was no engagement by the evaluator with anyone about any aspect of the evaluation process. Apparently simply responding to a questionnaire was considered participation.
Other evaluation reports varied widely in the extent to which the evaluator engaged with others in designing and implementing an evaluation. I prepared a simple tool for rating the extent to which an evaluator engaged others while carrying out the evaluation. The tool is attached below. I welcome comments on the usefulness of the tool and other ways of determining the extent to which an evaluation can meaningfully be called a participatory evaluation.
Tool Assessment of Strength of Participation in an Evaluation