Category Archives: TE Methods

Formulating Recommendations in TE

This post consolidates and expands previous discussions of recommendations on this site. This is my current thinking (November 2019) about this central element of TE.

Defining recommendation

I define a recommendation as a statement offered as worthy of acceptance or approval by stakeholders.  Based on available evidence, knowledge and experience the evaluator is saying that it is reasonable for stakeholders to implement the action/change included in the statement.

Recommendations should be developed in consultation with various stakeholder groups. Ideally, conflicts that emerge should be resolved through reflection as described below. If they cannot be resolved the core of the disagreement should be included in the report respectfully and fairly.

It is essential to keep in mind, however, that as stakeholders consider the recommendation in light of other factors they may decide reasonably not to implement the recommendation. If stakeholders are involved in the interpretation of evaluation results before the report is prepared, the report is less likely to contain recommendations that are not implemented.

In transformative evaluation there are two types of recommendations.

  • The first type is what you expect to see in any program evaluation report: description (based on evidence) of changes to implement to improve the chances that program goals and objectives will be achieved efficiently and effectively.
  • The second type is description of changes to increase the likelihood that the program will enable individual or social transformation regardless of the program goals and objectives.

Primary features of a recommendation

There are four features of a sound recommendation in TE reporting.

  1. The proposed change is based on verifiable, credible evidence as reported to stakeholders.
  2. A recommendation is actionable if it describes change that can implemented to improve the evaluated program.
  3. A recommendation is feasible if relevant stakeholders agree that the resources needed to implement the proposed change are available or can be acquired in a reasonable period.
  4. A TE recommendation is a proposed change that is a consequence of prayerful critical reflection. In-depth prayerful reflection will enhance both types of recommendations.

Using reflection to formulate recommendations that can make a difference.

In transformative evaluation the recommendations should be an outcome of prayerful reflection regarding all conclusions along with insights about how God is at work in the communities. In TE both individual and group reflection are important. In either case the primary purpose is to discern what God is calling forth in this situation from those who seek to live as they were created to live. In brief, reflection is an empowering activity that is essential for learning how to facilitate transformation through development work with communities and partners.

Facilitating a reflection event is much more than chairing a meeting.

Mutual trust and respect among the participants is required. This usually means that they have had positive interactions prior to the reflection event.

It is best if someone not involved with the program facilitates the event so that all key partners can participate fully.

Ideally the facilitator understands reflection from a variety of perspectives, such as

Freire’s (1996) approach to pedagogy (discussed below) and action learning. The sections that follow unpack this brief description.

Reflection is a dialogical activity.

Paulo Freire’s description of critical reflection is an excellent guide to planning reflection activities that lead to recommendations that really matter. Richard Schaull in the introduction to Freire’s Pedagogy of the Oppressed (1996) summed up Freire’s concept of critical reflection as comprised of dialogical activity, vocationally focused discussion and critical thinking that empowers individuals and groups to initiate humanizing change.

Freire’s starting point for understanding reflection is:

A person’s ontological vocation is to be a Subject who acts upon and transforms his world, and in so doing moves towards ever new possibilities of a fuller and richer life individually and collectively. In other words, meaningful “being” beyond merely existing involves making a positive difference in the world as an individual and in concert with others.

Don’t let “dialogical” distract or puzzle you. It simply means that reflection is a learning activity that involves talking and acting with other people on something that is important to everyone engaged in the reflection activity. The learning activity is much more than storing information. It involves identifying what is helpful and unhelpful in changing one’s personal and social context so as to experience living at its fullest.

Reflection is focused discussion.

Freire views the world as a problem to be worked on and solved. The world is the material that a person uses to create history and to overcome that which is dehumanizing at any particular time and place. Reflection is discussion that has such a vocational focus, or a focus on what God created us to do and be.

Focused discussion becomes reflection as emphasis is placed on identifying barriers to individual and social transformation. Program reflection is focused on specific information about the program rather than individual impressions or concerns. Documented quantitative and qualitative data are important for effective program reflection. Acknowledge and appreciate the good things that are happening, but concentrate on resolving issues or concerns.

For the Christian transformation occurs as God works within a particular context to redeem persons and groups from the consequences of sin. God often works through people to create opportunities for transformation, but transformation itself is God’s work. Focused discussion becomes reflection as the participants move beyond discussing what they can do to solve a problem to discussing their perceptions of the activity of God in their situation and how they can align their actions with it.

Reflection involves critical thinking.

Time should be given to enhancing critical thinking skills in each reflection event. Freire believed strongly that every person is capable of looking critically at his world in a dialogical encounter with others. Given the proper tools a person can gradually perceive his or her personal and social reality as well as the contradictions in it. He or she can become conscious of personal perceptions of that reality and deal critically with them. Each person, through such a process, wins back the right to say his own word, to name the world.

Critical thinking involves probing ideas from different perspectives looking for different understandings of what is real in the context. The probing must be done respectfully, assuming that each person’s perspective has value as the group describes the parts of reality that they have in common.

Reflection can lead to empowerment.

When people are able to name the world from the perspective of being fully human they are empowered with a new sense of dignity and hope. They are more likely to take upon themselves the struggle to change the structures of society which until now have served to oppress them.

Prayer throughout the reflection exercise is very important. For the Christian empowerment is based on understanding the will of God for the individual or group and intentionally aligning behavior with it.

Objectivity in TE

Objectivity is an essential feature of evaluation. It is more complex in social inquiry than people unfamiliar with relevant literature may realize.

In this post I make the case for objectivity in TE, which is social inquiry, as a process that  involves multiple objective observers who actively engage in rigorous dialogue concerning appropriate data collection, analysis, interpretation, and reporting throughout the inquiry process. The process itself is thoroughly documented and submitted to qualified persons for critique. Bias control and scope of cultural differences, subjected to competent independent verification, are the essential elements of objectivity in TE.

My case for this description is in the attached file. Constructive criticism is welcome.


Fundamental Logic of Causal Connections

I have argued that in general impact evaluation is not appropriate for community development programs. The primary purpose for impact evaluation is to judge if the evaluated program caused the documented outcomes. But collecting sufficient rigorous evidence to support that claim is complex and unreasonably expensive. I advocate using available evaluation resources for rigorous monitoring and other types of evaluation that provide valid and useful evaluation at reasonable cost.

LINK to Evaluating transformational development outcomes; includes discussion of difficulties in using impact evaluation

LINK to Creating a project monitoring system

Recently I have done more study on the nature of causation across disciplines. I believe now that there are ways we can examination causation in community development programs while practicing stewardship of meager resources for monitoring and evaluation.

Scriven’s (2008) description of General Elimination Methodology (GEM) includes the basic logic. There are two features:

  • Causation is directly and reliably observable. A causal connection is a complex pattern of perceptions (observations) that have been registered in the brain.
  • The benchmark for certainty regarding a causal connection in all disciplines is “beyond a reasonable doubt.”

Employing the GEM approach is a process of systematically ruling out plausible causes by:

  • Creating a wide ranging list of plausible causes for the effect of interest. For TE this requires deep understanding of the nature of transformation and knowledge of settings in which transformation has occurred.
  • Seeking rigorous relevant triangulated evidence for the validity of each cause on the list. Those causes that are judged invalid beyond a reasonable doubt based on rigorous evidence are eliminated. Criteria for “beyond a reasonable doubt” need to be agreed by the evaluation team and stakeholder representatives as a component of the evaluation design.

Note that experimentation is not logically required to establish a causal connection; random controlled trials (RCTs) are not required. Future posts will explore Scriven’s views on RCTs.


Scriven, Michael, A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research, Journal of MultiDisciplinary Evaluation, Volume 5, Number 9, ISSN 1556-8180, March 2008, pp. 11-24. Retrievable from (search for RCT)

Ten Seed Technique (TST)

The ten seed technique is a participatory monitoring and evaluation tool that documents a group’s perceptions about a wide range of topics and issues. The attached file describes and illustrates the technique. Click the link → Ten Seed Technique

Search the site using keyword “ten seed” to see examples of TE indicators based on TST.

Review the post on evaluating TD outcomes to keep in mind the essential characteristics of TE.  Click the link → TE

From Conclusions to Provocative Propositions

Evaluation conclusions and recommendations often do not inspire imaginative thinking about the future.  A provocative proposition describes what the organization would look like if it were designed to maximize the best practices, the peak experiences, in a sustainable way. I have prepared a hypothetical case study to illustrate how formulating provocative propositions can be included in an evaluation exercise. The process can encourage organizational transformation that can be used by God for social transformation. Click LINK Provocative Propositions

Evaluation Rubrics

How good is this program, really?

An evaluation rubric “paints a picture of what the evidence should look like at different levels of performance” (Davidson, 2013, p. 24). It is a set of descriptions, ideally created by key stakeholders working together, of what particular conditions will look like when a program is performing excellently, satisfactorily, or poorly.

Click link for information and an example for transformation development → Evaluation Rubrics


Have you read these books?

The essence of program evaluation involves sound reasoning about the linkages between good evidence and good conclusions. Internalizing the material in these books will provide a basic evaluative mindset grounded in sound reasoning regarding transformational development. Then you can read any text on evaluation theory and methods to increase your knowledge about evaluation within a Christian worldview. Or you can review evaluation texts you have read from a more holistic perspective.

Continue reading Have you read these books?

Working with stakeholder expectations

Meditate on Luke 18:31-34 

Jesus steadfastly set his face toward Jerusalem, for the journey to Jerusalem was God’s purpose for his life.  “The great thing to remember is that we go to Jerusalem to fulfill God’s purpose, not our own… in the Christian life we have no aim of our own (Chambers).”

Transformational development program stakeholders have a general purpose: to enable individual and social transformation by achieving the program goals. A key objective of transformative evaluation is to assist program stakeholders to understand at a deeper level what really matters in development. As an evaluator I will examine evidence related to accomplishment of program goals. But as a transformative evaluator I will work with stakeholders to facilitate humble exploration of their expectations in the light of God’s word.

Continue reading Working with stakeholder expectations

Discerning conclusions

A conclusion is the answer to a specific question included in the evaluation design, or an interpretation of outcomes related to a particular information need. 

In general discernment is the ability of an evaluator to see what is not obvious to the typical stakeholder. For transformative evaluation discernment goes beyond this to see what is right and wrong, good and evil, in the program from God’s perspective.

For a more detailed discussion go to page 19 in Evaluating Transformational Development Outcomes on this site. I encourage you to comment, perhaps starting a dialogue that will help me and others deepen understanding of this topic/issue.