All posts by Evaluator Frank

Transformation turns you inside out; outside in

Meditate on Ephesians 4:17-28

This is a message I spoke to community development staff in Uganda in November 1998.

As an evaluator I search for truth. As a facilitator I help others search for truth, accept truth, live by truth.

In searching for truth, you need to watch out for lies, especially clever lies. In the restaurant at Laston Hotel in Masaka there is a poster that says:

Pilsner Lager
Extra Strong
It’s got what you want

This poster tells a lie. A lie that is believed as true by many. Those who live by this lie cause harm and destruction to others and themselves. In my own family I have seen the harm and destruction that addiction to alcohol can cause. If this poster were to tell the truth, it might look like this:

Pilsner Lager
Extra Strong
If you want to cause harm and destruction to yourself and others,
It’s got what you want

Now hear the word of the Lord: Ephesians 4:17-28

Verse 28 describes the fundamental indicator for transformational development. The person who steals is to stop stealing. Not only that, the person is to produce useful things with his or her own hands, that he or she might share them with others. This is transformation, from stealing to gratify the self, to being productive on behalf of others.

This passage describes the key to planning development programs that are transformational, not just enhancements to living in the world. This passage describes the key to planning programs that will inspire hope for those who are in despair. It describes the key to bringing joy to those who are in very difficult circumstances. Transformational development is based on the truth as it is taught by Jesus.

To understand the truth that is in Jesus, we need to face the fact that each and every one of us in a sinner. As told in the early chapters of Genesis, we have inherited the tendency to act as if we are God rather than being a child of God. God, the omnipotent, loving Creator of all there is, was, and ever will be, allowed us the freedom to disobey, while longing deeply that we would not disobey. But we did, and so we are sinners.

The truth that is in Jesus is that God, through boundless grace, invites us as sinners to be in His presence as forgiven sinners. If we accept this invitation, if we turn our backs on the deceitful ways of the world, we can live in God’s emerging kingdom right now. Cleansed, purified, made new in attitudes of mind, free to put on a new self, free to be like God in true righteousness and holiness.

Free to stop stealing, free to work with our hands to make something useful so that we can share it with others. Free to turn inside out and outside in, free to say “no” to the lies all around us, free to live by the truth that is in Jesus

ToR, Evaluation Design, Evaluation Plan

There are different types of plans that guide an evaluation exercise. I define three of them in this post.

Terms of reference (ToR)… Stakeholder expectations to be met by the evaluator, and evaluator expectations to be met by the client. The document is like a contract.

Evaluation design… The purpose and objectives of the evaluation exercise, and the means for achieving them. The primary function of the design is to anticipate possible alternative explanations for program results, and describe data collection and analysis methods to render implausible as many of them as feasible, leaving a stronger case that results can be attributed to program implementation.

Evaluation plan… Day-by-day assignments for each member of the evaluation team for each day of the exercise.

Comments welcome.


Research Design, Evaluation Design

The purpose of this post is to introduce the concept of research design and evaluation design to people with little or no research training or experience. There are many types of evaluation designs, but two types are key to planning impact evaluation: experimental designs and quasi-experimental designs. The characteristics of each type are described.

LINK… Research design intro

Click on this link for an outline for an evaluation design document…

Readings for Cultivating a Christian Evaluator Worldview

Crafting one’s Christian worldview is a lifelong journey that can occasion intellectual stimulation superseded only by immersing the mind and heart in Scripture study and meditation. I have learned much from this collection of resources; I commend them to you for prayerful study. Of course there are other helpful texts. Caution: there are both science and Christian materials on this topic that are more like propaganda than reasoned analysis. Seek out material that broadens your understanding from the perspectives of Scripture, church tradition, reason and your experience of God.

Continue reading

Obstacles to Critical Thinking

The attached file describes nine obstacles to critical thinking; failure to overcome them can lead to flawed reasoning. Train yourself to spot these obstacles as you do evaluation work, and train yourself in overcoming them. Your work will have a better chance of leading to changes in community development planning and implementation that make the world a better place.

LINK… Obstacles to Critical Thinking in Evaluation

Agency Framework for Evaluation Policy

An agency should have a policy or framework that applies to all evaluation work it does. This is a set of broad statements about the minimum requirements for each evaluation, regardless of diversity in the evaluands and evaluation objectives. A framework is a set of high-level standards.

This post describes seven groups of statements to guide planning a program evaluation. It can serve as the core of an evaluation policy for an agency. Brian Wolters at the Grand Rapids [Michigan] Center for Community Transformation worked with me to develop the statements.

110Agency Eval Framework-Policy

As always, constructive criticism and suggestions are welcome.

Organization Guidelines for Developing Monitoring Systems

An organization framework for something is a set of guidelines for making reasoned decisions about that something. A framework for monitoring is a set of guidelines for areas where evidence of progress, risk level, and opportunities is collected and analyzed periodically. The results are used by management to make adjustments as needed to maintain fulfillment of mission and operations consistent with core values; some results are used to meet reporting requirements with partners.

A first  step in preparing a framework for monitoring is to list all commitments to partners for providing information on a periodic basis. Add to the list the periodic information that managers require related to maintaining effectiveness, efficiency, and timely reporting as required.

The second step is to decide on the qualities that the organization values in collecting and disseminating this type of information. Prepare guidelines for maintaining these qualities.

Then monitoring systems are developed per guidelines. Here is an example of a framework with five guidelines.


Each unit/department in the organization shall have a monitoring system. The monitoring system is a collection of components. Each component consists of these elements: target, indicator, adjustment procedure, and reporting requirements.

  • Work standard, project objective, criterion for satisfactory performance, tolerable level of threat, etc. This element can be called the target for the component.
  • Description of the information needed about the target to determine if it is functioning as desired. The description shall also include instructions for collecting and analyzing that information, and the frequency. This element can be called an indicator. Note that several indicators may be used for a target.
  • Description of a procedure for initiating adjustment of factors that affect the status of the target if the indicator results are not satisfactory. This element can be called an adjustment procedure.
  • Statement of those internal persons or external partners who must receive a report for the component, and the frequency of reporting. This element can be called the reporting requirements.


Someone in each unit shall be assigned responsibility for ensuring that the monitoring system is functioning well.


The results of the monitoring are presented so that the receiver of the results can see immediately where there are potential problems that need to be resolved.


The costs for the monitoring systems are reasonable according to the different groups of stakeholders that use the results produced by the systems.


The results are needed to assist managers and staff to do good work, or to maintain good relationships with groups of stakeholders.

Typical Monitoring Systems

Project Implementation

Management receives indicator results as specified in the detailed project design. Typically there are indicators that show results of activities, outputs and outcomes; status of threats to project completion; and implementation compliance with core values.

Project Grant

The organization that provides the grant will state the reporting requirements. A monitoring system should provide the required information on a regular basis to the grant manager so that the manager can initiate adjustments as needed and can report to the granting organization accurately on time.

Financial Operations

The Finance Officer monitors expenditures by each organizational unit, and sends each unit a regular report of actual expenses against budgeted expenses. The Unit Head will resolve deviations from budget.

The Finance Officer monitors revenue against planned revenue. When revenue deviates from the projections, then senior management makes adjustments in operations.

The Unit responsible for writing grants monitors opportunities for getting an appropriate grant, and monitors progress in writing and submitting the grant.

Mission Compliance

The Board of Directors for the organization usually monitors the Organization’s results and activities against a Mission Statement, Core Values, and policies.


This is not a comprehensive list of monitoring systems. The organization will need to develop its list based on the units in the organization.

See other monitoring posts on this site for details on creating community development program monitoring systems.