Monthly Archives: November 2016

Evaluation Advice

The information offered by an evaluator to stakeholders to guide or direct their decision making constitutes technical advice. Several terms describe different types of technical advice included in evaluation reporting. It is helpful to differentiate between conclusion, recommendation, suggestion, issue, concern, and opinion.

Conclusion: the answer to a specific question included in the evaluation design, or an interpretation of outcomes related to a particular information need. A conclusion must be logically consistent with available evidence, as well as relevant knowledge and experience. Sometimes the evaluation will conclude that the question cannot be answered by the available evidence, knowledge and experience. If this is the case, it should be clearly stated along with a description of what would be required to answer the question.

Recommendation: a statement offered as worthy of acceptance or approval by stakeholders. Based on available evidence, knowledge and experience the evaluator is saying that it is reasonable for stakeholders to adopt the action included in the statement. It is essential to keep in mind, however, that as stakeholders consider the recommendation in light of other factors, stakeholders may decide reasonably not to adopt the recommendation. If stakeholders are involved in the interpretation of evaluation results before the report is prepared, the report is less likely to contain recommendations which are not adopted.

Suggestion: a statement offered for consideration because it is associated with something of interest in the evaluation. Although the evaluator deems this information as desirable or fitting for consideration (otherwise it would not be included as advice), the evaluator believes further study is needed before stating the information in the form of a recommendation.

Issue: a statement of something about which reasonable persons can disagree. Sometimes the attempt to answer a specific question during the evaluation uncovers different perspectives about it, each of which could be regarded as an answer to the question. Detailed description of the perspectives, and the values associated with each, is valuable advice, even when no specific suggestion or recommendation for action can be supported by available evidence, knowledge or experience. This description may help stakeholders resolve or reduce any conflict created by the issue.

Concern: a statement of something which is important to someone because it poses a threat, or it is believed that it will lead to an undesirable consequence, or empirical verification is desired. Confirmation or disconfirmation of a concern, or illumination of a concern, is valuable advice for stakeholders.

Opinion: a statement of plausible consequences of studying or not studying a suggestion; adopting or not adopting a recommendation; confronting or ignoring an issue or concern.

This Evaluation Stinks!

The program manager was reading the evaluation report when he exclaimed, “This evaluation stinks!” The evidence showed little or no progress for most of the program objectives. There were numerous recommendations for improving the chances for accomplishing the objectives, but the cost would be high. The report was bad news for the manager, and he reacted defensively by lashing out at the evaluation.

A similar program was evaluated in a different setting. The results for achieving objectives were similar, but the evaluator had used a transformative approach. Throughout the exercise the manager had participated in several sessions of reflecting on relevant passages of scripture with the evaluator and other stakeholders. And data had been collected and analyzed on program alignment with the agency mission; the evidence showed good alignment.

Before she shared the report with manager and others, they reflected on 2 Corinthians 2:14-17. They spent time discussing savoring the fragrance when Christ was present, and how people could decide to be triumphant in their situation or to be defiant. People could accept bad news in the eyes of the world as good news in the kingdom. Or not.

The manager read the report and commented first on the results for alignment. “I feel like we are living near a fragrant field of flowers, but our land where we are working has a lot of weeds choking the “crops” we have planted. The achievements are disappointing, but the nearby fragrance is encouraging. We need to prayerfully re-examine our objectives and how we using our resources, rejoicing in the promised triumph that comes when we are faithful.”

The evaluation approach and the reporting style can be a ministry of death or a ministry of life. Work towards an evaluation exercise that offers life going forward, not by obscuring worldly failure, but by facilitating awareness of the fragrance of Christ that transforms attitudes toward worldly failure. Praise God for revealing wisdom in scripture that surpasses worldly wisdom.

Transformative Evaluation Overview–6 Slide Shows

This post contains links to six brief slide shows that provide orientation to the distinctive features of Transformative Evaluation. The shows are in PDF format.

94orientation1 TE overview…Brief description of each critical feature. Includes a group exercise to see if people in an agency want to explore implications for the agency.

94orientation2 core belief…Description of the core belief that justifies TE as an approach to evaluation for a Christian agency.

94orientation3 TE theory of program change…Description of the theory of change in individuals and institutions for transformation development, which is the primary evaluand for TE.

94orientation4 TE overview…Description of three critical features of objectivity in TE. There is a more detailed description of objectivity at objectivity-in-te.

94Orientation5 Mission Results in TE…Description of mission results that can be achieved through TE.

Orientation6 Spiritual Disciplines in TE…Description of use of spiritual disciplines in TE.

Search for other posts related to topics that interest you. As always, constructive critiques are welcome.