Reasoning in evaluation

Reasoning involves thinking through how pieces of information logically support an idea. More formally, reasoning is the process of forming conclusions, judgments, or inferences from facts or premises.

The quality of an evaluation is directly related to the quality of the evaluator’s reasoning skills. This brief post describes five areas where many of us could improve our reasoning skills. It is based on Rog (1995); read the article for more information.

Making recommendations.

Stakeholders often expect evaluators to make recommendations for program revision when the evaluators lack broad and deep expertise with the evaluand. Evaluators should develop skill in facilitating stakeholders to reason through the implications of evaluation findings.

Alternative interpretations of the data.

The reasoning you follow about dealing with missing data will affect the interpretations of that data. Also, judgments need to be made about the trustworthiness of qualitative data that can stretch your reasoning skills. Transformative evaluators need to develop these skills in depth to have credibility.

Communicating ambiguity in findings.

Evaluators need to think through how stakeholders will cognitively process the presentation of evaluation results. This is especially important if the evaluation findings will be used to decide if a program will continue. You want to present the findings in a format that accurately conveys conclusions supported by evidence but also describes complications you foresee in acting on the conclusions.

Complexity of the evaluand.

You need to reason through the program theory before you plan evaluation activities for a program. If you discover that the program is based on inadequate theory you have the challenge of encouraging stakeholders to support evaluation activities to understand the theory better. This involves understanding their reasoning that supports doing the evaluation. It also involves thinking through how to help them understand the complexity you see and how that should guide the evaluation design.

Role of the evaluator.

Stakeholders have a variety of perspectives on what an evaluator can and cannot do. Evaluators have different perspectives on what is appropriate and inappropriate professional behavior. Clear reasoning about what you as an evaluator are willing and not willing to do regarding program advocacy, highlighting positive findings, describing limitations of the evaluation, etc. can be challenging as you learn more about stakeholder expectations.

Reference

Rog, Deborah J. (1995). Reasoning in evaluation: Challenges for the practitioner. New directions for evaluation, 68, 93-100. San Francisco, California USA: Jossey-Bass Publishers.

Leave a Reply

Your email address will not be published. Required fields are marked *