An indicator elaboration worksheet has three main functions.
- Filling out the sheet helps planners to define properly the indicator and relate it to relevant development hypotheses or the theory of change underpinning the project.
- The precise description reveals the specific information that is required and the means for collecting and processing it. This allows the feasibility of using the indicator to be determined in terms of time, money and expertise required.
- Other people can understand the indicator and use it in other settings and at other times in ways that allow the readings to be comparable.
The attached file is such a worksheet. Comments and suggestions welcome.
LINK → Indicator Elaboration
Michael Scriven, a renowned evaluator for decades, developed the Key Evaluation Checklist (KEC) to guide evaluators in planning and implementing an inquiry that is a legitimate program evaluation. The KEC, which has gone through many revisions, is described in the Evaluation Thesaurus (1991, pp. 204-211). Each item on the checklist also has a separate entry in the Thesaurus. An outline is on p.83 of Foundations (1991). Another version is in Evaluation Models (1983).
The checklist identifies the multiple dimensions which must be explored prior to the final synthesis in an evaluation. Failure to do so increases the risk dramatically that the evaluation will include invalid arguments. Many iterations of the KEC are involved in the typical evaluation, generating a process of successive approximation. It is used in preparing the design, but it is also used during data collection and analysis and report preparation.
Another use for the checklist is to critique an evaluation report or a description of a completed evaluation inquiry. This link, KEC charts, gives my brief description of 14 features of legitimate program evaluation based the references to Scriven’s work. Study the references carefully if you decide to use this approach to test the value of your evaluation work.
For more information
Scriven, Michael. 1983. Evaluation Ideologies. In Madaus, George F.; Scriven, Michael; Stufflebeam, Daniel L. (Eds.). Evaluation Models: Viewpoints on Educational and Human Services Evaluation. 258-260. Sage.
Scriven, Michael. 1991. Evaluation Thesaurus (fourth edition). Sage.
Shadish, William R., Jr.; Cook, Thomas D.; Leviton, Laura C. 1991. Foundations of Program Evaluation: Theories of Practice. Sage.
A common program evaluation tool is a list of criteria or indicators of merit. Michael Scriven has given some helpful guidelines for constructing such a tool in his Evaluation Thesaurus, fourth edition, 1991, page 80-81. Examples of checklists are a checklist for program implementation and a checklist for program logic.
Here is a checklist for evaluation checklists. I welcome comments and alternative checklists.
* = required characteristic.
A checklist that is missing one required characteristic is a poor checklist. One that is missing two or more characteristics is a psuedochecklist.
( ) * The list includes all significant relevant dimensions of value.
( ) * Absolute requirements are marked and placed at the beginning of the list.
( ) * The criteria or indicators are measurable or objectively observable.
( ) * Weights are assigned to the criteria or indicators. The procedure for using weights is clearly described.
( ) There is minimal overlap between items. Overlap is clearly described. 0-3 points.
( ) A long list groups items into meaningful categories. 0-3 points.
( ) The checklist is easy to remember, understandable and easy to use. 0-3 points.
Assuming the required characteristics are satisfactory, the other three items have equal weights. The point totals can be interpreted as follows:
7-9 = good checklist
4-6 = fair checklist; think about spending some time improving it
0-5 = definitely should be improved