All posts by Evaluator Frank

Organization Guidelines for Developing Monitoring Systems

An organization framework for something is a set of guidelines for making reasoned decisions about that something. A framework for monitoring is a set of guidelines for areas where evidence of progress, risk level, and opportunities is collected and analyzed periodically. The results are used by management to make adjustments as needed to maintain fulfillment of mission and operations consistent with core values; some results are used to meet reporting requirements with partners.

A first  step in preparing a framework for monitoring is to list all commitments to partners for providing information on a periodic basis. Add to the list the periodic information that managers require related to maintaining effectiveness, efficiency, and timely reporting as required.

The second step is to decide on the qualities that the organization values in collecting and disseminating this type of information. Prepare guidelines for maintaining these qualities.

Then monitoring systems are developed per guidelines. Here is an example of a framework with five guidelines.


Each unit/department in the organization shall have a monitoring system. The monitoring system is a collection of components. Each component consists of these elements: target, indicator, adjustment procedure, and reporting requirements.

  • Work standard, project objective, criterion for satisfactory performance, tolerable level of threat, etc. This element can be called the target for the component.
  • Description of the information needed about the target to determine if it is functioning as desired. The description shall also include instructions for collecting and analyzing that information, and the frequency. This element can be called an indicator. Note that several indicators may be used for a target.
  • Description of a procedure for initiating adjustment of factors that affect the status of the target if the indicator results are not satisfactory. This element can be called an adjustment procedure.
  • Statement of those internal persons or external partners who must receive a report for the component, and the frequency of reporting. This element can be called the reporting requirements.


Someone in each unit shall be assigned responsibility for ensuring that the monitoring system is functioning well.


The results of the monitoring are presented so that the receiver of the results can see immediately where there are potential problems that need to be resolved.


The costs for the monitoring systems are reasonable according to the different groups of stakeholders that use the results produced by the systems.


The results are needed to assist managers and staff to do good work, or to maintain good relationships with groups of stakeholders.

Typical Monitoring Systems

Project Implementation

Management receives indicator results as specified in the detailed project design. Typically there are indicators that show results of activities, outputs and outcomes; status of threats to project completion; and implementation compliance with core values.

Project Grant

The organization that provides the grant will state the reporting requirements. A monitoring system should provide the required information on a regular basis to the grant manager so that the manager can initiate adjustments as needed and can report to the granting organization accurately on time.

Financial Operations

The Finance Officer monitors expenditures by each organizational unit, and sends each unit a regular report of actual expenses against budgeted expenses. The Unit Head will resolve deviations from budget.

The Finance Officer monitors revenue against planned revenue. When revenue deviates from the projections, then senior management makes adjustments in operations.

The Unit responsible for writing grants monitors opportunities for getting an appropriate grant, and monitors progress in writing and submitting the grant.

Mission Compliance

The Board of Directors for the organization usually monitors the Organization’s results and activities against a Mission Statement, Core Values, and policies.


This is not a comprehensive list of monitoring systems. The organization will need to develop its list based on the units in the organization.

See other monitoring posts on this site for details on creating community development program monitoring systems.

Mindful Evaluation

Note August 14, 2017: Errors in the original post have been corrected.


Giving  full attention to thoughts, sensations and feelings as well as what we are doing in the present moment; nonjudgmental sustained and focused observation of self; activities to understand at a deeper level what I know and how I know it.

Mindful evaluation is not an evaluation model. It is any evaluation in which the evaluator is deeply aware of the immediate situation and his influence on it throughout the entire evaluation exercise. Mindfulness is consistent with holistic thinking; I encourage transformative evaluators to explore it as an aspect of professional development.

Cullen Puente and Bender (2015) discuss seven steps to follow to increase mindfulness, which in turn increases sound decision making as an evaluator.

1.Take time to think through your intention for incorporating mindfulness principles into your evaluation practice. Visualize yourself being more mindful in each stage of the evaluation. Set aside a specific regular time to practice various ways of being more mindful; practice, practice, practice.

Many web sites describe varieties of exercises to cultivate different aspects of mindfulness. For example, this site describes exercises for experiencing sensations from your eyes, ears, nose, mouth, etc. at a deeper level, and then reflecting on the feelings and thoughts that follow. Retrieve from

2.Cultivate your ability to pay attention while disregarding distractors.

3.Become more aware of your emotions; accept them as legitimate aspects of who you are. The more you can allow them to be part of your conscious experience, the more you understand how they shape your thinking, the less they will interfere with objective data analysis and interpretation.

Explore your perspectives on cultural, economic, political, social and linguistic matters; these perspectives influence your interpretation of cognitive input with or without your awareness. Heighten your awareness to enrich your evaluation practice and reveal personal preferences that can taint evaluation findings.

4.Cultivate self-reflexivity by asking yourself what you are doing at the moment and why. Think through what constitutes evidence that supports a conclusion. Think through different types of truth, and the many ways we have of distorting our perceptions to fit our preconceived notions about probable evaluation findings. Continually explore your understanding of what is real and not real; what is credible knowledge and what is not.

5.Practice deep listening. Good listeners can elicit more information from others. Also, they may encourage others to reflect on what they are sharing, which may lead to richer information.

6.Stay curious and open. Practice being child-like, playful with your thoughts, as you apply your evaluative skills.

7.Creatively mitigate the influence of preconceived ideas and personal biases.

For a more detailed discussion read

Cullen Puente , Anne and Bender, April. (2015). Mindful Evaluation: Cultivating Our Ability to Be Reflexive and Self‐Aware. Journal of MultiDisciplinary Evaluation. 11:25, 51-59.

Related Posts

Am I a Captious Evaluator?

I came across the word “captious” today. I did not know this word, so I looked for definitions. It is used in two ways: (1) trivial fault-finding; (2) intended to provoke argument. I thought about applying these notions to evaluation practice.

A captious evaluation report is focused on stating every flaw in the evaluand, no matter how trivial. Reading the report raises questions about the evaluator’s capacity to be objective. Can this evaluator see anything good, or is he programmed to only see flaws?

In a captious conversation statements are made to provoke arguing rather than discussing or dialoging. Person A: “it’s not practical to have community members participate in planning this evaluation.” Person B: “In what way is it not practical?” Person A: “It just isn’t.” Person B: “Have you been in an evaluation where there was community participation?” Person A: “No, because I’m a practical evaluator.” Person A is making captious statements rather than engaging in a constructive conversation.

I wonder how often I have acted as a captious evaluator.

TD Is Engaging in Spiritual Warfare

Talking about the role of spiritual warfare in Transformational Development makes me uncomfortable. I have an hypothesis, however, that failure to see transformation occur in community development is due in part to failure to understand the role of spiritual warfare in communities. This lack of understanding is an obstacle to seeing and confronting evil forces undermining development efforts to enable transformation. So I am working at preventing my discomfort from avoiding the topic.

The attached file, 1 page, is an initial attempt to describe the role of spiritual warfare in community development. I will prepare more posts on this topic as led by the Holy Spirit; eventually I will explore implications for m&e.

Yancey spiritual warfare

Evaluation Model Framework for an Organization

An evaluation model framework is a set of guidelines for selecting or modifying an evaluation model. When an organization has such a framework, reasoned decisions can be made about which models are appropriate. Some elements of a model may not be consistent with the framework; those elements should be modified. Or parts of one model and parts of another model put together fit the framework better than either model.

This 2-page paper illustrates six topics to include in an evaluation framework… Constructing an Evaluation Framework

Two other post discuss evaluation frameworks and models.… A review of the concept of an evaluation model, and a discussion of models with some similarities to transformative evaluation.… A description of the features of transformative evaluation based on an evaluation model template.

World Vision Resources for TD

World Vision (go to is a global Christian relief, development and advocacy organisation dedicated to working with children, families and communities to overcome poverty and injustice. It is a large complex organization that has produced many tools for facilitating, monitoring and evaluating Transformational Development. Full disclosure: from 1985-2011 I worked as an evaluator in WVI, and was involved at different levels in developing some of the tools in their early versions.

I recommend that you explore the large number of resources that World Vision has made available to anyone involved in some way with community development intended to create opportunities for individual and social transformation as revealed in scripture. Go to

I will highlight some resources from a TE perspective in other posts.


Exercise: Understanding Spiritual Discernment

The attached file is an exercise to assist people to clarify their understanding of spiritual discernment. The file was revised 26 JAN 2017.

103 Understanding Spiritual Discernment Feb

It can be used by evaluation team members to facilitate consensus on data collection and analysis for this aspect of an evaluation.

It can be used with stakeholders as the evaluation is planned.

It can be used to facilitate use of spiritual disciplines that guide the activities of the evaluation team.

May your ministry be more pleasing to God as you develop deeper understanding of discernment, and internalize discerning practice.

Accord Network: Principles of Community Development That Can Create Opportunities for Transformation (TD)

In 1978 twelve agencies formed a network based on this vision: “all Christian relief and development professionals and agencies base their initiatives on biblical principles and work to re-engage the Church in holistic ministry among the poor and needy.” The network incorporated in 1978 as AERDO (The Association of Evangelical Relief and Development Organizations). In 2010, the membership of 75 agencies voted to change the name to Accord Network.

“At Accord Network we create a community where Christ-centered organizations, churches, and individuals leverage their combined learning to achieve the best in relief and development.

We help our members reach their full potential by operating in community—sharing knowledge, skills, and support with one another. Our members are not limited to their own learning curve—they have ready access to the collective knowledge of 75 organizations that collectively leverage over $4 billion of resources annually.

  • We create standards for high-quality work
  • We learn from each other
  • We advocate in Washington DC for effective development”


Accord Network has adopted eight principles of excellence in integral mission. Visit their website for more detail. As an exercise, propose indicators or rubrics for monitoring program planning and implementation against these eight principles.

  1. Our Christian faith is at the center of our identity, motive and manner of being.
  2. We acknowledge the reality and significance of the spiritual realm.
  3. The Church is central.
  4. Transformational practices start with us.
  5. We recognize the whole system of poverty.
  6. In our relationship journey with the church, our local partners, and the community, we enter as guests, co-labor as partners, and continue as friends.
  7. We support local communities and churches in measuring all that matters.
  8. We tell the story with integrity.

May reflection on these principles strengthen your ability to facilitate transformation through your ministry.

Becoming Aware of God’s Handiwork

Ephesians 2:10

For we are God’s handiwork [or workmanship], created in Christ Jesus to do good works, which God prepared in advance for us to do. NIV

My personal reflection (01/14/2017)

In 1965 I was drawn to program evaluation work by complex circumstances; at the time I envisioned myself as a researcher/scholar in the field of cognitive psychology. I applied my research skills to a few small evaluation tasks, but did not see myself as an evaluator.

There was a 15-year period between my initial work and accepting a position where I had major responsibility for evaluation. In that period I experienced a clear calling to serve the poor as a Christian. I knew in my heart that God had prepared me to be his servant; I accepted Jesus Christ as my Lord and Savior; I did effective work in community development outside my research training. During that period I did one program evaluation and one year of institutional research; otherwise I saw myself as a facilitator of community development through the church.

Twenty years after my first evaluation exercise I was called to serve as an evaluator in a Christian relief and development organization. It took me months of prayer and consultation with my wife to say yes, for it meant giving up my vision of being a university researcher/scholar. For thirty years since that “yes” I have experienced being guided by the Holy Spirit in facilitating evaluation exercises of community development programs. For the past twenty years I have been guided by the Spirit and my spiritual disciplines in developing the theory and practice of what I call Transformative Evaluation (TE).

I humbly see myself as God’s handiwork, poetic workmanship, through guidance of the indwelling Holy Spirit fulfilling, with many defects in attitudes and behavior, the divine plan for my life. I am grateful beyond words for God’s love, grace and mercy to be loving God and neighbor through developing TE.

Another kind of handiwork (10/30/2017)

Several years ago we decided to make a cat bathroom. I put linoleum on an unused closet floor for the litter pans. I tried to replace the molding so that it looked like it was done by a good carpenter, because I am not a good carpenter. I used a miter box to saw the ends; I drilled starter holes for the nails so that they were spaced evenly; I used a nail set and then spackled the nail heads; finally, I carefully painted the molding so that you could not see where it was fastened to the wall.

I did this as a spiritual exercise. No one will see it, except the cats and us when we change the litter. And God, who created me to do good works… not as a carpenter, but as a husband to do chores in a way that gives my wife pleasure. I am not a carpenter, but I can do small projects that cause someone to smile. I wonder what other good works God has prepared in advance for me to do?

Your personal reflection

I invite you to prayerfully reflect on your understanding of God’s handiwork in you, being open to prompting from the Holy Spirit.

Understanding Sustainable Community Development (TD)

This post is an exercise to facilitate understanding of community development sustainability from the perspective of an agency’s mission. During a workshop staff members can complete the exercise individually, and then the group can identify common threads in the responses and reflect on implications for programming. If there are major differences in responses, reflect on the underlying concepts and reasoning. Throughout the exercise participate in appropriate spiritual disciplines. Seek God’s wisdom in understanding this most important aspect of community development.

Exercise…100Understanding Sustainable Community Development exercise