Monthly Archives: June 2017

Organization Guidelines for Developing Monitoring Systems

An organization framework for something is a set of guidelines for making reasoned decisions about that something. A framework for monitoring is a set of guidelines for areas where evidence of progress, risk level, and opportunities is collected and analyzed periodically. The results are used by management to make adjustments as needed to maintain fulfillment of mission and operations consistent with core values; some results are used to meet reporting requirements with partners.

A first  step in preparing a framework for monitoring is to list all commitments to partners for providing information on a periodic basis. Add to the list the periodic information that managers require related to maintaining effectiveness, efficiency, and timely reporting as required.

The second step is to decide on the qualities that the organization values in collecting and disseminating this type of information. Prepare guidelines for maintaining these qualities.

Then monitoring systems are developed per guidelines. Here is an example of a framework with five guidelines.

Guideline

Each unit/department in the organization shall have a monitoring system. The monitoring system is a collection of components. Each component consists of these elements: target, indicator, adjustment procedure, and reporting requirements.

  • Work standard, project objective, criterion for satisfactory performance, tolerable level of threat, etc. This element can be called the target for the component.
  • Description of the information needed about the target to determine if it is functioning as desired. The description shall also include instructions for collecting and analyzing that information, and the frequency. This element can be called an indicator. Note that several indicators may be used for a target.
  • Description of a procedure for initiating adjustment of factors that affect the status of the target if the indicator results are not satisfactory. This element can be called an adjustment procedure.
  • Statement of those internal persons or external partners who must receive a report for the component, and the frequency of reporting. This element can be called the reporting requirements.

Guideline

Someone in each unit shall be assigned responsibility for ensuring that the monitoring system is functioning well.

Guideline

The results of the monitoring are presented so that the receiver of the results can see immediately where there are potential problems that need to be resolved.

Guideline

The costs for the monitoring systems are reasonable according to the different groups of stakeholders that use the results produced by the systems.

Guideline

The results are needed to assist managers and staff to do good work, or to maintain good relationships with groups of stakeholders.

Typical Monitoring Systems

Project Implementation

Management receives indicator results as specified in the detailed project design. Typically there are indicators that show results of activities, outputs and outcomes; status of threats to project completion; and implementation compliance with core values.

Project Grant

The organization that provides the grant will state the reporting requirements. A monitoring system should provide the required information on a regular basis to the grant manager so that the manager can initiate adjustments as needed and can report to the granting organization accurately on time.

Financial Operations

The Finance Officer monitors expenditures by each organizational unit, and sends each unit a regular report of actual expenses against budgeted expenses. The Unit Head will resolve deviations from budget.

The Finance Officer monitors revenue against planned revenue. When revenue deviates from the projections, then senior management makes adjustments in operations.

The Unit responsible for writing grants monitors opportunities for getting an appropriate grant, and monitors progress in writing and submitting the grant.

Mission Compliance

The Board of Directors for the organization usually monitors the Organization’s results and activities against a Mission Statement, Core Values, and policies.

Notes

This is not a comprehensive list of monitoring systems. The organization will need to develop its list based on the units in the organization.

See other monitoring posts on this site for details on creating community development program monitoring systems.

Mindful Evaluation

Note August 14, 2017: Errors in the original post have been corrected.

Mindfulness

Giving  full attention to thoughts, sensations and feelings as well as what we are doing in the present moment; nonjudgmental sustained and focused observation of self; activities to understand at a deeper level what I know and how I know it.

Mindful evaluation is not an evaluation model. It is any evaluation in which the evaluator is deeply aware of the immediate situation and his influence on it throughout the entire evaluation exercise. Mindfulness is consistent with holistic thinking; I encourage transformative evaluators to explore it as an aspect of professional development.

Cullen Puente and Bender (2015) discuss seven steps to follow to increase mindfulness, which in turn increases sound decision making as an evaluator.

1.Take time to think through your intention for incorporating mindfulness principles into your evaluation practice. Visualize yourself being more mindful in each stage of the evaluation. Set aside a specific regular time to practice various ways of being more mindful; practice, practice, practice.

Many web sites describe varieties of exercises to cultivate different aspects of mindfulness. For example, this site describes exercises for experiencing sensations from your eyes, ears, nose, mouth, etc. at a deeper level, and then reflecting on the feelings and thoughts that follow. Retrieve from http://www.practicingmindfulness.com/16-simple-mindfulness-exercises/.

2.Cultivate your ability to pay attention while disregarding distractors.

3.Become more aware of your emotions; accept them as legitimate aspects of who you are. The more you can allow them to be part of your conscious experience, the more you understand how they shape your thinking, the less they will interfere with objective data analysis and interpretation.

Explore your perspectives on cultural, economic, political, social and linguistic matters; these perspectives influence your interpretation of cognitive input with or without your awareness. Heighten your awareness to enrich your evaluation practice and reveal personal preferences that can taint evaluation findings.

4.Cultivate self-reflexivity by asking yourself what you are doing at the moment and why. Think through what constitutes evidence that supports a conclusion. Think through different types of truth, and the many ways we have of distorting our perceptions to fit our preconceived notions about probable evaluation findings. Continually explore your understanding of what is real and not real; what is credible knowledge and what is not.

5.Practice deep listening. Good listeners can elicit more information from others. Also, they may encourage others to reflect on what they are sharing, which may lead to richer information.

6.Stay curious and open. Practice being child-like, playful with your thoughts, as you apply your evaluative skills.

7.Creatively mitigate the influence of preconceived ideas and personal biases.

For a more detailed discussion read

Cullen Puente , Anne and Bender, April. (2015). Mindful Evaluation: Cultivating Our Ability to Be Reflexive and Self‐Aware. Journal of MultiDisciplinary Evaluation. 11:25, 51-59. http://www.jmde.com

Related Posts

http://evalfrank.com/2014/11/principles-for-holistically-planning-an-evaluation/

http://evalfrank.com/2014/11/improving-your-evaluation-work-through-reflective-practice/