Tag Archives: professional development

Mindful Evaluation

Note August 14, 2017: Errors in the original post have been corrected.

Mindfulness

Giving  full attention to thoughts, sensations and feelings as well as what we are doing in the present moment; nonjudgmental sustained and focused observation of self; activities to understand at a deeper level what I know and how I know it.

Mindful evaluation is not an evaluation model. It is any evaluation in which the evaluator is deeply aware of the immediate situation and his influence on it throughout the entire evaluation exercise. Mindfulness is consistent with holistic thinking; I encourage transformative evaluators to explore it as an aspect of professional development.

Cullen Puente and Bender (2015) discuss seven steps to follow to increase mindfulness, which in turn increases sound decision making as an evaluator.

1.Take time to think through your intention for incorporating mindfulness principles into your evaluation practice. Visualize yourself being more mindful in each stage of the evaluation. Set aside a specific regular time to practice various ways of being more mindful; practice, practice, practice.

Many web sites describe varieties of exercises to cultivate different aspects of mindfulness. For example, this site describes exercises for experiencing sensations from your eyes, ears, nose, mouth, etc. at a deeper level, and then reflecting on the feelings and thoughts that follow. Retrieve from http://www.practicingmindfulness.com/16-simple-mindfulness-exercises/.

2.Cultivate your ability to pay attention while disregarding distractors.

3.Become more aware of your emotions; accept them as legitimate aspects of who you are. The more you can allow them to be part of your conscious experience, the more you understand how they shape your thinking, the less they will interfere with objective data analysis and interpretation.

Explore your perspectives on cultural, economic, political, social and linguistic matters; these perspectives influence your interpretation of cognitive input with or without your awareness. Heighten your awareness to enrich your evaluation practice and reveal personal preferences that can taint evaluation findings.

4.Cultivate self-reflexivity by asking yourself what you are doing at the moment and why. Think through what constitutes evidence that supports a conclusion. Think through different types of truth, and the many ways we have of distorting our perceptions to fit our preconceived notions about probable evaluation findings. Continually explore your understanding of what is real and not real; what is credible knowledge and what is not.

5.Practice deep listening. Good listeners can elicit more information from others. Also, they may encourage others to reflect on what they are sharing, which may lead to richer information.

6.Stay curious and open. Practice being child-like, playful with your thoughts, as you apply your evaluative skills.

7.Creatively mitigate the influence of preconceived ideas and personal biases.

For a more detailed discussion read

Cullen Puente , Anne and Bender, April. (2015). Mindful Evaluation: Cultivating Our Ability to Be Reflexive and Self‐Aware. Journal of MultiDisciplinary Evaluation. 11:25, 51-59. http://www.jmde.com

Related Posts

http://evalfrank.com/2014/11/principles-for-holistically-planning-an-evaluation/

http://evalfrank.com/2014/11/improving-your-evaluation-work-through-reflective-practice/

Am I a Captious Evaluator?

I came across the word “captious” today. I did not know this word, so I looked for definitions. It is used in two ways: (1) trivial fault-finding; (2) intended to provoke argument. I thought about applying these notions to evaluation practice.

A captious evaluation report is focused on stating every flaw in the evaluand, no matter how trivial. Reading the report raises questions about the evaluator’s capacity to be objective. Can this evaluator see anything good, or is he programmed to only see flaws?

In a captious conversation statements are made to provoke arguing rather than discussing or dialoging. Person A: “it’s not practical to have community members participate in planning this evaluation.” Person B: “In what way is it not practical?” Person A: “It just isn’t.” Person B: “Have you been in an evaluation where there was community participation?” Person A: “No, because I’m a practical evaluator.” Person A is making captious statements rather than engaging in a constructive conversation.

I wonder how often I have acted as a captious evaluator.

Becoming Aware of God’s Handiwork

Ephesians 2:10

For we are God’s handiwork [or workmanship], created in Christ Jesus to do good works, which God prepared in advance for us to do. NIV

My personal reflection (01/14/2017)

In 1965 I was drawn to program evaluation work by complex circumstances; at the time I envisioned myself as a researcher/scholar in the field of cognitive psychology. I applied my research skills to a few small evaluation tasks, but did not see myself as an evaluator.

There was a 15-year period between my initial work and accepting a position where I had major responsibility for evaluation. In that period I experienced a clear calling to serve the poor as a Christian. I knew in my heart that God had prepared me to be his servant; I accepted Jesus Christ as my Lord and Savior; I did effective work in community development outside my research training. During that period I did one program evaluation and one year of institutional research; otherwise I saw myself as a facilitator of community development through the church.

Twenty years after my first evaluation exercise I was called to serve as an evaluator in a Christian relief and development organization. It took me months of prayer and consultation with my wife to say yes, for it meant giving up my vision of being a university researcher/scholar. For thirty years since that “yes” I have experienced being guided by the Holy Spirit in facilitating evaluation exercises of community development programs. For the past twenty years I have been guided by the Spirit and my spiritual disciplines in developing the theory and practice of what I call Transformative Evaluation (TE).

I humbly see myself as God’s handiwork, poetic workmanship, through guidance of the indwelling Holy Spirit fulfilling, with many defects in attitudes and behavior, the divine plan for my life. I am grateful beyond words for God’s love, grace and mercy to be loving God and neighbor through developing TE.

Another kind of handiwork (10/30/2017)

Several years ago we decided to make a cat bathroom. I put linoleum on an unused closet floor for the litter pans. I tried to replace the molding so that it looked like it was done by a good carpenter, because I am not a good carpenter. I used a miter box to saw the ends; I drilled starter holes for the nails so that they were spaced evenly; I used a nail set and then spackled the nail heads; finally, I carefully painted the molding so that you could not see where it was fastened to the wall.

I did this as a spiritual exercise. No one will see it, except the cats and us when we change the litter. And God, who created me to do good works… not as a carpenter, but as a husband to do chores in a way that gives my wife pleasure. I am not a carpenter, but I can do small projects that cause someone to smile. I wonder what other good works God has prepared in advance for me to do?

Your personal reflection

I invite you to prayerfully reflect on your understanding of God’s handiwork in you, being open to prompting from the Holy Spirit.

Role of Worldview in Evaluation Work

Clarifying one’s worldview is essential for the transformative evaluator who believes in being Christian first and evaluator second. The attached file emerged as I clarified my worldview. Currently I am reading several texts that may lead to further clarification, perhaps revision. I encourage you to spend a significant amount of time doing something similar.

The focus of the document is an examination of major scientific concepts related to evaluation work from a Christian perspective. I welcome suggestions and challenges.

Click LINK Worldview in Evaluation

Click category Worldview at the right for other worldview posts.

 

Improving Your Evaluation Work through Reflective Practice

Reflective practice is thoughtful consideration of one’s own professional actions for the purpose of improving one’s professional practice. It has a long history. Various professions encourage their practitioners to engage in this type of professional development, particularly education and medicine.

The attached paper introduces it to program evaluators. I believe it is an essential practice for transformative evaluators because TE includes perspectives and skills that are not common in program evaluators. Reflective Practice can assist you in identifying obstacles that you encounter as you seek to

  • understand a transformative approach to evaluation at a deeper level
  • become more skilled in devising and applying appropriate behaviors in collecting and analyzing valid information related to transformative outputs and outcomes
  • report results in ways that provoke transformative thinking about improving facilitation of transformational community development.

There are many resources on the Internet. Relatively recent books not included in the paper:

Moon. Jennifer A., 2005, Reflection in Learning and Professional Development: theory and practice, London: Kogan

Bolton, Gillie E J, 2010, Reflective Practice: Writing and Professional Development, Sage Publications

Thompson, Sue and Neil Thompson, 2008, The Critically Reflective Practitioner, Palgrave Macmillan

Click link to the paper: 66Reflective Practice

Evaluation Books and Papers I Keep Close at Hand

Previously I recommended five non-evaluation  books that help me formulate foundational principles for transformative evaluation.  Click link

There are two evaluation books that I consider most relevant for transformative evaluators that have a good understanding of the general principles of program evaluation.

Fourth Generation Evaluation, by Egon G. Guba and Yvonne S. Lincoln, is a comprehensive description of constructivist evaluation. This approach, which is an alternative to evaluation approaches that are based on a scientific epistemology, provides the framework for designing and implementing transformative evaluation.

Transformative Research and Evaluation, by Donna M. Mertens, describes evaluation as a means to promote human rights and social justice. This is an important component of transformative evaluation.

The attached file is a list of other evaluation references that are helpful to me. May they be helpful to you on your journey toward being a transformative evaluator.  Click link:  Eval Books 2015

Core TE research methodologies

In addition to the content of a basic monitoring and evaluation curriculum I believe that the following research methodology topics should be learned thoroughly by anyone engaged in Transformational Development monitoring and evaluation.

Technical competence needs to be nuanced by insights that come as he or she engages in spiritual disciplines such as Bible study and reflection, prayer, fasting, etc.

  • Field interview techniques for sensitive topics and in different cultures.
  • Participatory approaches to inquiry.
  • Focus group principles and methods. link to focus group guide
  • Ordinal measures and common statistics tests for ordinal measures.
  • Analysis and interpretation of Likert scale results from multiple perspectives.
  • Characteristics of trustworthy qualitative data and procedures to increase trustworthiness.
  • Logic of statistical inference and interpretation of statistical findings. Although statistics is advanced mathematics the logic of inference and interpretation can be understood without necessarily understanding the complex math formulas.

I know this will take time! A good evaluator continually updates his or her knowledge and adds skill sets.