An Engaged Evaluation Learning Checklist
In this post, Cathy Hearn and Megan Legault share a checklist tool designed to assist evaluation educators in planning and implementing engaged learning experiences with their students. They first presented this tool at the 2022 American Evaluation Association conference in New Orleans, Louisiana.
In early 2022, members of CEDER’s evaluation team set out to integrate a real-world evaluation project into a class offered to graduate students at the University of Michigan School of Education. While schools and centers across University of Michigan (e.g. The College of Engineering, the School of Information and the Ginsberg Center) have led the way in the development of engaged learning as an approach, this is the first time we were aware of this approach being used in the context of evaluation.
What is engaged learning?
“Engaged learning offers students opportunities to practice in unscripted, authentic settings, where stakeholders (including the students themselves) are invested in the outcome.” –University of Michigan Center for Research on Learning & Teaching
Engaged evaluation learning experiences provide students with valuable opportunities to develop professional competencies and gain insights into the nuances and logistics of real-world evaluation work. Opportunities to develop these kinds of competencies are often absent or limited in more traditional graduate-level methods classes.
How did we use engaged learning?
We integrated real-world evaluation activities into a graduate-level applied qualitative methods class taught by CEDER Evaluation Coordinator, Dr. Vicki Bigelow, during the 2022 Winter Semester. The participating students were all enrolled in Master’s programs at the School of Education, with the majority enrolled in CEDER's partner degree concentration: Program Evaluation and Improvement Research.
Throughout the course of the semester, the students engaged in some of the main elements of a real evaluation partnership. Tasks included reviewing materials to familiarize themselves with client and evaluation design, using evaluation questions to design an interview protocol, scheduling and facilitating two focus group interviews each, and engaging in data cleaning and analysis.
What did we learn?
There are four main areas to consider when designing an engaged evaluation learning experience:
-
Ensure that there is a fit between the evaluation project you have in mind and your class.
This includes checking that the objectives of your class align with the proposed evaluation activities and checking that the dates in your project plan align with the dates of your class or learning experience. -
Ensure that there are high standards for quality and ethics in your evaluation activities
This includes revisiting your IRB study and making the appropriate changes and ensuring that members of your evaluation team are available to supervise students when engaging in client-or participant-facing activities. - Plan out your classes to integrate the evaluation project.
This includes defining which stages of the project your students will be engaged in and considering how you will integrate the necessary methodological training with project-specific tasks. - Work through the logistics of conducting a learning experience and evaluation concurrently.
This includes establishing a secure shared filing system, working between the client, evaluation participants, students, and evaluation staff to schedule data collection, and planning for inevitable logistical challenges.
The checklist is designed to guide evaluation educators through these four areas when developing their own engaged learning experience.
Access a visual impairment and screen reader-accessible version
If you require additional assistance or accommodations to access this checklist, please contact Vicki Bigelow at vbigelow@umich.edu.
Authors: Cathy Hearn & Megan Legault
