Participatory Evaluation in Community-Engaged Research

How can researchers involve community members in participatory evaluation? 

Researchers should try to involve community members at every stage of the evaluation process, from design to dissemination and reporting, and choose methods that align with community values. This approach creates opportunities for community members to match evaluation activities with their priorities and needs. 

Participatory Evaluation in Action

The Community Engagement Alliance Consultative Resource (CEACR) serves as a channel to share responsive community-engaged practices with research teams interested in applying principles of community engagement to address differences in health outcomes and encourage participation in research programs. The CEACR team applies participatory evaluation principles in assessing their work. The two examples below demonstrate how CEACR used participatory methods to evaluate webinars and a community engagement focused toolkit. 

Example of a Low Degree of Community Involvement in Participatory Evaluation

CEACR conducted a webinar series for community-based organizations that were seeking and applying for NIH grant funding. To evaluate the webinars, CEACR surveyed attendees before and after each webinar. This participatory evaluation’s design is an example of a low degree of community involvement, as CEACR was unable to invite community members to review survey drafts or provide input on reporting methods. 

After surveys were completed, CEACR applied principles of participatory evaluation by sharing a plain-language summary of the survey results with attendees. The summary included infographics with clear, meaningful data visualizations and narratives. One participant responded with praise: “This report is really the best post-session tool I have seen to date; easy to digest and visually interesting. Bravo to everyone with a hand in this, especially the communications team.” 

Example of a High Degree of Community Involvement in Participatory Evaluation

CEACR created a webinar series and a toolkit for the Researching COVID to Enhance Recovery (RECOVER) Initiative’s writing teams. The goal of these materials was to help the RECOVER project team explore ways to involve community members in writing and dissemination activities and create meaningful dissemination products for academic and community audiences. 

For this project, CEACR conducted a participatory evaluation with a high degree of community involvement. CEACR collaborated with Patient, Caregiver, and Community Representatives to plan, design, and conduct the evaluation. This approach was a good fit for CEACR’s evaluation of the webinars and toolkit because the RECOVER initiative has a high level of community engagement.

During the planning phase, CEACR attended several meetings with Patient, Caregiver, and Community Representatives to gather feedback. To accommodate Representatives’ schedules, CEACR joined existing RECOVER network meetings. During the design phase, CEACR presented draft survey questions that focused on webinar attendees’ learnings and experiences and asked for Representatives’ feedback, verbally or in writing, to accommodate differing levels of ability and energy. CEACR compiled the feedback and shared a summary of revisions with those who contributed.

After collecting survey responses, CEACR reported the evaluation results in a community-centered way by presenting them during a meeting, as the Representatives requested. In this presentation, CEACR highlighted community voices and the importance of lived experiences.

How to get started with Participatory Evaluation

Step 1: Determine if participatory evaluation is a good fit for your research project by considering the following: 

• The purpose of community participatory evaluation in your project
• The community participants you are interested in involving and why
• A feasible and thoughtful timeline for the inclusion of community participation into your project evaluation (Gujit, 2014). 

Step 2: If participatory evaluation is a good fit for your project, make a plan. Assess the appropriate level of participation for your project and involve community members as early as possible. Identify the time points and opportunities for community involvement throughout your research project. Consider how community members could help design the evaluation plan and instruments, like surveys or interview guides. Reflect on opportunities for community members to decide how you will collect evaluation responses and report on the results.

Step 3: Commit to ensuring research reciprocity for community members who are collaborating on the evaluation, regardless of their level of participation. Ask community members what return on value means to them. Provide examples of what your project or institution can give that has equal value to the time and effort they have brought to the research project. Check out CEACR’s Establishing Trustworthiness and Research Reciprocity for Community-Engaged Research Toolkit to learn more about return on value to the community.

Step 4: Set up feedback loops to inform community members of how their feedback is being applied to make the evaluation more community-centered. 

Step 5: Provide the community with a summary of the evaluation results in plain language. Use meaningful data visualizations and storytelling to construct the summary’s narrative, with input from community members wherever possible.

As evaluators, be mindful of the power that you possess when partnering with communities, and approach the collaboration as a skill-sharing process (Suarez-Balcazar and Harper, 2014).