Introduction to Act Six
For a Local MPA to be effective long-term, it must be evaluated on an ongoing basis. Your Local MPA is a living document, and to stay relevant, it will need to adapt to changing priorities and circumstances over time. By the time your Implementation Team completes this Act, you will have:
- An evaluation dataset that measures the progress of specific tactics and assesses the overall effectiveness of the Local MPA
- A public-facing Evaluation Report that describes key takeaways from the dataset
- A refreshed Local MPA that has been updated to reflect necessary changes uncovered in the evaluation process
Step 1: Establish Evaluation Cadence, Model, and Task Force
Cadence
Evaluation ensures that you can demonstrate results, learn from experience, and improve over time. However, ongoing evaluation can put a strain on your capacity. We therefore recommend agreeing on a timeline where evaluation and refreshment of your MPA happen at regularly scheduled intervals.
If you have joined the Age-Friendly Community Network as part of your Local MPA process, you can follow the Age-Friendly Action Plan cycle to pace your evaluation process. If you have completed your Local MPA independently, you can set any cadence you prefer. We recommend evaluating your plan at least twice over the 10-year timeframe.
Model
Selecting an established evaluation model can be helpful to ensuring that your evaluation process is useful, relevant, and grounded in a foundation of academic research. There are numerous frameworks available for evaluation, ranging from fairly simple to highly sophisticated, each of which focuses on different evaluation outcomes. In Act I, we recommended beginning your planning process with your evaluation model in mind.
The steps and tools that follow can be adapted to inform any evaluation model, including the two recommendations in the Landscape Analysis Spreadsheet's bibliography tab, or they can be followed as-is.
Task Force
Assemble an Evaluation Task Force to ensure that your evaluation process is thorough, impartial, and inclusive of all community members the Local MPA serves. Use your Network Development Spreadsheet to identify and recruit task force members. For your Evaluation Task Force, consider the following:
- Local MPA Coalition Members: The purpose of your Local MPA Coalition is to represent the diversity of your community across sectors, services, and identities. Select a few members from your Coalition to act as a voice for your community during the evaluation process.
- Trained Specialists: You may want to identify trained professionals or academic researchers who specialize in plan or program evaluation, particularly if you have selected a highly sophisticated evaluation model or anticipate having a large dataset.
- Additional Community Members: As the end users of the positive outcomes of your Local MPA, community members should be robustly represented on your evaluation team. Consider recruiting a diverse set of motivated community members from your network to contribute to evaluation.
- Implementation Team Members: Ideally, your Implementation Team will contribute to the development of the evaluation dataset, but step back during the evaluation itself. Carefully consider whether Implementation Team members can contribute to an impartial evaluation before including them on the task force.
Evaluation Task Force Tip:
Give careful consideration before including Implementation Team members on the Evaluation Task Force.
Because your Implementation Team is so closely involved with the implementation of the Local MPA, they may struggle to maintain impartiality in the evaluation process.
Step 2: Compile Evaluation Dataset
The first task for your Evaluation Task Force is to compile the evaluation dataset. They will gather data from two main sources: tactic progress data, which will inform evaluation of specific programs and projects, and community engagement data, which will inform evaluation of the Local MPA overall. Your Evaluation Task Force will use similar techniques to gather and evaluate your dataset that your Development Team used in Act I. The Evaluation Dataset Spreadsheet is designed to help input, track, and organize the evaluation data.
Collect Tactic Progress Data
- Request progress reports from implementation team. Follow up with your Implementation Team and request a current progress report for each active tactic. Once received, input each progress report in the Tactic Tracker.
- Input progress report data. Share Tactic Tracker data with the Evaluation Task Force so that they can include progress report data in the Evaluation Dataset Spreadsheet.
- Gather additional impact data. The Evaluation Task Force may identify and input additional data points if desired, either via desk research or by requesting from the Implementation Team.
Collect Public Engagement Data
- Input existing data. You should already have a series of public comments about the Local MPA that have been submitted since publication. Share public comments with your Evaluation Task Force for entry in the Evaluation Dataset Spreadsheet.
- Conduct Listening Sessions. Schedule a series of listening sessions to gather public feedback on the Local MPA’s perceived impact on your community – fewer than you did in Act I. Use the exact same questions. Share the notes from each listening session with your Evaluation Task Force for entry in the Evaluation Dataset Spreadsheet.
- Conduct a Needs Assessment Survey. Conduct a Needs Assessment Survey according to the same procedure you used in Act I, but on a smaller scale. Use the exact same questions, and similar distribution methods. Keep Needs Assessment Survey data separate from your qualitative dataset.
Tool Spotlight: Evaluation Dataset Spreadsheet
The Evaluation Dataset Spreadsheet helps your Evaluation Task Force compile and analyze the data that will inform their Evaluation Report.
Step 3: Analyze data
In this step, your Evaluation Task Force will use comparative and thematic analysis techniques to identify the resonant takeaways that will inform their Evaluation Report. Their analysis will focus on:
- Needs Assessment Data Comparison
- Tactic Progress Data Analysis
- Public Engagement Data Analysis
- Global Takeaways Narrative
Needs Assessment Data Comparison
First, your Evaluation Task Force will compare the results from your initial Needs Assessment Survey (the one you did in Act I) to your most recent Needs Assessment Survey to determine:
- What outcomes have improved?
- What outcomes need improvement?
- How well is this Local MPA serving specific populations?
The comparative analysis will yield resonant data points that are relevant to individual tactics. As takeaways emerge, add them to the “Tactic Progress Data” tab in the Evaluation Dataset Spreadsheet.
Tactic Progress Data Analysis
Next, your Evaluation Task Force will identify major themes from the Tactic Progress Dataset to identify:
- How well is the Local MPA delivering the outcomes of its tactics?
- How well is the Local MPA adhering to its stated timeline?
- What themes emerge regarding challenges and opportunities?
Once the analysis is complete, copy the themes to the Global Takeaways Analysis tab in the Evaluation Dataset Spreadsheet.
Public Engagement Data Analysis
Third, your Evaluation Task Force will conduct a similar thematic analysis to analyze your public engagement data to determine:
- How has the Local MPA impacted the lived experience of older residents in your community?
- Do specific outcomes or outputs of the Local MPA have community recognition?
- What needs, successes, barriers, and opportunities emerge?
Once the analysis is complete, copy the themes to the Global Takeaways Analysis tab in the Evaluation Dataset Spreadsheet.
Global Takeaways Narrative
Finally, your Evaluation Task Force will synthesize the themes from the Tactic Progress Data Analysis and the Public Engagement Data Analysis to create a set of global takeaways. These “meta-themes” will explain:
- How has the Local MPA impacted the lives of older residents across each Domain of Livability?
- What challenges and opportunities emerge, and how can they be addressed?
- How has the Local MPA impacted members of historically underserved communities?
The global takeaways will generate compelling narratives and references to source material that can be used directly in the Evaluation Report.
Step 4: Prepare Evaluation Report
In this step, your Evaluation Task Force will synthesize their data analysis results into an Evaluation Report. The Evaluation Report Template contains an outline to help organize the Evaluation Report. As they prepare the Evaluation Report, the task force should consider:
- Audience: Who will read and use the Evaluation Report? Is it written in a way that is accessible to all audiences?
- Purpose: Are the evaluation goals clearly articulated in the Evaluation Report, and does the report fulfill those goals?
- Recommendations: Are recommendations feasible, and do they align with the evaluation data and the current circumstances?
When the Evaluation Report is complete, the Evaluation Task Force should present it to the Implementation Team, the Implementation Workgroup, and the Local MPA Coalition.
Step 5: Create and Release Updated Local MPA
Convene your Implementation Team to carefully review the Evaluation Report. Create a new draft of your Local MPA and incorporate the recommended changes from the Evaluation Report as appropriate. Follow the Public Comment procedure outlined in Act Four to ensure that your Implementation Team remains committed to transparency.
Step 6: Repeat Evaluation Cycle
Effective planning is a cycle. Repeat the steps outlined in Act Six according to your evaluation cadence over the lifespan of your Local MPA. Congratulations to you, your Implementation Team and Workgroup, your Local MPA Coalition, and your entire community on your commitment to elevating and transforming the wellbeing of people of all ages!