Want to win in impact? Maximise measurements and increase value
Want to win in impact? Maximise measurements and increase value
Alisha Greenberg and Kathy Babiak present a two-part series on trends in measurement and evaluation in SDP.
Imagine a Super Bowl or World Cup champion team showing up for the big game without a plan or a strategy for how to execute their plays and measure their performance. What metrics would they use to evaluate the quality of their play on the field both during the game and even after the trophy has been raised? It likely wouldn’t happen.
Every organisation, no matter what size, what city, what cause, is similar to any championship team. Organisations want to win and most do have a game plan. The key to knowing whether or not the game plan is working involves collecting data, analysing results and adapting as necessary based on the analysis. This process is often termed measurement and evaluation. Like many different types of organisations - teams, businesses, nonprofits, sport for development organisations can also benefit from thinking strategically about measurement and evaluation.
In this brief article, we outline a few of the trends we’ve seen around measurement and evaluation and offer some examples of best practice.
Investing time and resources in measurement has many advantages including:
- Strengthen and improve programme design, implementation and innovation
- Adapt organisational activities to changing stakeholder needs and interests
- Discover programme / service gaps and areas of improvement
- Realize programme / organisational goals more effectively
- Communicate and engage with stakeholders - community partners, funders, policy makers
- Demonstrate transparency and accountability of programme outcomes
- Illustrate impact through storytelling
Measurement and evaluation in the sport for development context can be logistically challenging (with many organisations serving diverse populations in locations around the world) and often resource constrained. This function however, has become critical for support from key stakeholders such as funders, sponsors and government partners. When all is said and done, evaluation helps you demonstrate how your programmes and services are making a difference and how you are engaging in continuous improvement.
Many reading this have likely heard of standard tools like surveys, focus groups and interviews. As the opportunities for data collection evolve, we are sharing a few trends you should pay attention to now and in the future.
Using theory and data as a guide
No one is alone is feeling overwhelmed at times by the evaluation process. The good news is that there are resources to help.
A good starting point is to build a framework, similar to a theory of change. This is another way of saying that your organisation should ask itself questions regularly to reach the success you desire.
Other resources include national research and data reports that align with your mission and desired outcomes. You don’t need to justify your work all on your own, find results that have been published and apply them to your work.
For example, are you looking for outcomes related to social and emotional learning? Look at what the Robert Wood Johnson Foundation (RWJF) has published recently on this topic. Other areas RWJF has researched include the connection between race, racism and health and the importance of preventing adverse childhood experiences (ACE’s).
Many organisations have also started aligning themselves with the UN Sustainable Development Goals, blueprint to achieve a better and more sustainable future for all. These tools can be helpful markers for your own organisation’s evaluation efforts.
Customising / adapting: Who are you serving?
As participant demographics continue to change and grow more diverse, it is important to tailor your evaluation tools to reach these audiences. For example, do you offer surveys in both Spanish and English? Do you have a paper version in case accessing it via computer is a barrier for completion? It is also important to take into account cultural differences as you develop focus group questions and interview scripts.
When it comes to evaluating youth, have you thought about what is appealing and accessible to that age participant? For example, kids love technology and offering them a way to respond in the form of a video game might yield more results than a standard survey. Keep it fun for kids! Below are some examples of innovations in access that some organisations are using:
- The San Francisco Giants Community Fund is successfully collecting data from youth using ipads on a baseball field. They regularly arm their coaches and youth leaders with ipads to collect data from kids right on the ballfield—that data is then sorted out by a third party evaluator
- Another organisation, Doc Wayne, based in Boston, worked with an academic partner to develop fun, computerised ways to collect data from kids. They would host a “carnival” type of event where the kids would answer questions as they went from station to station and earn prizes for their participation
- Positive Tracks an organisation using sport to empower youth as a catalyst for change, worked with us to measure their impact on youth leadership development. We designed a survey that participants could complete via phone / text and adapted the survey language to engage kids in a fun and meaningful way
All of these trends can be customised to meet the needs of your individual organisation. What will work best for you? Think about the following questions as you approach your measurement and evaluation strategy:
- How do you know right now that you are making the difference you desire?
- Do you have a logic model or theory of change?
- Are you measuring impact against desired strategic outcomes?
- Have you seen programme participant demographics change? How have you adapted your data collection to meet these changes?
Stay tuned for part two of this article in the next sportanddev newsletter!
Alisha Greenberg, Rounding Third LLC, and Dr. Kathy Babiak, Director of the Center for Sport and Social Responsibility at the University of Michigan, are trusted leaders working at the intersection of sports, philanthropy and social impact. We offer a unique combination of experience as practitioners, researchers and programme funders that give us access to an extensive global network.