Overcoming M&E challenges

SAD’s project ‘Women on the Move’ uses sport and play to enable women affected by civil war and violence to better cope with daily challenges and encourage recovery from traumatic experiences. Project activities are continuously monitored and results / outcomes were measured by combining focus group discussions (FGD) with standardised interviews.

Interviewing for M&E
Interviews and FGD’s are extensively used M&E tools, but are also among the most challenging forms of measurement. They require sensitivity and adaptability as well as the ability to stay within the bounds of the designed protocol and avoid biases, therefore, enhancing data quality.

Local staff, thus, received intensive training to build capacity on principles and techniques for conducting focus group discussions and standardised interviews. An important part consisted of conducting mock interviews in groups of two with peers. Following these, participants were asked to carry out practice sessions with friends or family members to gain experience and confidence.

Obstacles
During the mock interviews, we realised participants struggled to apply what they had learned to the “real” interview situation. Despite several attempts, interviewers repeatedly made exactly those mistakes that they were previously taught to avoid (e.g. suggestive questions, changing the wording of the questions, not probing adequately, suggesting answers). We were frustrated and the participants started losing interest and motivation. To ensure a successful training outcome we needed to change our approach.

Role playing
We decided to conduct mock interviews, one after another in front of the whole group. Everyone was designated interviewer and interviewee at least once. Before each role play, we instructed the interviewee to fake a specific problem or issue that was previously covered in the training (e.g. refusing to answer a question).

The interviewer’s task was to identify the problem and react accordingly. The rest of the group took notes and gave feedback on techniques or questions they thought were done correctly, and those that could be improved.

It turned out that the participants really enjoyed role playing, which immediately drew interest to the topic itself. Immediate feedback from us and the group was needed, coupled with the actual demonstration and concrete experience of a specific issue, to generate learning. It ensured the refining of the skills and application to the actual interview situation, building confidence among participants.

M&E capacity building
This example clearly illustrates the importance of investing enough time in building capacity. As outlined in part three in the series, an M&E practitioner often faces limited resources. Sometimes it may seem unnecessary to allocate already scarce financial resources to training staff. But we believe capacity building is one of the most crucial aspects of M&E.

When conducting workshops, of course planning is important, but you should always remain flexible and adapt to the needs, demands, and interests of participants. What works for one group may not be suitable in another context. Be creative and choose the methods that best suit the interests and capacities of participants.

For people to learn and do a good job, they need to be motivated. Incentives can assume different shapes and forms. M&E can and should be fun to motivate and encourage staff.

Don’t miss part five in the M&E in S&D series, which will highlight M&E capacity building resources and best practices.

Pia Ammann joined the Swiss Academy for Development in 2010, where she has been involved in conducting a rigorous impact evaluation of psychosocial interventions for children orphaned by AIDS in rural Zambia. Currently she works on the implementation and evaluation of the project “Women on the Move: Sport & play based psychosocial interventions for traumatised women in South Sudan”.

She is also involved in the development of a handbook for humanitarian aid workers on ”Sport and Physical Activities as Psychosocial Interventions” by SAD, PS Centre, ICSSPE and TUM. Pia Ammann holds a Master’s in Psychology from the University of Bern in Switzerland.