New comprehensive guidance for monitoring and evaluating anticipatory action
As forecast-based financing and action grows in depth, breadth and scope, the need for robust ways to monitor its impacts and evaluate its effectiveness is paramount. Evidence about the effects of anticipatory action - positive or negative - is still limited. The demand for monitoring and evaluation (M&E) guidance and tools for measuring whether anticipatory action makes a difference has been growing.
To meet this demand, in 2021 the World Food Programme (WFP) developed ‘Monitoring and evaluation of anticipatory actions for fast and slow-onset hazards - Guidance and tools for Forecast-based Financing’, with support from the Red Cross Red Crescent Climate Centre.
“We hope that this new, comprehensive guide on monitoring and evaluation of anticipatory action will help not only WFP country offices, but also the entire anticipation community, to generate more and better evidence on what works and why, and where we can do better,” says Jesse Mason, WFP’s anticipatory action lead.
A comprehensive, state-of-the art guide
This 79-page document presents the most comprehensive and state-of-the-art guide on anticipatory action M&E to date, complete with framing, examples and a range of practical tools.
The aim is to help practitioners answer the overarching question: does forecast-based financing make a difference to reducing or mitigating the impacts on affected populations? Its three main sections take practitioners through: the key considerations for the M&E of anticipatory actions for slow- and fast-onset hazards; a step-by-step process for planning and implementing M&E; and assessing results. The guidance document is accompanied by a range of editable templates, for example indicator frameworks and M&E plans, as well as questionnaires and topic-specific guidance.
While this M&E guide was developed for WFP’s programmatic context and key indicators (e.g., on food security and coping strategies), its approaches and tools can be adapted for other institutions and programmes.
What to measure, when and how
The M&E guide hones in on the particular challenges posed for M&E for anticipatory action for both slow- and fast-onset hazards, and offers solutions to these.
What to measure
The rationale for investing in anticipatory action is that humanitarians can act earlier than they normally would. Therefore, basic data on the timelines, type and reach of actions is important. Going beyond this, we also want to understand whether anticipatory actions contribute to better outcomes for vulnerable people, and whether the differences they experience are attributable to the anticipatory actions. The M&E guide offers suggestions on how to do this.
When to act, and when to measure results
When is the ideal time to deploy anticipatory action? When can the intended (and unintended) effects of anticipatory action be felt, and when should these be measured? The answers to these are highly context-specific, but can be generalized through making the distinction between slow-onset hazards (e.g. drought) and fast-onset hazards (e.g. cyclones). In both cases, results should be measured soon enough for people to remember the benefits of anticipatory action (i.e. before they are either forgotten or ‘eroded’ by time or repeated shocks) and late enough for the results to have fully materialized. This M&E guide includes suggestions for how to determine these points in time.
How to attribute results to actions
Different strategies can be used to make a causal attribution of observable outcomes to specific actions. Each strategy has its own strengths, limitations and suitability, according to the specific programme context and the capacity of the implementing organization. The M&E guide proposes a quasi-experimental design as a practical solution for most implementers. This involves collecting outcome data from a sample of anticipatory action beneficiaries, and from a comparison group of equally vulnerable and hazard-exposed people who were not reached by anticipation but who benefited from the ‘regular’ response assistance.
Better evidence for better anticipatory actions
Monitoring and evaluation are essential components of any anticipatory action programme and an unparalleled way for the community of practice at large to learn about the effectiveness, barriers, successes and failures of anticipatory action activations. M&E can be complex to get right, with many factors and capacities to consider. This new M&E guide offers actionable ways to create and deploy robust and effective M&E plans, acknowledging the highly context-specific nature of anticipatory action programmes.
It complements ongoing efforts by the Anticipation Hub to strengthen the generation and use of evidence on anticipatory action, including the Monitoring, Evaluation, Accountability and Learning Practitioners Group and the evidence database.