6. Activate, Monitor, Evaluate

Summary

The activation of the EAP is the culmination of all the work that went into making a National Society FbF ready, developing the EAP and getting it approved. With the EAP in place and the FbF system set up, ideally there are no more decisions to take; once the trigger is hit, the activation of the EAP can begin. Everyone involved should know what will be done where at what moment and by whom.

For an EAP to be activated different factors must coincide: forecasts must predict an event of a magnitude and strength that reaches the defined threshold, with sufficient lead time to carry out the early actions and in a region that is covered by the EAP; and of course, the EAP must already be validated and legal agreements in place. In order to be able to determine, whether the activation was successful, the forecasted event must then materialize in the area where early actions were carried out. FbF being a relatively recent approach for those cases, where all these conditions are met, it is of high importance to carefully monitor and evaluate the activation in line with the EAP M&E plan, to ensure learnings are documented, evidence can be built and the FbF system and the EAP adjusted based on the lessons learned.

This chapter provides guidance through the activation process addressing questions such as:

  • What points to go through when an extreme event is approaching but a trigger not met?
  • How to evaluate the impact of the activation?
  • How to systematize learnings?
  • How to share the results?
Step 0: Monitor the forecasts and get ready

While the activation process starts, when the trigger is reached, in most cases forecasts already indicate a few days before the critical threshold is reached that an extreme event is approaching. This period can provide the National Society with crucial extra time to get ready for activation. Questions to be answered include: Is the automatic trigger mechanism in place and functioning? Or where there is no automatic system, does the person in charge of monitoring know whom to inform? Are all items available? Are necessary agreements with providers in place? Do all those involved know what to do? Are transport options ready to go? For branches that look most likely to be involved in activation, is all contact information up to date? Are local authorities informed that an activation might happen?

The few days before the activation give time for a last double-check that everything is in place. However, all necessary arrangements and plans should already be made during EAP development and set-up of the FbF system. Especially, the implementation process chart, detailing what will be done where at what moment by whom as well as the communication protocol, comprising contact information and who should communicate to whom, what and when, need to already be in place. (See Chapter Develop your EAP)

Step 1: Activate the EAP

The activation process starts with the message that the trigger has been reached (on Day 1). It is important to immediately send the trigger notification to IFRC to ensure availability of funds. In line with the communication protocol alert the relevant actors.
The activation should then follow the implementation process as defined during the development of the EAP.

The Monolian Red Cross Society staff is conducting dzud assessments in the field - Foto by IFRC
© IFRC | EAP Activation Dzud in Mongolia 2020 – Read more here.

Test activations

Sometimes, an extreme event might be approaching your country, but you are still working on your EAP or it has not been approved by the FbA by the DREF yet. Whenever possible, you should still activate the early actions that you are considering. This can be very useful for testing potential early actions and the FbF system, but more importantly allows you to take action to mitigate or reduce the impact on the vulnerable population.

See below a list of test activations.

EAP test activation heatwave in Vietnam 2019

Read more here.

EAP test activation flood in Bangladesh 2019

Read more here and view more pictures here.

EAP test activation cyclone in Mosambique 2019

Read more here.

EAP test activation coldwave in Peru 2019

Read more here and view more pictures here.

Step 2: Monitor the activation

Monitoring should start right with EAP activation to ensure the FbF team will learn whether it was successful in acting early and as planned. The monitoring process should follow the details laid out in the M&E plan and the EAP monitoring form.

  • Use the EAP monitoring form for the triggered EAP to monitor implementation – you can print the form and give it to NS staff and volunteers who are deployed to the field and then return them to the M&E focal point for consolidation.

You can track the EAP implementation progress either online or offline, depending on your preference.

Materials offline:

  • Print out the EAP monitoring form on a large sheet of paper (A3 or larger), or copy to a flipchart, and continuously populate it as new information comes in. This has several benefits:
    • Everyone can see what information is being tracked and where the team stands vis-a-vis the EAP.
    • It increases the likelihood that the team captures all relevant pieces of information, new observations and learnings
    • Remaining empty fields will quickly (and visually) indicate potential focus areas where increased effort is needed.

Materials online:

Step 3: Collect and analyse data to assess early action impact

When developing the M&E plan, the indicators, approach, and steps to gather and analyse data on the impact of the extreme event and the early actions were defined and it was decided who would carry out the data collection (consultant, university, volunteers etc.). See also chapter Design your M&E Plan. Following the activation, in line with your M&E plan carry out the impact evaluation. Ensure that those collecting the data (through survey, discussions, interviews etc.) have received sufficient training to carry out these tasks. It needs to be communicated clearly to communities for what purpose this data is collected, to avoid raising wrong expectations. This is especially the case, if the data collection is done by volunteers of the National Society, as assessments by Red Cross and Red Crescent carried out following a disaster are often a needs assessment for later humanitarian assistance.

Some actions will show impact at different times, so depending on the prioritized impact to be reduced by each action and when the outcome would be visible, data might need to be collected at different times. For example, depending on the action, it might take half a year to see if impact on livelihoods was really reduced (e.g. by protecting crops), while the impact on health might be easiest to assess a few days or weeks following the event.

Step 4: Evaluate the trigger system

Evaluation of the activation not only serves to see whether the EAP actions had the desired impact, but also to assess whether the FbF system as it was set up, was set up correctly or based on the right assumptions. One major element of this system is the trigger.

Each activation provides an opportunity to assess whether the trigger was defined appropriately and if and how it could be improved. Evaluation of EAP triggers typically follows three main questions: (1) Did we learn something new about the elements that form the basis of the trigger? (Is one of the elements different from what it was when triggers were initially defined? Has anything changed about the datasets we use that requires a review of triggers?); (2) Do we know more about the accuracy of the forecast models used? (How accurate was the forecast compared to the actual event? Are the forecast models used still the best available?); (3) Were the probabilities and impact levels of the original trigger appropriate? This evaluation process will guarantee a continuous learning and improvement of the trigger system.

In order to assess the effectiveness of set triggers, conduct interviews, and at best a workshop with  all relevant FbF stakeholders (see the Mozambique Semi-structured interview guide and Workshop Formats here:

Step 5: Organize a workshop to systematize learning

Once EAPs have been triggered and evaluation data have been collected and analysed, there are many opportunities to learn more about the adequacy of the FbF system, including triggers, early actions, implementation procedures, effectiveness and beneficiary impacts and discuss discrepancies between expectation and results, the effectiveness of the activation and necessary changes to the EAP and MEAL system. These questions may best be answered by bringing together all staff and stakeholders involved.

Organize a workshop to host an EAP activation learning exercise to contribute to the understanding of all parties involved about the performance of their EAP activation, and to help promote learning and accountability.

Following a recent FbF Early Action Protocol (EAP) activation, the exercise typically takes a 1-3 day workshop format and brings together all.

More specifically, the learning exercise:

  • Provides a space for implementers and partners to capture key learnings after an EAP activation.
  • Provides space to evaluate: NS performance, EAP implementation feasibility and triggers.
  • Generates lessons learned that can be shared across the implementing organizations and the wider FbF community, as applicable.
  • Identifies recommendations to the FbF team (management, staff and partners) for improving the EAP and/or other early action-related plans and decisions.
Step 6: Write a report to share learnings internally

It is important to capture the results of the data collection and analysis (the impact study or evaluation) and the outcomes of the workshop in a format that ensures that the findings are understood and utilized, for example, to inform an improvement of the EAP or the action implementation process. Therefore, before writing a report, it should be clear who the main audience is, e.g. the FbF project team; individuals in government departments; donor and partner staff; development professionals working with similar projects or programmes; students; or community groups?

Regardless of who the target audience is, the report should be readable, straight to the point, and use a plain language writing style that promotes understanding and ensures accessibility to the information for all stakeholders.
It is good practice to follow the “One-Three-Twenty-Five (1:3:25) principle”: a 1 page outline, a 3 page executive summary and 25 pages to present the findings and methodology. Use visual aids (charts, organigrams, tables) where they help to clarify the main messages, and remove any “chart junk” that does not add informative value. It is suggested to use descriptive chart titles so that charts can stand alone without needing to read the evaluation report to understand the information they convey.

A good evaluation report contains these basic components:

  • An executive summary containing a condensed version of the most important aspects of the evaluation (see previous point).
  • A summary of the evaluation’s focus, with a discussion of the purpose, objectives and questions used to direct the evaluation.
  • A summary of the evaluation plan.
  • A discussion of the findings of the evaluation, with complete statistical and case study analysis. A discussion of the evaluation’s conclusions and recommendations.
  • Any additional information required, such as terminology, details of who was involved in the evaluation, etc. in an appendix.

Resource:

Step 7: Report to IFRC and share results externally

A maximum of 90 days after the activation is completed, financial and narrative reports need to be submitted to IFRC in line with the requirements of the Funding Agreements under FbA by the DREF.

Beyond the reporting obligations, if the evaluation has brought about learnings that are relevant for other stakeholders involved in FbF or anticipation or contributes to building evidence for FbF, consider to publish the results, for example in the form of a research report or a case study.

Step 8: Update the EAP

Once an EAP has been activated, the EAP must be updated and resubmitted to the IFRC in order to be re- validated and receive new funding from the FbA by the DREF. The inclusion of an analysis of activation learnings including trigger, early actions, and roles and responsibilities is a key component to qualify for revalidation.