6. Activate, Monitor, Evaluate

While the activation process starts, when the trigger is reached, in most cases forecasts already indicate a few days before the critical threshold is reached that an extreme event is approaching. This period can provide the National Society with crucial extra time to get ready for activation. Questions to be answered include: Is the automatic trigger mechanism in place and functioning? Or where there is no automatic system, does the person in charge of monitoring know whom to inform? Are all items available? Are necessary agreements with providers in place? Do all those involved know what to do? Are transport options ready to go? For branches that look most likely to be involved in activation, is all contact information up to date? Are local authorities informed that an activation might happen?

The activation process starts with the message that the trigger has been reached (on Day 1). It is important to immediately send the trigger notification to IFRC to ensure availability of funds. In line with the communication protocol alert the relevant actors.
The activation should then follow the implementation process as defined during the development of the EAP.

© IFRC | EAP Activation Dzud in Mongolia 2020 – Read more here.

Sometimes, an extreme event might be approaching your country, but you are still working on your EAP or it has not been approved by the FbA by the DREF yet. Whenever possible, you should still activate the early actions that you are considering. This can be very useful for testing potential early actions and the FbF system, but more importantly allows you to take action to mitigate or reduce the impact on the vulnerable population.

See below a list of test activations.

Monitoring should start right with EAP activation to ensure the FbF team will learn whether it was successful in acting early and as planned. The monitoring process should follow the details laid out in the M&E plan and the EAP monitoring form.

• Use the EAP monitoring form for the triggered EAP to monitor implementation – you can print the form and give it to NS staff and volunteers who are deployed to the field and then return them to the M&E focal point for consolidation.
  • Tool: EAP monitoring form template or EAP online monitoring form  with examples.

You can track the EAP implementation progress either online or offline, depending on your preference.

Materials offline:

• Print out the EAP monitoring form on a large sheet of paper (A3 or larger), or copy to a flipchart, and continuously populate it as new information comes in. This has several benefits:
  • Everyone can see what information is being tracked and where the team stands vis-a-vis the EAP.
  • It increases the likelihood that the team captures all relevant pieces of information, new observations and learnings
  • Remaining empty fields will quickly (and visually) indicate potential focus areas where increased effort is needed.

Materials online:

• Or, use electronic tools for data collection. Consolidate information with M&E focal point and share with everyone involved in implementing actions.

When developing the M&E plan, the indicators, approach, and steps to gather and analyse data on the impact of the extreme event and the early actions were defined and it was decided who would carry out the data collection (consultant, university, volunteers etc.). See also chapter Design your M&E Plan. Following the activation, in line with your M&E plan carry out the impact evaluation. Ensure that those collecting the data (through survey, discussions, interviews etc.) have received sufficient training to carry out these tasks. It needs to be communicated clearly to communities for what purpose this data is collected, to avoid raising wrong expectations. This is especially the case, if the data collection is done by volunteers of the National Society, as assessments by Red Cross and Red Crescent carried out following a disaster are often a needs assessment for later humanitarian assistance.

Evaluation of the activation not only serves to see whether the EAP actions had the desired impact, but also to assess whether the FbF system as it was set up, was set up correctly or based on the right assumptions. One major element of this system is the trigger.

Each activation provides an opportunity to assess whether the trigger was defined appropriately and if and how it could be improved. Evaluation of EAP triggers typically follows three main questions: (1) Did we learn something new about the elements that form the basis of the trigger? (Is one of the elements different from what it was when triggers were initially defined? Has anything changed about the datasets we use that requires a review of triggers?); (2) Do we know more about the accuracy of the forecast models used? (How accurate was the forecast compared to the actual event? Are the forecast models used still the best available?); (3) Were the probabilities and impact levels of the original trigger appropriate? This evaluation process will guarantee a continuous learning and improvement of the trigger system.

Once EAPs have been triggered and evaluation data have been collected and analysed, there are many opportunities to learn more about the adequacy of the FbF system, including triggers, early actions, implementation procedures, effectiveness and beneficiary impacts and discuss discrepancies between expectation and results, the effectiveness of the activation and necessary changes to the EAP and MEAL system. These questions may best be answered by bringing together all staff and stakeholders involved.

Organize a workshop to host an EAP activation learning exercise to contribute to the understanding of all parties involved about the performance of their EAP activation, and to help promote learning and accountability.

Following a recent FbF Early Action Protocol (EAP) activation, the exercise typically takes a 1-3 day workshop format and brings together all.

More specifically, the learning exercise:

• Provides a space for implementers and partners to capture key learnings after an EAP activation.
• Provides space to evaluate: NS performance, EAP implementation feasibility and triggers.
• Generates lessons learned that can be shared across the implementing organizations and the wider FbF community, as applicable.
• Identifies recommendations to the FbF team (management, staff and partners) for improving the EAP and/or other early action-related plans and decisions.

It is important to capture the results of the data collection and analysis (the impact study or evaluation) and the outcomes of the workshop in a format that ensures that the findings are understood and utilized, for example, to inform an improvement of the EAP or the action implementation process. Therefore, before writing a report, it should be clear who the main audience is, e.g. the FbF project team; individuals in government departments; donor and partner staff; development professionals working with similar projects or programmes; students; or community groups?

Regardless of who the target audience is, the report should be readable, straight to the point, and use a plain language writing style that promotes understanding and ensures accessibility to the information for all stakeholders.
It is good practice to follow the “One-Three-Twenty-Five (1:3:25) principle”: a 1 page outline, a 3 page executive summary and 25 pages to present the findings and methodology. Use visual aids (charts, organigrams, tables) where they help to clarify the main messages, and remove any “chart junk” that does not add informative value. It is suggested to use descriptive chart titles so that charts can stand alone without needing to read the evaluation report to understand the information they convey.

A good evaluation report contains these basic components:

• An executive summary containing a condensed version of the most important aspects of the evaluation (see previous point).
• A summary of the evaluation’s focus, with a discussion of the purpose, objectives and questions used to direct the evaluation.
• A summary of the evaluation plan.
• A discussion of the findings of the evaluation, with complete statistical and case study analysis. A discussion of the evaluation’s conclusions and recommendations.
• Any additional information required, such as terminology, details of who was involved in the evaluation, etc. in an appendix.

Resource:

• Better Evaluation section on “Report and Support Use of Findings”

A maximum of 90 days after the activation is completed, financial and narrative reports need to be submitted to IFRC in line with the requirements of the Funding Agreements under FbA by the DREF.

Beyond the reporting obligations, if the evaluation has brought about learnings that are relevant for other stakeholders involved in FbF or anticipation or contributes to building evidence for FbF, consider to publish the results, for example in the form of a research report or a case study.

Once an EAP has been activated, the EAP must be updated and resubmitted to the IFRC in order to be re- validated and receive new funding from the FbA by the DREF. The inclusion of an analysis of activation learnings including trigger, early actions, and roles and responsibilities is a key component to qualify for revalidation.