Terms of Reference (ToR) For External Evaluation of Desert Locust Joint Response

1. Background of the Joint Response

ThisTOR is developed by SOS Children’s Villages for performing an external evaluation of the Dutch Relief Alliance (DRA) Joint Response Program in response to the Desert Locust Crisis (DLJR), financed by the Dutch Ministry of Foreign Affairs (MFA). The response is anacute response with a duration of 6 months, with a start date on the 22^nd of April 2020 until the 21^st of October 2020.

While the Dutch Relief Alliance has 16 partners, 8 selected partners take part in this response: Cordaid, Dorcas, ICCO and ZOA in Ethiopia, Save the Children, Oxfam, SOS Children’s Villages (SOS CV) and Tearfund Netherlands in Somalia and Somaliland. SOS Children’s Villages Somaliland is the lead organization for this consurtium. The prioritized regions within this response are:

1. Ethiopia:Babile; East Hararghe Zone; Oromia Region; Somalia Region; Moyale-Barana; GursumWoreda
2. Somaliland: Sool, Awdal, Maroodi Jeh, Sahil and Togdheer regions
3. Somalia: Nugal (Puntland),

Based on the needs and gaps identified in the Ethiopia, Somaliland and Somalia HRP and the global COVID HRP, the priority sectors within this humanitarian response are Food Security and Livelihoods, Water Sanitation & Hygiene, Health and Multipurpose Cash.The response aims to support people affected by the locust plague. The total amount allocated to the response is 4 million EUR.

In line with the Ethiopia and Somalia HRP, and the global Covid-19 HRP, the response contributes to the following objectives:

1. Ethiopia HRP 2020:
2. SO 1: Reduce morbidity in geographically prioritized areas.
3. SO 1.1 Food needs of 4.6 million acute food insecure people are met.
4. Somalia HRP 2020:
5. S01 Prevalence of acute malnutrition and health needs affecting more than 1.8 million people reduced by the end of 2020.
6. COVID appeal:
7. Prepare and be ready: prepare populations for measures to decrease risks, and protect vulnerable groups, including older people and those with underlying health conditions, as well as health services and systems.
8. Prevent, suppress and interrupt transmission: slow, suppress and stop virus transmission to reduce the burden on health-care facilities, including isolation of cases, close contacts quarantine and self-monitoring, community-level social distancing, and the suspension of mass gatherings and international travel.

Furthermore, the response carries out activities in the areas of Localization and Accountability, two DRA strategic objectives (See logframe). The evaluation can further build on Rapid Real Time Review (Annex 3) conducted in August 2020 .

2. Purpose of the evaluation

The objective of the evaluation is to provide a comprehensive assessment of the results of the DLJR acute response, this is done in order to learn from the implementation and to ensure downward and upward accountability. The evaluations aims:

1. To review the overall performance of the project on the ground, in particular to know the strengths, gaps and barriers to the project in implementation areas.
2. To analyze the design of the project and validate the assumptions of the intervention logic; and evaluate the efficiency and effectiveness of the project results
3. To analyze the projects results according to the OECD DAC evaluation criteria and the additional criteria mentioned under section 4.
4. To identify and record lessons learned, best practices, strengths and provide recommendations for improvement.
5. To identify the contribution of the DLJR to the strategic priorities of the Dutch Relief Alliance

3. Scope of the evaluation

The geographical locations of the project are: Awdal, Maroodijeh, Sahil, Togdheer, and Sool in Somaliland and Nugal in Somalia. And in Ethiopia: Babile, East Hararghe Zone; Oromia Region; Somali Region; Moyale-Barana; Gursum Woreda. The DLJR is targeting the population in Locust affected IPC3 and IPC3+, in the most food insecure areas of Somaliland,Somalia and Ethiopia. Women and children are disproportionately affected, and special efforts will be made to reach the most vulnerable women and children including PLW and children under the age of 5. The sectors addressed are Food Security & Livelihoods, MPC,Health (COVID-19 related) and WASH (COVID-19 related). In addition, the response carries out activities in the areas of Localization and Accountability (these are strategic objectives defined by the DRA and be found in the logframe). Protection is an important element and partners are expected to mainstream protection throughout the activities.

Indicator per sector

Activities

Food Security Livelihoods

1. SO 1: Reduce morbidity in geographically prioritzed areas.
2. SO 1.1 Food needs of 4.6 million acute food insecure people are met.
3. S01 Prevalence of acute malnutrition and health needs affecting more than 1.8 million people reduced by the end of 2020.

Number of people provided with resources to protect and start rebuilding livelihood assets

1. Supplementary livestock feed and veterinary service
2. Food for animals.
3. Awareness raising for households on preparedness and mitigation focusing on coping with the locust infestation
4. Provision of tractor hours and land preparation support
5. Provision of agricultural tools (wheelbarrow, rake, hoe, shovels, seeds, etc.)
6. Training for households on climate resilient agriculture

Number of people enabled to meet their basic food needs

1. Cash for Work (ground control interventions) and distribution of CfW tools
2. Cash Transfer for food assistance

WASH (related to COVID-19)

# HRP: COVID objectives

Number of people having regular access to soap to meet hygienic needs

1. Hygiene kit distribution
2. Distribution of Jerrycans

Number of people having access to sufficient and safe water for domestic use

1. Repair and rehabilitate 10 hand dug wells
2. Install handwashing stations
3. Installation/preparation of bore holes

Number of people reached with hygiene promotion/awareness raising activities

1. Hygiene promotion: Public information campaigns in the project areas on how to protect yourself and others from the COVID-19 virus to prevent the spread of COVID, to reach the most affected households.

HEALTH (related to COVID-19)

# HRP: COVID objectives

Number of outbreak alerts responded to

20. Training of 200 health care workers on COVID-19 drug guidelines and MHPSS

MPC

1. SO 1: Reduce morbidity in geographically prioritzed areas.
2. SO 1.1 Food needs of 4.6 million acute food insecure people are met.
3. S01 Prevalence of acute malnutrition and health needs affecting more than 1.8 million people reduced by the end of 2020.

Number of people benefitting unconditional and unrestricted cash

1. UCTto the most vulnerable households

Number of people benefitting from conditional and unrestricted cash

1. CfW financed rehabilitation of productive community assets;
2. ground control interventions and distribution of CfW tools;
3. manual locust control through trench digging including capacity strengthening on locust control;
4. working in soil bunds to mitigate land degradation; support in mechanical efforts controlling the locust under the lead of the national taskforce ToT.

Localisation

% of budget that went to national and local actors

1. Community mobilization, Benefeciary selection, registration and verificaton, Direct implementaion of the cash for work activities, disbursement of cash transfer, procurement, distribution of inputs

Number of national and local actors supported with capacity-building

1. Build capacity on Staff on CHS, Cash Transfer Programming, Financial systems and M&E using Open Data Kit (ODK)

Accountability

Number of crisis-affected people who are also involved in the design, implementation, montoring and/or evaluation of the programme

1. Local community members, including local leaders, women representatives and government officials involved in decision-making, design and implementation of the project, especially in terms of beneficiary selection, PDM, Post hygiene awareness survey
2. Periodic meetings with Local Government workers, community based health workers, local village leaders and social welfare personnel will be organized to collect inputs. Community complaint and feedback mechanism will be set up (a.o. help desk at registration and distribution points, suggestion boxes and a hotline).

Number of adaptations in the design and/or implementation of the JR, as a result of the input from beneficiaries.

1. Scaling up manual locust control with beneficiaries, as it has shown to raise awareness on the issue and strengthen the capacity of communities on locust control. Communities during a previous assessment highlighted they need more support on awareness raising and manual control, including tools.

4. Evaluation questions

For specific questions, please refer to Annex 2.

The evaluation will further built on the findings of the RRTR and is framed around the following Accountability criteria (linked to OECD-DAC criteria and other criteria):

1. Relevance and appropriateness:Assessing whether humanitarian activities are in line with local needs and priorities (and donor policies). It refers to the overall goal and purpose of a programme.
2. Effectiveness:the extent to which the activity achieves its purpose, or whether this can be expected to happen on the basis of the outputs. The recipients should be disaggregated for age (0-18 and >18) and sex (male and female). If possible, double counting of individuals that received various services/items/cash should be avoided. When avoiding double-counting is not possible, aggregation of reach data of various interventions within a joint response is to be reconsidered.
3. Efficiency: measures the outputs – qualitative and quantitative – in relation to the inputs. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been used.Cost-effectiveness looks beyond how inputs were converted into outputs, to whether different outputs could have been produced that would have had a greater impact in achieving the project purpose.

5. Methodology

As part of his/her assignment, the consultant will provide a detailed planning on his/her proposed methodologies based on these Terms of Reference. SOS-CV NL and SOS-CV Somaliland will review the planned methodologies proposed by the evaluator and provide feedback before the evaluation process begins.

1. Mixed qualitative and quantitative methods are required. Methods can be determined together with an evaluator, but finding stories of change as well as validation of outputs in the log frame is mandatory
2. Field work (collecting data) is preferable if possible with the existing COVID-19 restrictions
3. Links to resource portals to be included if relevant.

6. Deliverables

1. An inception report (outlining the methodology, complete tools and planned procedure and approach to the evaluation).
2. A briefing session on proposed methodologies and tools (venue to be mutually decided).
3. A debriefing and validation session to SOS CV Somaliland, SOS CV NL and other interested DRA partners to debrief on the major qualitative and quantitative findings before initiating the report
4. A draft and final report in English, not exceeding 25pages (excluding annexes). For a detailed table of content, please see Annex 3.

The report should include:

✓ Executive Summary (max 3 pages)

✓ Introduction

✓ Background and context

✓ Summary description of Methodology, including limitations

✓ Main findings

✓ Conclusions

✓ Lessons learned, best practices, recommendations

Annexes to the report should include:

✓ Work schedule

✓Photographs of the evaluation areas (separate file).

✓Bibliography of consulted secondary sources.

✓Finalised data collection tools.

✓List of people interviewed

✓Financial report supported by copies of related invoices

Note: Pictures and consent form signed by all interviewees is mandatory.

7. Team composition

[depending on discussion with consultant]

1. Team leader (at least 5 years of experience conducting evaluations of emergency programmes).
2. Local consultant (knowledge of the local context and at least 2 years of experience conducting in collecting data for evaluations of emergency programmes).

8. Contract and Budget

The lead agency for the Joint Response (SOS Children’s Villages) will provide a consultancy contract. During the visit in country, the evaluator and his/her team will resort under the security policies of SOS CV Somaliland. A total amount of 28.000 EUR is available for this evaluation. This should include fees of field visits and staff hired to complete the evaluation, write the final report and arrange for dissemination locally and in the Netherlands; as well as costs for local travel, accommodation, communication and so on. In case travel is not possible, we expect the consultant to propose suitable digital solutions, such as phone interviews or zoom meetings.

Payment schedule of fees:

1. 40% of fees upon approval of the inception report
2. 60% of fees after delivery, dissemination and approval of final report

Interested consultants are required to provide a budget including travel costs, number of working days per specific activity, daily rate and any other costs. Please consider that hard and soft copies of relevant documents will be provided by SOS CV NL.

1. Timeline

Activity

Deliverable

Indicative Dates

Recruitment of Evaluator

Published TOR

17-09-2020

Contracting of Evaluator

Signed contract

01-10-2020

Discussions with HQ

05-10-2020

Desk review of secondary data and project documents, further analysis to formalize methods and tools

Inception Report shared

13-10-2020

Inception with SOS-CV NL, DRA partners and local partners

15-10-2020

Data collection (keeping in mind that the response finishes 21/10/2020)

Data Collected

09-11-2020

Presentation of Initial findings at field office and debrief to

PPT presentation

23-11-2020

Draft sharing: internal review and inputs

Reviewed report with feedback

23-11-2020

Final Report Writing

Final Report with all feedback addressed

31-12-2020

Submission to donor

Final report

20-01-2021

2. Disclosure of information/ethics

It needs to be understood and agreed that the consultant shall, during and after the effective period of the contract, treat as confidential and not divulge, unless authorised in writing, any information obtained in the course of the performance of the contract. The ethics process and research needs to comply with specified requirements (e.g. Code of Conduct, Research Policy and Standards).

3. Required experience and competences

1. Demonstrated experience in leading Humanitarian response evaluations.
2. Experience with the DRA is an asset.
3. Proven experience in Humanitarian program management, preferably including consortia programs
4. Proven knowledge of humanitarian programmes and principles, including use of common standards, preferably in the sectors FSL, WASH, Health, Protection, MPC.
5. Ability to process and analyze different types of data in a critical manner
6. Ability to work effectively in intercultural settings. Knowledge of the Locustcrisis is an asset.
7. Excellent facilitation and reporting skills
8. Experience in the use of participatory research methodology
9. Able to write concise, yet comprehensive and attractive reports
10. Appropriate language skills in English and local language e.g. Somali and Amharic
11. Experience of working with participatory approaches and research methodologies
12. Excellent skills in Excel, Access and SPSS and other qualitative software (quantitative and qualitative skills)
13. Experience working in Ethiopia or Somalia/Somaliland is an asset

1. Annexes

Annex 1) Examples of DRA Acute Response Evaluations

Annex 2) Specific questions

Annex 3) Rapid Real Time Review

Annex 1: examples of DRA Acute Response Evaluations

Annex 2 – questions per criterion[1]

OECD-DAC criteria

Efficiency

1. Were activities of individual organisations and for the overall consortium cost-efficient?
2. Was the programme or project implemented in the most efficient way compared to alternatives?

Effectiveness

1. To what extent were the objectives achieved?
2. What were the major factors influencing the achievement or non-achievement of the objectives?
3. What can be done differently to complete the project more effectively?

Relevance/

Appropriateness

1. Did the project address the needs of the right target group?
2. Were the needs of target group(s) in project-targeted area met?
3. To what extent was the programme able to adapt and provide appropriate response to changing local needs and the priorities of the people?
4. Where relevant humanitarian standards considered?

Other relevant criteria

Timeliness

1. Was the programme approved in time by MoFA to be able to start in time?
2. Could funds be re-allocated in time during implementation to respond to new developments?
3. Was implementation done in a timely manner? (please also use the learnings from the RRTR)

International standards

1. How satisfied are beneficiaries with their involvement in the project?

Gender equality

1. How was gender equality taken into consideration in all relevant areas?
2. Did the programme conform to the implementing organisation‘s gender equality policy?

MPC

1. Where activities and outputs in line with the set objectives and targets?
2. What can we learn from MPC in Ethiopia versus Somalia? Are there different challenges and solutions found in each country? To what extend did harmonization and coordination take place between the partners on this topic?

FSL

1. Were the activities and outputs of the project consistent with the FSL objectives?

Objective 1:To increase timely, permanent and diverse access to a varied food basket for victims of violence, people in return or confinement processes or affected by natural disasters.

Objective 2:To prevent morbidity and mortality associated with malnutrition and nutritional deficiencies, with emphasis on children under five, pregnant and lactating women.

Health

Objective 1: To increase access to health services for the most vulnerable groups, communities confined and/or affected by recurrent emergencies within the framework of the armed conflict, widespread violence and/or double affectation (disasters and armed conflict).

Objective 2: To contribute to the reduction of health risks within emergency contexts, through attention and referral to comprehensive care routes in mental, sexual and reproductive health and perinatal maternal health.

WASH

1. Where the activities and outputs of the project consistent with the protection objectives?

11. Objective 1: To increase access to water for human consumption and adequate and equitable sanitation and hygiene services, including vector control and solid waste management

Inclusion

1. How were vulnerable and marginalised groups (e.g. elderly, disabled, children, people living with HIV) and other groups that suffer discrimination and disadvantage taken into consideration throughout the intervention cycle?

Other

1. What are the effects of the project on the organization (e.g., organizational pride, enhanced networking, and partnerships)?
2. what was the added value of working in a joint response form / consortium? E.g. more leverage in terms of advocacy?

Annex 3: Rapid Real Time Review

[1] This is a broad range of questions of which a selection of questions can be made