Terms of Reference (ToR) for Midline Evaluation
1. General Information
Post/Title:
Midline Evaluation of the Education Sector Program Implementation Grant (ESPIG)
Program
GPE Funded ESPIG
Reporting to:
Program Manager – Education Sector Program Improvement Grant
Duration
60 days
Closing date
2nd May 2021
2. Background Information
2.1. About Education Sector Program Implementation Grant (ESPIG)
The Global Partnership for Education (GPE)-funded Education Sector Program Implementation Grant (ESPIG) is aligned with the priorities of the Federal Government of Somalia’s Education Sector Strategic Plan (ESSP 2018-2020). The overall grant objective is to increase access to quality education for more than 50,000 out–of–school children; enhance the quality of primary education; and improve the capacity of the MOECHE at Federal/Member States to regulate and manage the education sector. The grant is being implemented by the Federal Government of Somalia Ministry of Education, Culture and Higher Education (MoECHE) in collaboration with the Federal Member States Ministries of Education in Jubaland State, South West State, Galmudug State, Hirshabelle State and Benadir regional administration with CARE as the Grant Agent and Concern Worldwide as sub-contractor.
The main objective of the ESPIG is to improve equitable access to and quality of education outcomes for all Somali primary school children through strengthened system capacity to design evidence-driven ESSP reforms and collaborate effectively with partners in their implementation. ESPIG was envisioned to maximize the potential of development investments in education in Somalia through the combination of four key principles:
· Use of evidence-based approaches to address key gaps in access, learning, and retention, and inform tailored efforts to reach marginalized groups;
· Creation of conditions to implement solutions that address the intersection of multiple barriers to access, retention, and learning;
· Generation of synergies with privately managed schools/ networks, development actors, and the private sector to ensure complementarity of efforts and leverage existing capacity/ investment;
· Capacity building of federal, state, regional, and district-level actors for a cohesive, efficient, and dynamic approach to the design and implementation of solutions.
The program’s five components are described below:
· Equitable access to quality education will increase through changes in three intermediate outcomes: Increased access to education for out-of-school children; strengthening the capacity of community education committees, and community actions towards improved school safety.
· Enhanced quality of education, leading to grade-appropriate improved learning outcomes, will be achieved through the following intermediate outcomes: effective monitoring of teacher education and management policy, strengthened and harmonized efforts in teacher pre-service training; improved access to teaching and learning materials; and a strengthened assessment framework.
· Enhanced system capacity to regulate and manage the education sector will emerge from the combination of the following intermediate outcomes: (i) strengthening regulatory and monitoring system for private, community, and government schools and (ii) improved capacity of education officers for planning, budgeting, policy implementation, coordination, and progress tracking.
· Build a strong framework for program monitoring, accountability, and communication, generating evidence to track progress, inform adaptive management, and support planning and management processes.
· Effective and efficient program management, embedded within government systems at the local level and leveraging the Grant Agent’s capacity and experience in-country and globally
2.2. Program beneficiaries
Direct program beneficiaries will include: (i) about 50,000 out-of-school poor or marginalized primary children who will benefit from school inclusion grants; (ii) about 600 Community Education Committees who will benefit from training to improve inclusion, security and learning at their schools; (iii) 6,570 teachers who will benefit from teaching guides associated to the new curriculum; (iv) 240 primary pre-service student teachers who will benefit from new teacher training institutes; (v) 2,000 teachers benefitting from tailored in-service teacher training; (vi) 297,168 Grade 1-8 students who will receive textbooks linked to the new curriculum; (vii) 150 education officers will benefit from professional development to improve skills in planning, coordination, and monitoring.
*3. The objective of the Midterm Evaluation
The Education Sector Program Implementation Grant (ESPIG) is seeking to procure the services of a consultancy company to conduct a midterm review to assess the degree of progress in the implementation of the ESPIG components; the extent to which the expected results of the ESPIG intervention have been achieved; and provide recommendations for the adaptation of intervention components and their implementation. The midterm evaluation analysis should take into consideration the current crises affecting Somalia, in particular the impact of the COVID-19 pandemic and political crisis, and how those have affected the implementation of activities.
3.1 Specific Objectives of the Midterm Evaluation
The consultant will critically evaluate the outputs and outcomes achieved under each component and how those have impacted on the situation of target groups and beneficiaries. This will require critical examination of the individual activities to understand their contribution towards the achievement of the results. The consultant is expected to reference changes in school conditions at the midline vis a vis the baseline and assess to what extent such changes vary by location and level of impact by crises. This review will cover a set of activities related to the project and as outlined in the project’s components, result framework, and implementation plans.
Component 1: Equitable access to quality education
Main activities:
· Provide capitation grants to target schools to increase access to education for out of school children
· Strengthen capacity of community education committees
· Community actions towards improved school safety **
Evaluation questions
· What was the impact of the intervention on children’s access to school? How did the impact vary per location and gender? To what extent did the intervention result in the enrolment of children from marginalized groups?
· What was the impact of the school grants component on school infrastructure?
· Are community education committees better prepared to manage schools? Have they developed skills and capacities to deliver needed services?
· What was the impact of the intervention on education outcomes in targeted schools, particularly in terms of enrolment, attendance, and retention?
· To what extent did the federal member states and communities in underserved areas benefit from the intervention?
· Were capitation grants implemented in an efficient and transparent manner? To what extent are community education committees and local education officials involved in their oversight?
· What were the main uses of capitation grants?
· What activities are being implemented by community education committees? Has the intervention contributed to changes in CEC activities (types of activities, support to subgroups of students, and/or how those activities are being implemented)?
Component 2: Enhanced quality of education, leading to grade-appropriate improved learning outcomes
Main activities
· Monitor teacher education and management policy
· Construct and equip two teacher training institutes (not yet implemented; one teacher training institute being constructed)
· Distribution of primary school textbooks
· Early grade assessments
· Tailored in-service teacher training
Evaluation questions
· What was the impact of the provision of primary school textbooks on promotion rates? How has these contributed to the implementation of the national curriculum?
· Are learners benefiting from increased access to teaching/learning resources? How?
· Are teachers benefitting from increased access to teaching/ learning resources? How?
Component 3: Enhanced system capacity to regulate and manage the education sector
Main activities
· Strengthen regulatory and monitoring system for private, community, and government schools
· Strengthen capacities of education officers for planning, budgeting, policy implementation, coordination, and progress tracking.
· EMIS system development and support
· Conduct joint review of the education sector
· Support ESA/ESSP processes
Evaluation Questions
· In what ways has these program enhanced fiduciary management and utilization of resources at the MOECHE and FMS MOEs?
· To what extent are the policies developed under the ESPIG, such as the private school policy and CEC training harmonization, being implemented?
· How have the various training, infrastructural support enabled MOECHE and respective state authorities to handle activities within their mandate effectively?
· To what extent is the new EMIS operational, compared to the previous version?
· How is the newly developed EMIS different from the previous versions? How does the system enable MOECHE to deliver services better?
· To what extent have planning processes, such as the JRES, ESA, and ESSP been supported by the ESPIG?
Component 4: Build a strong framework for program monitoring, accountability & communication
Main Activities
· Conduct evaluation studies – baseline, midterm study, and final evaluation
· Support the MoECHE and Federal Member States (FMS) to carry out monitoring activities to verify progress against ESPIG / ESSP indicators
· Learning and communication
· Feedback and Complaints Response Mechanism (FCRM)
Evaluation Questions
· How have the recommendations of the baseline survey informed adaptation and reprogramming efforts?
· Are the MoECHE and FMS Ministry of Education undertaking monitoring activities? Has support to monitoring activities enabled accountability and adaptive management?
· Are there platforms to share learning and/or communication? How is this working?
· How is the Feedback and Complaints Response Mechanism working? How can it be improved?
Additional Cross-cutting Questions
· How has Covid-19 affecting student learning? How are you managing the issues?
· How are the tiered ways of working (FGS/FMS) and relations influencing or affecting educational outcomes specific to the ESPIG Project?
4. Technical Approach and Methodology
The consultancy company is expected to propose a relevant methodology to assess the intervention’s contribution to the Somali education system, responding to the evaluation questions above. In addition, the evaluation will assess the program’s *relevance, efficiency, appropriateness, sustainability, coordination, and lessons learned*. The selected methodology should be able to show the extent to which the program achieved its intended results thus far and reached the intended target populations. The proposed methodology should also allow the program to identify and quantify the impact of contextual factors on the expected results.
4.1 Document Review
The consultant will be expected to review all relevant documentation in order to understand the assignment and the context beforehand. This will enable the consultant to gather secondary data that will complement the primary data that will be collected. Key documents include the program document (https://www.globalpartnership.org/content/program-document-education-sec…), JRES report (https://www.globalpartnership.org/sites/default/files/document/file/2020…). CARE will provide other relevant documentation on implementation once services are contracted.
4.2. Primary Data Collection
The primary data will be collected from various qualitative and quantitative data collection methods namely:
· School survey (longitudinally tracking the schools sampled at the ESPIG baseline study);
· Analysis of school enrolment records, retention and transition records, test scores and attendance records, and observation of students’ attendance.
· Interviews with CECs, teachers and headteachers;
· Key informant interviews with the regional and district officials, other MOE and MOECHE staff, and partner agencies;
· Case studies with beneficiaries, including but not limited to students (particularly girls) who have enrolled/ re-enrolled through capitation grants.
Other methods may be added as appropriate.
4.3. Data Collection, Processing, and Analysis
The consultant is expected to train enumerators on data collection methodologies and tools; quality assurance’; confidentiality; ethics and informed consent. CARE will provide training on child protection and prevention of sexual harassment, exploitation, and abuse (PSHEA). In addition, the consultant will be expected to supervise data collection in the field, translation, and transcription of qualitative data, ensuring quality is maintained throughout the entire process. The data collection tools will be prepared and shared with CARE before the training and thereafter piloted and revised accordingly after the pre-test exercise to ensure they are acceptable and match the program’s needs. The Consultant will then analyze the data collected, prepare the midterm report and present the findings, conclusions, and recommendations to CARE who will then share it with consortium partners, the donor, and other key stakeholders.
*5. Deliverables
Deliverables under this consultancy will include the following:
· Inception report: Including the proposed methodology, data collection tools, analysis framework, and a detailed work plan.
· Data collection report
· Complete datasets and syntax files used for analysis.
· Complete transcriptions of qualitative data and respective audio/video/photo files.
· Draft report: Will be presented to CARE for input and feedback. CARE will give feedback within seven days of receipt of the draft report.
· Final report inclusive of CARE’s feedback.
· Summary of findings and presentation.
*6. Duration and Time
The exercise is expected to commence immediately after the contract is signed by all parties. The estimated deadline for completion is October 2021.
7. Desirable Qualifications:
This assignment requires previous experience in assessing large-scale system development programs and in particular, large-scale education programs, preferably in the region and/or in Somalia. In addition, the consultancy company has experience in working in fragile, conflict-affected contexts. The consultant will be required to devise appropriate strategies to generate as much information as possible within the allocated time frame, considering the ongoing crises and potential challenges to reach some locations.
Education and background required:
· Minimum 8 years of working experience in evaluation, qualitative and quantitative analysis and
demonstrated technical expertise in the large-scale education programs. Evidence documents like contracts of experience and sample work will be required.
· Advanced university degree in one or more of the disciplines relevant to the following areas: Evaluation expertise, economics, and social sciences.
· Excellent report writing skills in English. Somali language proficiency desired.
· Adequate knowledge of the context, including the Somalia education system, policies, and guidelines.
· Previous experience of conducting community-based and school surveys in Somalia, evidence documents like contracts of experience and sample of work will be required.
· Ability to roll out large-scale data collection and ensure quality in the South Somalia context.
8. Core Values & Critical Key Competencies
· Strong analytical skills with strong ability to do editing and proofreading
· Ability to think critically and strategically in difficult conditions
· Excellent interpersonal communication, relationship building, and networking skills
· Ability to work under pressure and meet tight deadlines in stressful conditions**
9. Administrative/ Logistical Support
9.1 Budget
The consultancy company will submit to CARE a financial proposal covering consultancy fees, operational costs, reimbursable, flight costs, accommodation in line with the methodology described in the technical proposal.
9.2 Logistics
CARE will provide the following support towards the successful execution of this consultancy:
· Provide all relevant internal documents needed during the desk review.
· Provide linkages with key stakeholders and actors at all levels to facilitate key informant interviews.
· Pay consultant’s fees upon satisfactory completion of the assignment.
10. Conditions of Work
The consultant will be supervised by the ESPIG Program Manager with support from ESPIG Education Advisor, M&E Lead, and the Field office heads. The consultant will be required to abide by the organization’s applicable rules and regulations, code of conduct, and child safeguarding policy. CARE upholds the principle of working independently of political, commercial, military, or religious objectives. CARE has a zero-tolerance approach toward sexual harassment, exploitation and abuse, and child abuse.
*11. How to apply
All interested consultants/firms are requested to write an expression of interest following the attached EOI format ONLY by email to SOM.Consultant@care.org. Please indicate ‘Mid Term Evaluation – Education Sector Program Implementation Grant’’ as the subject heading. Application deadline 2nd May 2021. Any canvassing will lead to automatic disqualification.
How to apply
All interested consultants/firms are requested to write an expression of interest following the attached EOI format ONLY by email to SOM.Consultant@care.org. Please indicate ‘Mid Term Evaluation – Education Sector Program Implementation Grant’’ as the subject heading. Application deadline 2nd May 2021. Any canvassing will lead to automatic disqualification.