Evaluability Assessment and Formative Evaluation of the Explosive Ordnance Risk Education (EORE) programme in Syria (International Consultant), Damascus, Syria, 70 working days within 4 months

UNICEF works in some of the world’s toughest places, to reach the world’s most disadvantaged children. To save their lives. To defend their rights. To help them fulfill their potential. 

Across 190 countries and territories, we work for every child, everywhere, every day, to build a better world for everyone. 

And we never give up. 

For every child, commitment

 

BACKGROUND (RATIONALE AND CONTEXT):

Twelve years of crisis in Syria has created a long-term threat of contamination of Explosive Ordnances (EO)[1] to children, men and women, with catastrophic humanitarian consequences. Approximately a third of communities across Syria are affected by some form of explosive contamination^[2] and according to the Humanitarian Needs Overview 2023[3], 11.5 million people are potentially at risk from the threat of explosive ordnance (EO).

Although the actual scale and scope of the EO contamination in Syria are still unknown as no comprehensive countrywide survey has taken place, EO affects access to agricultural fields and local markets for livelihood needs, as well as access to schools and other public buildings and hospitals, as well as private residences. For the children of Syria, the impact of EO on their ability to play safely in their communities is especially severe. An additional risk is to the movement of refugees and Internally Displaced People (IDPs) in Syria through unsafe roads, areas and homes that have not yet been cleared of mines or EO, as well as communities returning to former homes.

It is estimated that between 100,000 and 300,000 EO have failed to detonate in contaminated areas and are placing communities in Syria at risk[4] but to date, the actual scale and scope of the explosive contamination in Syria is still unknown as no comprehensive countrywide survey has taken place[5].

United Nations Mine Action Service (UNMAS)-coordinated casualty reports recorded 12,345 EO casualties in Syria between 2013 and 2020, resulting in 4,389 deaths and 7,956 injuries.[6]

UNICEF Syria EORE programme has focused on promoting safe behaviour through providing children with skills and knowledge to protect themselves from risks associated with the dangers of explosive ordnance through various modalities including: 1) community-based risk education, 2) school-based risk education, and 3) communication (including mass media).

UNICEF Syria EORE programme is currently operating in all 14 governorates through 17 local partners including Ministries, INGOs, NGOs and Syrian Arab and Palestinian Red Crescent who, in turn, work through trainers, facilitators and field coordinators.

There is no theory of change for the EORE programme as it is one component of the UNICEF Syria Child Protection programme. The programme logic is that if people receive risk education, and they understand what is communicated, then they will be more knowledgeable about risks and if they use their improved knowledge to adopt safe behaviour, then there will be a reduction of casualties. In terms of measuring results, the EORE programme target in 2023 is for 1,460,000 children to be reached with landmine or other explosive weapons prevention and/or survivor assistance interventions[7], which only measures reach rather than measuring knowledge or behaviour change.

In terms of coordination, UNMAS is the Mine action Area of Responsibility lead in Syria and leads the Technical Working Group.

How can you make a difference? 

 

OBJECTIVES OF THE ASSIGNMENT (PURPOSE OF THE ASSIGNMENT):

The purpose of the consultancy is to conduct an evaluability assessment and formative evaluation of UNICEF Explosive Ordnance Risk Education. This exercise comes at an important time to generate evidence and learning to inform the future strategic direction of the UNICEF Syria EORE programme including evidence-based programming and resource mobilisation.

The evaluation is both a reflective exercise on approaches taken by UNICEF Syria Country Office for EORE programming and a forward-look on what are the building blocks for effective EORE programming in Syria, including programme structure, processes and resources, enabling factors and how they can deliver future results for children in Syria.

The objectives of the evaluability assessment and formative evaluation are detailed in the attached TOR for the assignment

Scope:

The time period that this evaluation will cover is from 2021 – 2023; EORE activities have taken place prior to this time period but 2021 onwards represents a clearer programmatic approach that it would be useful to reflect on.

It will cover activities across the 14 governorates in Syria where UNICEF operates through 6 field offices.

The draft evaluation questions can be found in the attached TOR, in which they indicate the main area of interest and guide the focus of the assignment.

 

Methodology

This evaluation will be conducted in accordance to the 2016 United Nations Evaluation Group Norms and Standards for Evaluation[8] and the UNICEF Evaluation Policy (2018)[9].

The methodology of the consultancy and related deliverables should focus on the following three areas:

Review of existing monitoring and reporting data

1. Review of programme and project documentation, including a detailed mapping of available key documentation identifying key gaps. Available data includes but is not limited to 4Ws reports, programme visit reports (which include inquiries around accountability to affected populations), and regular reporting mechanisms. UNICEF Syria EORE programme monitoring is done through partner reports, validated by third party monitoring reports or field trips conducted by programme officers. Financial records from partners, and funds utilization information are also available. The evaluation should also review training and awareness-raising materials.

Primary data

2. Interviews and focus group discussions should be conducted with key internal and external stakeholders, based on programme stakeholder mapping that can be validated by UNICEF Syria EORE programme[10]. A potential key evidence source of secondary data on beneficiary knowledge and behaviour, as well as on programme implementation approaches, is the Third-Party Technical Facilitators, contracted by UNICEF Syria to provide technical support to partners on programme implementation in the field.
3. Small-scale primary qualitative data collection in Syria is possible via focus groups conducted by UNICEF Syria Independent Third-Party Monitoring data collectors in UNICEF-supported service centers run by implementing partners. The design and conduct of this data collection should be developed by the consultant during the evaluation design in consultation with UNICEF Syria EORE programme and PME section but will focus on small-scale appropriate ways to engage with beneficiaries to establish levels of awareness on risk education.  The consultant should design data collection tools, conduct training of and supervise the UNICEF Syria Third Party Monitors, and analyse the collected raw data[11]. If appropriate and if possible, in terms of access restrictions[12], efforts will be made for the consultant to conduct trainings and supervise data collection directly.

It is intended that this primary data collection could focus on establishing levels of knowledge and awareness of risks of explosive ordnance, helping to establish an indicative proxy baseline for future programming, as well as encouraging critical feedback on programme design and implementation to help inform future programming. The consultant will ensure appropriate training, supervision, and quality control of data collection remotely to minimize any bias and improve the credibility of the data. Any engagement with communities for data collection during the consultancy will be subject to UNICEF and UNEG ethical standards and would require ethical approval.

Review of data and results frameworks

4. The consultant should conduct a stocktake and assessment of monitoring and reporting systems, including mapping and assessment of data availability and indicators, and make recommendations for strengthening M&E systems to support monitoring, reporting and programme learning.

In particular, the consultant should consult with UNICEF Child Protection programme to i) reconstruct the current programme in an initial Theory of Change and then ii) propose an adjusted Theory of Change that better reflects the objectives of EORE programme and supports better data collection and evidence generation for future programming.

Risks and limitations

Current monitoring data is limited in terms of having been focused on quantitative indicators related to reach of risk education activities and materials. There is overall a lack of national data on mine risk and also a shortage of comparative data on mine risk mitigation.

Programme planning, monitoring and reporting data may be incomplete or not have generated enough information to undertake a meaningful assessment of UNICEF Syria EORE programming design, implementation or results. Where this is the case, the consultant should document the gaps, identify any possible proxies, and recommend how to improve in future.

Another risk is related to engaging with communities in Syria, and it is difficult to access individuals outside of UNICEF-supported facilities. Primary data collection is therefore envisaged in UNICEF-supported facilities.

It is important that the evaluation is timely in terms of the needs of the EORE programme and the recruited consultant will need to ensure adherence to workplan.

 

LOCATION (GEOGRAPHIC AREA/ DUTY STATION): Damascus, Syria

Off-site working days: 70 days, anticipated to be performed remotely 

 

DURATION:

The selected consultant will work for 70 days within a 4-month period. It is anticipated that the work will be done remotely. The exact schedule of the activities will be agreed with the consultant based on the contract implementation progress. The deadline for submission of final deliverables to UNICEF is by the end of the contract. Specific dates for the deliverables should be detailed and agreed in the inception report.

 

SUPERVISOR:

The consultant will be supervised and report to the UNICEF Syria Chief of Planning, Monitoring and Evaluation who will provide ongoing technical support and quality assessment of deliverables in line with evaluation standards, in close collaboration with UNICEF Syria Child Protection Officer, with guidance from UNICEF Syria Child Protection Specialist. On a day-to-day basis, the consultant is expected to work with a small working group formed of the UNICEF Syria Monitoring & Evaluation Specialist, UNICEF Syria Evaluation Specialist and UNICEF Child Protection Officer as programme focal point. A small Reference Group will be formed to provide technical guidance and quality review of deliverables and support to the management of the evaluation.

 

DESCRIPTION OF ASSIGNMENT (TASKS, SMART DELIVERABLES, DEADLINES):  

PROPOSED PAYMENT SCHEDULE

By Deliverables.

The payment of professional fees will be paid in installments against the agreed deliverables and accepted implementation report, as follows:

1. Approval of inception report by Reference Group linked to 30% payment of fees;
2. Approval of draft report by Reference Group linked to 30% payment of fees;
3. Approval of final report & final action plan by Reference Group linked to 40% payment of fees;

 

To qualify as an advocate for every child you will have… 

1. Education: Advanced (Masters or above) university degree in evaluation, social sciences, or related field. 
2. Work experience:

• At least five years of relevant professional work experience in research/evaluation activities including work experience in complex emergencies and protracted crises is required;
• Direct experience of conducting evaluations or research in the MENA region/in Syria is desirable;
• Experience conducting evaluability assessments/technical support for programme design/M&E is desirable;
• Experience conducting evaluations for UN agencies/UNICEF is desirable.

3. Technical knowledge/competencies:

• Knowledge and experience of child protection and risk mitigation programming is required;
• Knowledge of child protection in MENA region/in Syria is desirable;
• Experience in child protection/youth evaluation/evaluability/programme design/M&E in the field of humanitarian work is desirable.

4. Language
   • High proficiency in reading and speaking in English is required;
   • Proficiency in report-writing in English is required;
   • Proficiency in reading and speaking Arabic is required.

 

EVALUATION PROCESS (OF APPLICATIONS):

Qualified candidates are requested to submit:

1. Applications through UNICEF’s Talent Management System (TMS)

2. Cover Letter(1-2 pages) that highlights previous experience in evaluation (including any specific examples from working with UN and in MENA/Syria) and technical experience with evaluation/research of child protection, risk education include reference to at least two past work examples/experiences.

3. Technical proposal (2-4 pages) that details methodology, focus areas and challenges and mitigation measures.

4. Financial quote as daily rate for this consultancy;

5. CV. clearly stating the relevant past experience and similar assignments

6. At least 3 Referees from Direct Supervisors

Shortlisted applicants may be invited for further technical assessment. Final recommendation will be made based on “best value for money”, i.e. the hiring section/office shall normally select the individual who quoted the lowest fee from among the candidates who are assessed as suitable for achieving all tasks on time, as per the criteria stipulated in this ToR, and based on the outcome of the evaluation/assessment conducted.

 

For every Child, you demonstrate… 

UNICEF’s core values of Care, Respect, Integrity, Trust, Accountability, and Sustainability (CRITAS). 

To view our competency framework, please visit here. 

UNICEF is here to serve the world’s most disadvantaged children and our global workforce must reflect the diversity of those children. The UNICEF family is committed to include everyone, irrespective of their race/ethnicity, age, disability, gender identity, sexual orientation, religion, nationality, socio-economic background, or any other personal characteristic.

UNICEF offers reasonable accommodation for consultants/individual contractors with disabilities. This may include, for example, accessible software, travel assistance for missions or personal attendants. We encourage you to disclose your disability during your application in case you need reasonable accommodation during the selection process and afterwards in your assignment. 

UNICEF has a zero-tolerance policy on conduct that is incompatible with the aims and objectives of the United Nations and UNICEF, including sexual exploitation and abuse, sexual harassment, abuse of authority and discrimination. UNICEF also adheres to strict child safeguarding principles. All selected candidates will be expected to adhere to these standards and principles and will therefore undergo rigorous reference and background checks. Background checks will include the verification of academic credential(s) and employment history. Selected candidates may be required to provide additional information to conduct a background check. 

Only shortlisted candidates will be contacted and advance to the next stage of the selection process. 

Individuals engaged under a consultancy or individual contract will not be considered “staff members” under the Staff Regulations and Rules of the United Nations and UNICEF’s policies and procedures, and will not be entitled to benefits provided therein (such as leave entitlements and medical insurance coverage). Their conditions of service will be governed by their contract and the General Conditions of Contracts for the Services of Consultants and Individual Contractors. Consultants and individual contractors are responsible for determining their tax liabilities and for the payment of any taxes and/or duties, in accordance with local or other applicable laws. 

The selected candidate is solely responsible to ensure the health insurance required to perform the duties of the contract are valid for the entire period of the contract. Selected candidates are subject to confirmation of fully-vaccinated status against SARS-CoV-2 (Covid-19) with a World Health Organization (WHO)-endorsed vaccine, which must be met prior to taking up the assignment. It does not apply to consultants who will work remotely and are not expected to work on or visit UNICEF premises, programme delivery locations or directly interact with communities UNICEF works with, nor to travel to perform functions for UNICEF for the duration of their consultancy contracts. 

 TOR EORE_evaluation_2024 _final (TMS).pdf