Two education consultants/emergencies - evaluation education in emergencies capacity development

Background

The UNICEF West and Central Africa Regional Office (WCARO) is looking for technical expertise from individual consultants to conduct a multi-country evaluation study of its emergency preparedness and response (EPR) and conflict/disaster risk reduction (C/DRR) capacity development and institutionalization efforts in the education sector. A team of two consultants will be required to undertake the work in up to 6 countries that have agreed to take part in this exercise. This is, therefore, an invitation to individual consultants with adequate expertise and capacities in evaluation, particularly of capacity development initiatives in EPR and C/DRR, to submit applications to conduct this evaluation study between November 2011 to end of February 2012. Consultants will work the equivalent of 3 working months during this time period.

Scope and Focus

The study will look at both individual and institutional capacity development bearing in mind that as a result of the workshops and other capacity development activities, participants representing the national and decentralized levels were expected to have learned the following competencies/be able to perform the following tasks:

1. Be aware of and learn the key technical components, approaches and quality standards of effective emergency response in education through interactive and participatory approaches, from conducting rapid assessments and establishing temporary learning spaces to developing emergency education curricula and devising psychosocial support strategies;
2. Have the knowledge and skills to apply these technical components for strengthened preparedness at national and decentralized levels. Participants develop action plans for capacity development activities for improved preparedness and disaster risk reduction in the most emergency-prone localities;
3. Indicate priority advocacy and policy action to inform national education sector policy, planning and budgeting for strengthened preparedness, response and risk reduction so that education in emergencies is addressed in a more systematic and sustainable manner;
4. Map capacity of education actors at national and local levels for emergency education and identify roles and responsibilities for effective coordination through new or existing mechanisms including education clusters;
5. Initiate the process of contingency planning for the education sector at local levels in coordination with the relevant disaster management authorities;
6. Plan for the roll-out of further capacity development at local levels, as required.

- The capacity development strategy adopted addresses both emergency preparedness and response (EPR) by improving UNICEF and its partners’ capacities to respond, but also by mitigating the damage resulting from future crises through the institutionalization of emergency preparedness and disaster risk reduction (DRR) in the education sector’s policy and planning. EPR and DRR institutionalization - meaning the vertical (centralized/decentralized) but also horizontal (all MoE’s departments/sections) mainstreaming of emergency awareness throughout the MoE’s structure, policy and planning and its linkage with overall disaster management structures - was suggested as the best strategy to ensure the overall capacity development intervention was effective and sustainable.

- For emergency preparedness and DRR to become embedded in ministries’ actions at all levels, a multi-step approach is required. This involves sensitization, awareness raising, mentality change, training, institutionalization (roles, responsibilities, structures, etc.), behavioral change and ultimately emergency risk informed programming.

- Key indicators of impact of the above should be reflected in: MoE contingency planning; integration of EPR/DRR into education sector policy, plans and budgets curricula, EMIS, Medium-Term Expenditure Frameworks at national and district levels, school and community plans; MoE participation in national disaster management; co-ordination mechanisms and identification of roles and responsibilities for EPR/DRR at the national and sub-national levels; sector review documents; MoE participation in education clusters; meeting minutes; utilization of the training in the event of actual emergencies.

- The study will analyze the contribution of capacity development on institutionalization with the understanding that institutionalization should contribute to building resilience but this study will not actually measure resilience. The evaluation is both summative to assess the outcome as donor funding is coming to an end and formative since support for EPR and DRR will continue in the future.

- The EEPCT Programme has already been reviewed/evaluated twice but this new proposed study focuses on the Program’s capacity development and institutionalization components, responding to recommendations made in the two previous studies (see Annex for details). The proposed study will expand the evidence base by incorporating other countries that were not covered in the two previous studies. The geographical scope of the evaluation and the selection of countries to be evaluated will be done in cooperation with UNICEF WCARO and Country Offices, taking into account the following criteria:

- Balanced inclusion of countries according to potential emergency scenarios (man-made and natural disaster, chronic emergency, complex emergencies);

- Preference for countries with high levels of vulnerability;

- Stratification of sample according to phases of emergencies and transitions (mid-crisis, early recovery, or transitional); 

- Stage/progress of the institutionalization efforts as reported by the UNICEF Country Offices;

- Ability of the country to facilitate or absorb an additional external mission at relatively short notice.

Based on the above mentioned criteria, countries would need to be chosen from the 24 countries covered through the UNICEF West and Central Africa Regional Office. The validity of the evaluation is premised on its being conducted by a completely independent source, although UNICEF will provide logistical support and access to government and other partners at the country level.

Purpose of the study

The study will systematically and objectively assess the effectiveness and sustainability of the EPR/DRR capacity development strategy and its contribution to institutionalization in the education sector in the West and Central Africa Region and draw lessons for future programming including the development of a robust monitoring and evaluation framework. The study is both necessary and timely as it is expected to:

• Help country offices identify and understand achievements and challenges arising from capacity development/institutionalization efforts, including the perspectives of participants and duty bearers;
• Provide the country and regional offices with recommendations to inform decision-making at a critical time i.e. the end of the current EEPCT grant;
• Provide evidence for future advocacy and fund raising efforts;
• Identify a way forward for enhancing the future quality of EPR and DRR capacity development and institutionalization in the education sector, including the development of robust planning and M&E frameworks.

Objectives

The proposed evaluation study has 4 objectives:

1. To assess the adequacy, effectiveness and sustainability of the regional (ToT) and national (Roll-out) EPR/DRR capacity development activities (with a focus on outcomes and impact).
• What did participants actually learn and how is it applied?
• How did trained staff react to an emergency compared to untrained staff (where applicable)?
• What were the achievements and the challenges in terms of the competencies that those who were trained developed and were they sustained?
• How far has the cascade training approach gone and what has the impact been at each level, including the community/school level (where applicable)?
• What mechanisms have been put in place to ensure the sustainability of the activities and how effective are they?
• How are the activities monitored and evaluated at the regional and national levels and how can this be enhanced?

2. To assess how the capacity development strategy has contributed to the institutionalization of EPR and DRR at national/education sector level (national policy and planning) and at sub-national level (as applicable).
• How have trainings and other capacity development activities helped to build effective institutional capacities?
• How have capacity development activities been translated into a process towards EPR/DRR integration into the education sector’s policy and programming at the national and sub-national levels?
• What are determining factors that made that process happen?
• What is the role of the education cluster or sector coordination group in this integration?
• To what extent are roles and responsibilities clear and institutionalized at all levels of the system?

Additional questions raised by the EEPCT Programme Review and Evaluability Study or PRES:
• ‘Are issues of conflict vulnerability and disaster risk reduction accounted for within the design and implementation of [MoE] programme activities?
• Are the contextual features of areas affected by chronic crises also appropriately considered?’

3. To identify and analyze any other initiatives at the CO level which have contributed to enhanced EPR and DRR in the education sector.
• What other capacity development activities (such as cluster coordinator trainings, needs assessment training, awareness raising and sensitization on Education in Emergencies, longer-term support to integrate C/DRR into Education Sector Plans etc.) if any, have taken place that have impacted on EPR and DRR at the country level and what impact have they had both in terms of capacity development and institutionalization?
• What are the linkages with other sectors (WASH, Protection, Health, Nutrition, Communication for Development etc.)?

4. To identify and recommend measures to address capacity gaps and institutionalization failures to inform future EPR/DRR capacity in WCAR and develop a clear road map and logical framework for the way forward.
• Where are the existing capacity gaps and what are the potential strategic responses including key actions required for effective EPR/DRR institutionalization?
• What are the principal lessons learned – both positive and negative?
• What key elements need to be incorporated into future planning and M&E?

Methodology

The evaluation study will take place in 4 steps:

(i) Inception Phase (Home base)
- The consultant team will produce an overall roadmap of the consultancy which shows in a work plan the tasks, the deliverables and deadlines against the TOR. This road map will be known as the Inception Report.
- The Inception Report will include a thorough description of the scope of the exercise and the methodological approach for data collection and analysis.
- For the data collection, a combination of tools will be expected, and will include, inter alia, interviews, written or oral tests of trainees, on line surveys, focus groups and multi-stakeholder consultations, using an appreciative inquiry approach.
- For the analysis, an evaluation framework will be developed that lists and defines each evaluation criteria to be used by the evaluators. Quantitative but also qualitative indicators will have to be developed to measure the EPR/DRR institutionalization by the MoE and its partners.
- The framework will explicitly analyze if the capacity development/institutionalization strategy adequately insured the inclusion of Human Rights, Gender (including gender-based violence), Equity and Results Based Management, and the extent to which the integration of EPR/DRR into education sector planning was inclusive of these issues.
- The Inception Report will be the subject of discussion in a meeting with the Reference Group.
- The study design will take the following recommendation into account from the EEPCT Programme Review and Evaluability Study: “future evaluative work may attempt to undertake standardized approaches (such as knowledge, attitude and practice (KAP) surveys) to measure the effectiveness of previous or ongoing capacity development activities (including trainings and workshops)”.

(ii) Orientation and Desk Review Phase (Home base + Dakar)
- An initial distance briefing meeting (through Webex or teleconference) or through initial mission of consultant team to Dakar will take place with the Regional Education Adviser, the Regional Chief, Monitoring and Evaluation, the Education Specialist (Emergencies) who will comprise the Reference Group for the study.
- For the Desk Review, core documents and contacts will be made available by the Education Specialist (Emergencies).
- These documents will include previous EEPCT Studies, training materials, list of participants, training evaluations, national capacity development roll-out plans, national sectorial policy documents and documentation relevant linked to other capacity development activities (beyond training).

(iii) Implementation Phase/Country Visits
- The consultants will organize country missions depending on availability of the country.
- The logistics of data collection will be supported by the UNICEF Education Specialist (Emergencies) and the country offices.
- Data collection will include the following steps:
• Initial consultation with UNICEF Country Office staff;
• Structured interviews with key stakeholders (see also list of informants below): programme managers at UNICEF and NGO, project managers from (implementing) partners, officials from the MoE and other relevant ministries at the national and sub-national level (where applicable), co-lead and members of education clusters (where cluster approach is activated) or sector coordination groups;
• Participant profile analysis to define the level of institutional change that the capacity development initiative could have influenced and that which it could not;
• Review of key documents – policies, plans and budgets, monitoring and evaluation reports, etc. at national, sub-national and school levels (if applicable);
• Group discussions on draft findings with UNICEF and partners in country.

Consultants are expected to spend up to a maximum 10 days in each selected country to ensure an in-depth study of capacity development activities at national, but also at the sub-national level. UNICEF will provide logistical support and access to government and other partners at the country level.

(iv) Reporting Phase
- A draft mid-term report will be delivered to a Reference Group for feedback. Final approved Report will be due latest on 29 February 2012.

Required Skills/Competencies

The consultant should combine the following competencies:

• Strong expertise in education sector policy and planning;
• Demonstrated experience in evaluation, particularly of capacity development approaches and of institutional change programs;
• Understanding of conflict/disaster education programming;
• Some exposure to/knowledge of UNICEF work in education in the humanitarian/post-conflict sector;
• Demonstrated understanding of EPR/DRR capacity mapping and capacity development approaches in the education sector;
• Knowledge of global emergency education issues and how these “translate” in the West and Central Africa Region;
• Demonstrated technical skills in the field of conflict/disasters, particularly the social and human dimensions of disaster impact, and understanding of the concepts of ‘risk’, ‘exposure’ and ‘vulnerability’ and of how disaster impacts are mediated by different factors;
• Analytical and conceptual ability; good communication skills;
• Effective written and verbal communication skills in French and English required;
• Good inter-personal skills with cultural sensitivity;
• Experience in working for UNICEF and/or other UN or humanitarian organizations an advantage;
• Personal and professional integrity;