Mid-term reviews of LDCF - “Implementing NAPA priority interventions to build resilience in Sudan”

Background

Purpose:

In accordance with applicable policies for UNDP/GEF projects, all GEF-funded projects implemented by UNDP are subject to a mid-term and a final independent evaluation. The purpose for this independent Mid-Term Evaluation is to undertake at the end of the third year of implementation an evaluation that will determine progress being made towards the achievement of outcomes and will identify course correction if needed. It will focus on the effectiveness, efficiency and timeliness of project implementation; will highlight issues requiring decisions and actions; and will present initial lessons learned about project design, implementation and management. Findings of this review will be incorporated as recommendations for enhanced implementation during the remainder of the project’s term. The evaluation is to be undertaken in accordance with the “GEF Monitoring and Evaluation Policy” (see related link).

Project description:

The project is being co funded by the Least Developed Countries Fund (LDCF) for Adaptation to climate change (USD 3,300,000), and UNDP Sudan Country Office (USD 500,000) and in kind support from the government of Sudan (USD 3,000,000). The project is being implemented through the National Implementation Modality (NIM) by the Higher Council for Environment and Natural Resources (HCENR). In terms of project ‘supervision’ (as opposed to ‘implementation’), UNDP is the GEF Implementing Agency and provides strategic, technical and administrative support to the HCENR.

The project is currently implementing NAPA priority interventions to build resilience in the agriculture and water sectors to the adverse Impacts of Climate Change in Sudan which aims to minimize and reverse the food insecurity and enhance adaptive capacity of small-scale farmers and pastoralists resulting from climate change, including variabilities in (originally 5) vulnerable regions representing the different ecological settings in Sudan now reduced to (4 regions) due to cessation of the South Sudan. The project is to be implemented over four -year period, having started in November 2009 and expected to end in late 2013.

This Midterm Evaluation will take place, therefore at a crucial moment for the project, when key activities are beginning to show results, the project is gaining visibility all through the states where it operates.

The PRODOC has identified three components (outcomes) outcomes and several results-oriented indicators. The outcomes that will contribute to the achievement of the project

Objectives are:

• Resilience of food production systems and food insecure communities in the face of climate change
• Institutional and individual capacities to implement climate risk management responses in the agriculture sector strengthened
• A better understanding of lessons learned and emerging best practices captured and up-scaled at the national level

Duties and responsibilities

Evaluation audience:

This Mid-term Evaluation of the UNDP/LDCF Project is initiated by UNDP as the GEF Implementing Agency. It aims to determine progress being made towards the achievement of outcomes and will identify corrective actions, if needed. It aims to provide managers (the Project Team, the Implementation Agency (HCENR), UNDP-Sudan Country Office and UNDP-GEF at all levels) with strategy and policy options for more effectively and efficiently achieving the project’s expected results and for replicating the results. It also provides the basis for learning and accountability for managers and stakeholders.

The Evaluation will highlight issues requiring decisions and actions; and will present initial lessons learned about project design, implementation and management.

Evaluation objectives and scope:

The overall purpose of the evaluation is to measure the effectiveness and efficiency of project activities in relation to the stated objective so far, and to produce possible recommendations on how to improve the management of the project until its completion in 2013. The Mid-term Evaluation serves as an agent of change and plays a critical role in supporting accountability. Its main objectives are:

• To strengthen the adaptive management and monitoring functions of the Project;
• To ensure accountability for the achievement of the LDCF objective;
• To enhance organizational and development learning;
• To enable informed decision-making;

Particular emphasis should be put on the current project results and the possibility of achieving all the outcomes in the given timeframe, taking into consideration the speed, at which the project is proceeding. More specifically, the evaluation should assess:

Project design and its relevance - The evaluators will assess the project design. They should review the problem addressed by the project and the project strategy, encompassing an assessment of the appropriateness of the objectives, outcomes, outputs, planned activities and inputs as compared to cost-effective alternatives. in relation to:

• Development priorities at the national level;
• Stakeholders – assess if the specific needs were met;
• Country ownership / drivenness – participation and commitments of government, states, local authorities, and communities;
• UNDP mission to promote assisting the country to build its capacities in the focal area of adaptation to climate change;
• Meeting the LDCF adaptation guidelines: Demonstrating increases in adaptive capacity and resilience for climate change and assess whether and how the engagement of communities has had a particular contribution and added value to community adaptation to climate change.

Project outcomes, outputs and indicators - The evaluation will assess the outcomes, outputs, and indicators achieved by the project as well as the likely inroads to sustainability of project results. This should encompass the following:

Attainment of objectives and planned results:

• Evaluate how, and to what extent, the stated project objectives are being achieved; taking into account the “achievement indicators”. In addition, the team will assess the indicators matrix as to its utility for determining sustainability and replicability impact.
• Assess the level to which the project has followed guidelines of the LDCF Strategic Priority on Adaptation and recommend ways to further strengthen this linkage.

Achievement of outputs and activities:

• Assess the scope, quality and usefulness of the project outputs produced so far in relation to its expected results.
• Assess the feasibility and effectiveness of the work plan in implementing the components of the project.
• Assess the quality, appropriateness and timeliness of the project with regard to:
• Satisfying the following GEF objectives;
• Delivering global environmental benefits; and
• Achieving financial and environmental sustainability for the project intervention.

Management arrangements focused on project implementation:

• General implementation and management: evaluate the adequacy of the project, implementation structure, including the effectiveness of the Project Board, partnership strategy and stakeholder involvement from the aspect of compliance to UNDP/GEF requirements and also from the perspective of “good practice model” that could be used for replication;
• Financial accountability and efficiency: assess efficiency against the so far achieved results, including an assessment of the National Implementation Modality and the cost effectiveness of the utilization of LDCF resources and actual UNDP co-financing for the achievement of project results; Assess the contribution of in-kind co-financing to project implementation and to what extend the project has been able to leverage additional funding so far.
• Monitoring and evaluation on project level: assess the adoption of the monitoring and evaluation system during the project implementation, focusing to relevance of the performance indicators, that are Specific; Measurable; Achievable and Attributable; Relevant and Realistic and time bound (SMART indicators)

Timeframe:

Considering the time left till the project’s foreseen termination, the difficulties faced by it in its first three years of implementation and the resources effectively available for programming, is the timeframe set still realistic? If applicable, outline recommendations for revising this timeframe with proposed benchmarks for the remainder of the project implementation time.

Overall success of the project with regard to the following criteria:

• Sustainability - assessment of the prospects for benefits/activities continuing after the end of the project,
• Changes: Assess any changes that may have resulted from the project implementation and its impact.
• Contribution to capacity development - extent to which the project has empowered target groups and have made possible for the government and local institutions to use the positive experiences; ownership of projects’ results;
• Replication – analysis of replication potential of the project positive results in country and in the region,
• Synergies: with other similar projects, funded by the government or other donors.

In addition to a descriptive assessment, all criteria should be rated using the following divisions: Highly Satisfactory (HS), Satisfactory (S), Marginally Satisfactory (MS), Unsatisfactory (US) with an explanation of the rating.

Issues of special consideration

The Evaluation Report will present the experience and recommendations for the benefit of design and implementation of other LDCF-funded adaptation projects. Especially, the aspects of developing ecosystem resilience will be looked into, including the ways of improving the protection modalities to enhance the ecosystem functions and maintain its services in the face of climate change risks. Identification of nature-based solutions to improve coastal resilience to sea level rise, increasing storminess and flood events will be learned, based on this evaluation. capacity for adaptation, communication and awareness-raising to support climate change adaptation, integration of climate change risk considerations and adaptation into policy and planning processes, as well as the specific management practices for natural resources to support adaptation to climate change, shall be assessed.

For future development support in the region, UNDP is especially interested in the assessment of the support model applied in the project, its implications for the long-term impact and sustainability of the project results.

The Evaluation Report will present recommendations and lessons of broader applicability for follow-up and future support of UNDP and/or the Government, highlighting the best and worst practices in addressing issues relating to the evaluation scope.

Evaluation methodology:

An outline of an evaluation approach is provided below; however it should be made clear that the evaluator is responsible for revising the approach as necessary. Any changes should be in-line with international criteria and professional norms and standards (as adopted by the UN Evaluation Group). They must be also cleared by UNDP before being applied by the evaluation team.

The evaluation must provide evidence-based information that is credible, reliable and useful. It must be easily understood by project partners and applicable to the remaining period of project duration.

The evaluation should provide as much gender disaggregated data as possible.

The evaluation will take place mainly in the field. The evaluation team is expected to follow a participatory and consultative approach ensuring close engagement with the UNDP Country Office, states’ governments, Higher council for Environment and Natural Resources, the Ministry of Environment and Physical Development, Project Board, project team, and key stakeholders.

The evaluation team is expected to consult all relevant sources of information, such as the project document, project reports – incl. Annual Reports (PIRs), project budget revision, progress reports, project files, national strategic and legal documents, and any other material that the team may consider useful for evidence based assessment.

The evaluation team is expected to use interviews as a means of collecting data on the relevance, performance and success of the project. The evaluation team is also expected to visit the project sites.

The methodology to be used by the evaluation team should be presented in the report in detail. It shall include information on:

• Documentation reviewed;
• Interviews;
• Field visits;
• Questionnaires;
• Participatory techniques and other approaches for the gathering and analysis of data.

Although the evaluation team should feel free to discuss with the authorities concerned, all matters relevant to its assignment, it is not authorized to make any commitment or statement on behalf of UNDP, GEF, LDCF or the project management.

The Evaluation team should reflect sound accounting procedures and be prudent in using the resources of the evaluation.

Deliverables:

• The output of the mission will be the Evaluation Report in English. The length of the Report should not exceed 30 pages in total (not including the annexes).
• Initial draft of the Evaluation Report will be circulated for comments to UNDP (both CO and Bratislava Regional Office), and the Project Manager. After incorporation of comments, the Evaluation Report will be finalized.

Timing and duration:

The evaluation will be conducted by two evaluators. An International and a national consultant. The total duration of the evaluation will be 20 days for the International consultant and 15 days for the national consultant, to start within the period of 1st – 15th November 2012, according to the following plan:

Home based desk review (5 working days):

• Collection of and acquaintance with the project document and other relevant materials with information about the project;
• Familiarization with relevant policy framework in Sudan;
• Design the detailed evaluation scope and methodology (including the methods for data collection and analysis);
• Set up the mission dates and detailed mission programme preparation in cooperation with the Project manager and UNDP CO. The Project manager will organize the schedule of the mission and will arrange transportation for the consultant; will arrange for translation/interpretation when necessary;
• Communication with the project staff to clarify matters.

Mission to Sudan (10 working days):

• Briefing with the PMU
• Visits to project sites
• Meeting with the National Project Manager, project Board members and stakeholder groups
• Presentation of main findings to UNDP and project management on the final day of the field visit.

Elaboration of the draft report -home based:

• Additional desk review

The write up will be led by the International Consultant (5 working days) with support from the national consultant:

• Completing of the draft report
• Sharing the draft report for comments and suggestions
• Additional information and further clarification with UNDP, project management and project staff
• Report finalization will be the sole responsibility of the International consultant
• Incorporation of comments and additional findings into the draft report
• Finalization of the report

The draft Evaluation report shall be submitted to UNDP for review within 15 working days after the mission. UNDP and the stakeholders will submit comments and suggestions within 7 working days after receiving the draft.

The finalized Evaluation Report shall be submitted latest 31st December 2012

Competencies

• Conceptual thinking and analytical skills;
• Computer literacy.

Required skills and experience

Education:

• University degree in technical, economics or environment related issues.

Experience:

• Recent experience with result-based management evaluation methodologies;
• Recent experience in evaluation of international donor driven projects;
• Recognized expertise in the field of natural resource management and climate change adaptation issues.
• Work experience in relevant areas for at least 8 years;
• Project evaluation experiences within United Nations system will be considered an asset.

Language requirements:

• Excellent English communication skills;

The evaluation team must be independent from both the policy-making process and the delivery and management of assistance. Therefore applications will not be considered from evaluators who have had any direct involvement with the design or implementation of the project, or have conflict of interest with project related activities. This may apply equally to evaluators who are associated with organizations, or entities that are, or have been, involved in the delivery of the project. Any previous association with the project, the Executing of national implementing Agency or other partners/stakeholders must be disclosed in the application. This applies equally to firms submitting proposals as it does to individual evaluators.

If selected, failure to make the above disclosures will be considered just grounds for immediate contract termination, without recompense. In such circumstances, all notes, reports and other documentation produced by the evaluator will be retained by UNDP.