Final evaluation of Africa Adaptation Programme (AAP)

Background

With funding of $92.1 million from the Government of Japan, UNDP launched the programme, “Supporting Integrated and Comprehensive Approaches to Climate Change Adaptation in Africa (hereafter called the Africa Adaptation Programme or AAP)” in partnership with the United Nations Industrial Development Organisation (UNIDO), the United Nations Children’s Fund (UNICEF) and the World Food Programme (WFP). The AAP assists 20 countries across the African continent in incorporating climate change risks and opportunities into national development processes to secure development gains under a changing climate. The Programme helps countries establish an enabling environment and develop the capacity required at local and national levels to enable them to design, finance, implement, monitor and adjust long-term, integrated and cost-effective adaptation policies and plans that are robust within a wide range of possible changes in climate conditions.

Within the framework of the AAP, Mozambique started the “Climate Change Adaptation Action and Mainstreaming” project in March 2010, seeking to mainstream climate change adaptation mechanisms in policy, development and investment frameworks. The key implementing partners for this initiative are the National Institute for Disaster and Risk Management, Ministry of Environment and Ministry of Planning and Development. The project is also being implemnted in collaboration with the following institutions: Eduardo Mondlane University; National Institute of Meteorology; Technical Secretariat for Food Security and Nutrition, Ministry of Agriculture; and the National Directorate of Water.

Similarity to other AAP countries, the Mozambique the project includes five major outputs, namely : (i) Establishment of long term planning mechanisms that will address the most pressing CC risks in Mozambique, along with the production of a suite of sector specific risk and vulnerability assessments which will increase sectors’ awareness of the respective potential impacts and costs, as well as opportunities of CC; (ii) Strengthened CCA leadership and institutional frameworks to effectively manage CC risks and opportunities, and provide guidance to investments, where the agreement on the long-term institutional arrangements for CCA is key ;(iii) Enhanced CCA policy framework, including climate resilient polices and measures in priority sectors, where small scale pilot adaptation projects at the community level are particularly important to generate lessons learnt on successful adaptation in Mozambique; (iv) Explore national adaptation financing options for CCA, where awareness and capacity of MF and the involvement of FUNAB are central; and (v) Effective capitalization of CC knowledge, including communication, awareness, and outreach actions targeting community level and the public, as well as high-level decision makers.

The project strongly emphasizes the development of capacities of beneficiaries from GoM (at the national, provincial and district levels), communities, students and civil society.

Main activities include mainstreaming of climate change in plans and budgets, national and sectoral policies and strategies support to the establishment of early warning systems and disaster risk management mechanisms, training of national technical staff on themes related to Climate Change, implementation of pilot projects and adaptation measures, establishment of mechanisms for sharing best practices, and capitalization of CCA knowledge.

These activities are being implemented mostly at the central/national level, with some interventions in Gaza (Chicualacuala, Massangena and Chigubo), Nampula, and Zambezia and Cabo Delgado provinces.

This final evaluation will produce an evaluation report containing a detailed list of lessons learned. The evaluation report is aimed at critically assessing the stages of the AAP and its products through participatory approaches, measuring to what extent the objective/outputs/activities have been achieved against the results and resources framework, and identifying factors that have hindered or facilitated the success of the project. The lessons learned section is aimed at capturing key lessons to assess what adaptation approaches/measures were effective in various thematic areas (e.g. water, agriculture, health, disaster risk reduction, coastal zone management) at multiple special scales (e.g. national, sub-national, local levels). This part is therefore forward-looking and is aimed at promoting AAP’s lessons so that the legacies of the AAP will be replicated and sustained beyond the project lifetime.

Duties and responsibilities

AAP Mozambique will be evaluated using the following criteria: relevance, effectiveness, efficiency, timeliness, and sustainability. The final evaluation will focus on the following aspects: A) project objective/outputs; B) processes; C) sustainability of results; D) monitoring and evaluation; and E) conclusions and lessons learned. For each aspect, a wide array of factors will be considered, including but not limited to:

Project objective/outputs:

• Effectiveness and efficiency of project activities
• Progress in the achievement of outcomes/outputs, measured against the baselines and indicators set at the outset of the project (- Annex 1- Updated Results and Resources Framework).

Processes

• Institutional arrangement

- Formulation and implementation stages
- Consultative processes
- Technical support by global and regional teams during formulation and implementation
- Capacity building initiatives
- Assumptions and risks
- Project related complementary activities

• Partnerships

- Assessment of national level involvement and perception of partners
- Assessment of local partnerships and their involvement
- Assessment of collaboration between government, non-governmental organizations, the private sector, and regional/international organizations

• Processes and administration

- Project administration procedures
- Milestones (log-frame matrix, RRF)
- Key decisions and outputs
- Project oversight and active engagement by UNDP Country Office and the project board
- Coordination between UNDP Country Office and government executing agency

• Disbursements

- Overview of actual spending against budget expectations
- Analyze disbursements to determine if funds have been applied effectively and efficiently

• Budget procedures

- Effectiveness of project document to provide adequate guidance on how to allocate the budget
- Audits and any issues raised in audits and subsequent adjustments to accommodate audit recommendations
- Review budget revisions and provide an opinion on the appropriateness and relevancy of such revisions

• Coordination mechanisms

- Appropriateness and efficiency of coordinating mechanisms and approaches between implementing partners and oversight bodies
- Propose improved coordination mechanisms and approaches

Sustainability of results:

• Evaluate AAP’s strategy to promote the sustainability/replicability of results
• Identify evidence showing that the results/lessons of AAP have been replicated to other regions/countries/communities
• Analyse risks to ensuring sustainability of the project outcomes and results (i.e. country ownership, financial, institutional capacity)

Monitoring and evaluation:

• Identify problems/constraints, which impacted on successful delivery of the project identified at the project design stage and subsequently as part of the Mid-Term Review (MTR)
• Identify threats/risks to project success that emerged during implementation and strategies implemented to overcome these threats/risks
• Analyse impact of MTR recommendations
• Assess the Monitoring & Evaluation systems and plans, whether they were well designed, implemented and budgeted, and their contribution to the compulsory quarterly and annual reporting processes at the national and regional levels
• Assess the extent, appropriateness and effectiveness of adaptive management at all levels of the project implementation

Conclusions, lessons learned:

• Assess substantive reports (e.g. risk assessment, progress reports of certain adaptation measures, lessons learned documents)
• Identify key lessons emerging from countries
• Identify effective approaches/measures (by sector and spatial scale)
• Identify elements hindering or promoting success

Expected outputs

The consultant will be expected to produce:

• An inception plan. The plan should outline the overall strategies, actions and timeline of the evaluation.
• An evaluation report. The report should not be more than 40 pages. It should be structured along the outline indicated in Annex 2. It includes a detailed lessons learned component and a list of all people interviewed in annex.

A consultant will be recruited. S/he will undertake evaluation through the following 3 main steps:

• Review of documentation (home-based);
• Interviews in the field with stakeholders (mission);
• Follow-up inquiries by phone/email and develop final products (home-based). Before the mission, the consultant will coordinate closely with project manager and respective UNDP Officer to get necessary documents for home-based desk review and schedule mission appointments.

Competencies

Competency in the following is required:

• Demonstrated ability to assess complex situations in order to succinctly and clearly distil critical issues and draw forward looking conclusions
• Prove the facilitation skills with a trade record of minimum 5 years

The evaluation will be undertaken in-line with the following principles:

• Independence
• Impartiality
• Transparency
• Disclosure
• Ethical
• Partnership
• Competencies and Capacities
• Credibility
• Utility

Required skills and experience

Education:

• Minimum academic qualification of a Masters’ Degree in Sustainable Development or Monitoring and Evaluation or Climate Change Adaptation or Environment or related areas.

Experience:

• The consultant should ideally have minimum experience of 5 years, with the following competencies and attributes:
• Capacity building and strengthening institutions
• Policy framework strengthening/mainstreaming
• Climate change adaptation
• Good knowledge of the UNDP Evaluation Policy; Results Based Evaluation Policies and Procedures and NIM Guidelines and Procedures;
• Knowledge of Result-Based Management Evaluation methodologies and the participatory monitoring approaches;
• Experience applying SMART indicators and reconstructing or validating baseline scenarios;
• Some prior knowledge of the Africa Adaptation Programme and working experience in Africa will be considered an asset.

Language requirements:

• Excellent Portuguese and English writing and communication skills