International consultant & team leader: Mid term evaluation of community-based adaptation to climate change through coastal afforestation in Bangladesh

Project Context and Background

Bangladesh is highly vulnerable to the impacts of climate change. Climate change will exacerbate many of the natural hazards the country already faces, posing a significant challenge for future development. Multiple national assessments, including the Government’s own National Adaptation Programme for Action, have suggested that climate change impacts of particular relevance to Bangladesh will include the increased frequency and severity of climatic events such as flooding, cyclones and drought, leading to increased mortality and loss of assets and livelihoods; the undermining of macro-economic growth; reductions in food security; and increasing migration pressures. Climate change impacts are already posing threats in the coastal areas and the development efforts are increasingly at risk.

The threats are particularly acute for coastal communities living in the low-lying deltaic regions in Bangladesh. The Intergovernmental Panel on Climate Change Fourth Assessment Report of 2007 suggests that an expected sea level rise by up to 45cm, will directly affect the lives of 35 million people living in coastal areas. Without adaptation actions like coastal afforestation, the low lying coastal zones of Bangladesh are likely to experience a submergence of 17.5 percent of the country’s land mass, increasing salinity trends in coastal fresh water resources, growing drainage congestion, dynamic changes in coastal morphology and a decline in the functioning of protective ecosystems.

Recognizing Bangladesh as one of the countries worst affected by the impacts of climate change, the Government of Bangladesh is implementing the NAPA follow-up project ‘Community-based Adaptation to Climate Change through Coastal Afforestation (CBACC-CF)’ funded by the Least Developed Countries Fund (LDCF) and UNDP Bangladesh. The objective of the CBACC-CF project is to reduce the vulnerability of coastal communities to the impacts of climate change-induced risks, and to strengthen institutional mechanisms to support these communities to adapt to climate change impacts.

The project is the second LDCF-funded adaptation project in Asia and innovative in the way that it brings together climate change adaptation and sustainable economic development through coastal afforestation. As a pilot, the project is working across 14kms of Bangladesh’s 710km coastline which is particularly vulnerable to the impacts of climate change.

Objective

The overall objective of the project is to reduce vulnerability of coastal communities to the impacts of climate change-induced risks in four of the most vulnerable coastal areas of Patuakhali, Bhola, Noakhali and Chittagong Coastal Forest Divisions.

Project Outcomes

• Outcome 1: Enhanced resilience of vulnerable coastal communities and protective systems to climate risks;
• Outcome 2: Climate risk reduction measures incorporated into coastal area management frameworks;
• Outcome 3: National policies revised to increase climate risk resilience of coastal communities;
• Outcome 4: Learning, evaluation, and adaptive management enhanced.

Duties and Responsibilities

Objectives of the Mid Term Evaluation (MTE):

The purpose of this MTE is to examine the performance of the project since the beginning of its implementation. The review will evaluate progress in project implementation, as measured against planned outputs set forth in the Project Document in accordance with rational budget allocations and managerial processes involved in achieving those outputs, as well as the initial and potential impacts of the project, as measured by attainment of project outcomes and objectives. The review will also address underlying causes and issues contributimg to targets not adequately achieved.

The Mid-Term Review is intended to identify weaknesses and strengths of the project design and recommend necessary changes in the overall design and orientation of the project. It will assess the adequacy, efficiency and effectiveness of project implementation, and assess the status of project outputs and outcomes to date (including reasons for non-achievement). Consequently, the review mission is also expected to make detailed recommendations on the work plan for the remaining project period to help the project achieve its intended impact. It will thereby provide an opportunity to assess early signs of the project success or failure and prompt necessary adjustments. The findings and lessons learned from the MTE will be incorporated into the project to enable necessary adjustments in the work plan and the project document, and define timely and appropriate steps to sustain project activities after 2012.

Scope of the Mid Term Evaluation

The MTE will comprise the following elements:

• Assess whether the project design is clear, logical and commensurate with time and resources available;
• Provide a summary evaluation of the project and all its major components undertaken to date, and determine progress towards achievement of the project’s objective, outcomes and outputs;
• Review project performance in relation to the indicators, assumptions and risks specified in the strategic results framework matrix and the project document;
• Assess the scope, quality and significance of projects outputs produced to date in relation to expected outcomes;
• Analyze the extent of local and national stakeholder participation and involvement in the project;
• Assess the functionality of the institutional structure and implementation mechanisms established by the project and the role of the Project Board, Steering Committee, and any additional Technical Support and Advisory bodies;
• Identify and, to the extent possible, quantify any additional outputs and outcomes that have been achieved beyond those specified in the project document;
• Identify any substantive programmatic and financial variance and/or adjustments made during the first two years of the project, their conformity with decisions of the Project Board and their appropriateness in terms of overall objectives of the project;
• Evaluation the effectiveness and efficiency of project coordination, management and administration provided by the Project Management Office. This evaluation should include specific reference to:
(i) Organizational/institutional arrangements for collaboration among the various partner institutions involved in project execution; (ii)The effectiveness of the monitoring mechanisms currently employed by the project manager in monitoring on a day to day basis the progress in project execution; (iii) Administrative, operational and/or technical problems and constraints that have influenced the effective implementation of the project (including recommendations for necessary operational changes and alignments); and (iv) Financial management of the project, including the balance between expenditures on administrative and overhead charges in relation to those on the achievement of substantive outputs and outcomes;
• Review the financial planning and sustainability of the project, including the timely delivery and use of committed co-financing;
• Assess the extent to which project outputs to data have scientific credibility and potential for replication;
• Assess the extent to which scientific and technical information and knowledge have influenced the execution of the project activities;
• Assess the degree to which the overall objectives and expected outcomes of the project are likely to be met by the end of the project;
• Summarize key lessons learned during project implementation;
• Recommend any necessary corrections and adjustments to the overall project work plan and timetable for the purposes of enhancing the achievement of project objectives and outcomes.

Other issues to be considered

• The GEF and UNDP are paying particular attention to risk analysis and management. UNDP has developed a risk management system within ATLAS and guidance on using this system, which is also now incorporated in the annual Project Implementation Review (PIR).
• The evaluators are requested to determine how effectively the risk management system is being used as an adaptive management tool.
• Risks may be of a financial, socio-political, institutional, operational, environmental (or other) type;
• Considering that UNDP is concerned about poverty reduction, local governance and the promotion of gender equity through a rights-based approach, the review requires looking at these cross cutting issues;
• Describe the main lessons that have emerged in terms of: strengthening country ownership; strengthening stakeholder participation; application of adaptive management strategies; efforts to secure sustainability; knowledge transfer;
• Capacity Development: Assess the extent to which national project implementing partners have been adequately trained and enhanced their capacity to take over technical and professional responsibilities as envisaged in the project design.

Products Expected from the Evaluation

The review team will provide the following deliverables to UNDP, UNDP/GEF-LDCF and the Project Board:

• A presentation of key MTE findings to key stakeholders;
• An executive summary, jointly prepared by the reviewers, including key findings and recommendations;
• A detailed evaluation report covering point 3. Above (’ Scope of the Mid-term review’) with detailed attention to lessons learnt and recommendations; and
• List of annexes prepared by the consultants including TOR’s, itinerary, List of Persons interviewed, summary of field visits, list of documents reviewed, questionnaire and summary of results, co-financing and leveraged resources, etc.

The report together with the annexes shall be written in English and shall be presented in electronic form in MS Word format.

A possible structure for the evaluation report is as follows:

• Executive Summary (Brief description of project, context and purpose of the evaluation, main findings, conclusions and recommendations)
• Introduction (Purpose, methodology and structure of evaluation)
• The project and its development context (Problems that the project seeks to address; key objectives, outcomes and outputs; expected impact);
• Findings and Conclusions:
• Project formulation and design, Project management and implementation and Project results and impact;
• Recommendations: Revisions in the Strategic Results Framework, Adjustments in project management/implementation arrangements, Adjustments in project monitoring and evaluation, Actions to follow-up or reinforce initial benefits from the project.

MTE Team Composition and Responsibilities

The MTE mission will comprise an international and a local consultant which together form the evaluation team. The international consultant will be the Team Leader (TL) and is required to have an in depth understanding of UNDP and GEF projects including evaluation experience. The international consultant will have the overall responsibility for developing the evaluation methodology, leading the evaluation, and delivering the key products expected from the evaluation, including coordinating the inputs from the national consultant. The national consultant will provide professional back up and support with local consultations, translation, and arrangement of local meetings.

The consultants will meet with all project partners and institutions and gather information and opinions on implementation-related processes such as project management and coordination, fund release mechanisms, and project management capacity and evaluate to what extent each of them has been supportive to the project delivery. They will visit field sites and consult extension agents and communities and directly record issues, benefits and gaps, and relate them to the project achievements. Finally, the consultants will compile the findings into a report highlighting both constraints and opportunities, formulating lessons learned and suggesting corrective measures for the remaining part of the project period.

The consultants will sign an agreement with UNDP Bangladesh and will be bound by the terms and conditions set out in the agreement.

Methodology

The MTE will be conducted in a participatory manner, starting with a desk review of relevant project documentation (including the approved Project Identification Form (PIF), the final UNDP project document, the inception workshop report, the Strategic Results Framework of the project, annual budgets and work plans, the annual Project Implementation Review (PIR), Project Board and meeting minutes as available, and other technical reports and documents as relevant). A list of key documents is given in Annex 1.
The evaluation methodology should be clearly documented in the final evaluation report including comprehensive details on the following:

• Documents reviewed;
• Interviews conducted;
•  Consultations held with all key stakeholders;
• Project sites visited;
• Techniques and approaches used for data gathering, verification and analysis.

The evaluation team will work independently but is required to liaise closely with the UNDP Country Office (CO) and implementing partners from the Ministry of Environment & Forest, Forest Department, Bangladesh Forest Research Institute, Ministry of Fisheries and Livestock, Ministry of Land and Department of Agriculture Extension. The team will visit the project sites to ensure adequate consultation with key stakeholders. Towards the end of the field evaluation, the findings will be presented to all key stakeholders in Dhaka.

While the evaluation team is free to determine the actual layout of the final evaluation report, the document must include the minimum content requirements mentioned earlier under section 4 “Products Expected from the Evaluation”. The TL will submit a draft report to key stakeholders for review and comments, and the final report to UNDP Bangladesh for onward distribution to all stakeholders. The evaluators will be responsible for the contents, quality, timeliness and veracity of the report.

Tentative Schedule for the MTE

The MTE will take place in October 2011 and requires 7 days country mission in Bangladesh as well as a desk review (prior to the country mission) and drafting and finalization of the report (following the country mission). The draft evaluation report should be submitted to the UNDP CO for circulation to relevant stakeholders within two weeks after the completion of the evaluation mission to Bangladesh. The consultants will finalize the report within two weeks upon receiving comments and feedback from stakeholders compiled by UNDP CO and APRC.

Special Tasks for International Consultant/Team Leader and Qualification

The international consultant will report to the Assistant Country Director, UNDP Bangladesh, and act as the Team Leader (TL) for the MTE. He/she will have overall responsibility for the work and operation of the evaluation team, including the coordination of inputs from different team members and stakeholders. The TL will be accountable for the timeliness and quality of the agreed outputs. He/she will:

• Provide guidance to the national consultant in undertaking the MTE;
• Undertake a thorough desk research of existing project-related documents, survey/research/evaluation reports and field reports;
• Conduct fieldwork together with national counterparts and interview stakeholders, extension agents and communities to generate authentic information/opinions;
• Identify and summarize key lessons learned;
• Provide guidance and specific recommendations on how the project team and UNDP can improve performance (both substantive and management) during the remaining duration of the project;
• Make a presentation of key findings highlighting project strengths, weaknesses, opportunities and risks to relevant decision makers and stakeholders;
• Write and finalize the evaluation report.

Evaluation

Individual consultants will be evaluated based on the following methodology: Cumulative analysis.

When using this weighted scoring method, the award of the contract should be made to the individual consultant whose offer has been evaluated and determined as:

• Responsive/compliant/acceptable, and
• Having received the highest score out of a pre-determined set of weighted technical and financial criteria specific to the solicitation.
- Technical Criteria weight; 70%
- Financial Criteria weight; 30%

Only candidates obtaining a minimum of 49 points (Out of 70 points) will be considered for the Financial Evaluation.

• Technical-70 Points
• Overall experience in the provision with services- 30 points
• Substantive experience in reviewing and evaluating similar projects, preferably GEF, LDCF, SCCF or other United Nations development agencies or major donors-15 points
• Knowledgeable of participatory monitoring and evaluation processes-15 points
• Working Experience with UNDP-10 points
• Financial-30 points.

Competencies

• Excellent English writing and communication skills;
• Demonstrate ability to assess complex situations, succinctly distil critical issues from a broad array of stakeholder feedback, and draw forward-looking conclusions and recommendations;
• Highly knowledgeable of participatory monitoring and evaluation processes, and experience in evaluation of technical assistance projects with major donor agencies;
• Ability and experience to lead multi-disciplinary and national teams, and deliver quality reports within the given time;
• Familiarity with the challenges developing countries face in adapting to climate change;
• Familiarity with the Bangladesh context; and
• Excellent in human relations, coordination, planning and team work.

Required Skills and Experience

Academic qualifications:
• Postgraduate University degree in environmental sciences, social science, geography and/or relevant disciplines;

Experience:
• 10 years of relevant experience and professional background in fields related to Climate Change Adaptation, Disaster Risk Reduction, Community Development or Integrated Coastal Zone Management;
• Substantive experience in reviewing and evaluating similar projects, preferably those involving GEF, LDCF, SCCF or other United Nations development agencies or major donors;
• Highly knowledgeable of participatory monitoring and evaluation processes, and experience in evaluation of technical assistance projects with major donor agencies.