Team leader for UNDP/GEF project evaluation

Consultancy (International Consultant) Terms of Reference for the Terminal Evaluation of Sustainable Land Management Project, Vanuatu

Project background

The Medium Sized Project (MSP) on Building Capacity and Mainstreaming Sustainable land management in Vanuatu is a Global Environment Facility (GEF) funded project through the United Nations Development Program (UNDP). The project is implemented by the Department of Lands. The project commenced in April 7th, 2008 and due for completion on April 7th, 2011 but was granted an extension until June 2012 (following a project mid-term evaluation in 2011).

Despite the growing official recognition of the problem of land degradation in Vanuatu, SLM objectives have not been adequately mainstreamed into policies, regulations, strategies, plans and educational systems. There is a lack of understanding of on the part of decision makers that land degradation is significant barrier to sustainable development. Although integrated farming systems are a way of life for local communities, the planning of local resource utilization is mostly guided by more specific sectoral objectives and policies. This suggests a strong need to create awareness and build capacity for integrative dialogue and land use planning among all stakeholders.

The capacity gaps in land degradation include: i) individual level –lack of technical capacity (district level and community level for implementation); ii) institutional level – financial and human resources, monitoring capacity for enforcement of its rules and regulations); iii) lack of baseline data state and national level); iv) systematic level – there is a lack of common understanding and mechanisms to coordinate and address common land management issues.

Project objectives and expected outputs

Objectives : Objectives of the MSP are to enhance and develop the individual, institutional, and systemic capacity for Sustainable Land Management (SLM), to mainstream SLM considerations into national development strategies and policies, to improve the quality of project design and implementation in the development arena, to develop a National Action Plan for SLM, as well as a medium term investment plan, while ensuring that all relevant stakeholder views are reflected and integrated into the process.

Objectives of the evaluation

The Monitoring and Evaluation (M&E) policy at the project level in UNDP/GEF has four objectives: i) to monitor and evaluate results and impacts; ii) to provide a basis for decision making on necessary amendments and improvements; iii) to promote accountability for resource use; and iii) to document, provide feedback on, and disseminate lessons learned. A mix of tools is used to ensure effective project M&E. These might be applied continuously throughout the lifetime of the project – e.g. periodic monitoring of indicators, or as specific time-bound exercises such as mid-term reviews, audit reports and independent evaluations.

In accordance with UNDP/GEF M&E policies and procedures, all regular and medium-sized projects supported by the GEF should undergo a final evaluation upon completion of implementation. A final evaluation of a GEF-funded project (or previous phase) is required before a concept proposal for additional funding (or subsequent phases of the same project) can be considered for inclusion in a GEF work program. However, a final evaluation is not an appraisal of the follow-up phase.

Final evaluations are intended to assess the relevance, performance and success of the project. It looks at early signs of potential impact and sustainability of results, including the contribution to capacity development and the achievement of global environmental goals. It will also identify/document lessons learned and make recommendations that might improve design and implementation of other UNDP/GEF projects.

The overall objective of this TE is to review progress towards the project’s objectives and outcomes, assess the efficiency and cost-effectiveness of how the project has moved towards its objectives and outcomes, identify strengths and weaknesses in project design and implementation, and provide recommendations on design modifications that could have increased the likelihood of success, and on specific actions that might be taken into consideration in designing future projects of a related nature.

Scope of the evaluation

Overall evaluation of the project
The evaluation will address the following specific issues:

Project design

The evaluation will assess the extent to which the overall project design remains valid. The evaluation team will review the project’s concept, strategy and approach within the context of effective capacity development and sustainability. Specifically, the team will:
• assess the extent to which the underlying assumptions remain valid;
• assess the approach used in design and whether the selected intervention strategy addressed the root causes and principal threats in the project area;
• assess the plans and potential for replicating or scaling up the site-based experiences;

The evaluation team will also attempt to ascertain the current level of comprehension of the project concept, focusing on three specific sets of actors: (i) project management team; (ii) field officers; and (iii) local communities.

Project implementation

The evaluation will assess the extent to which project management and implementation has been effective, efficient and responsive. Specifically, it will:

• Assess overall institutional arrangements for the execution, implementation, management, monitoring and review of the project. This covers a number of issues, including: the appropriateness of joint implementation and coordination; whether there has been adequate periodic oversight of activities; the effectiveness of government counterparts; and the effectiveness of relationships between key stakeholders;
• assess the use of logical framework as a management tool during implementation;
• assess indicators of adaptive management;
• assess the quality and relevance of project reporting;
• assess the mechanisms for information dissemination (advocacy and awareness raising) in project implementation and the extent of stakeholder participation in management;
• analyze the project financing, specifically how the project has materialized/leveraged co-financing for various components (this is preferably presented in a matrix form).
• Review the effectiveness and the methodology of the overall Programme structure, how effectively the Programme addressed responsibilities especially towards capacity building and challenges, its main achievements and overall impact as well as the remaining gaps.
• assess the extent to which programme design, implementation and monitoring have taken the following cross cutting issues into consideration: Human rights, Equity, Institutional strengthening and Innovation or added value to national development

Results

The evaluation will examine the relevance, efficiency, effectiveness and sustainability of operational activities and results achieved by the project to-date, by showing how the component(s) processes and outcomes have contributed (or have the potential to contribute) to the achievement of project and GEF environmental goals. The Evaluation will:

• assess, quantitatively and qualitatively, the achievements and impact in terms of outputs and its contribution to outcomes as defined in the project document;
• assess to what extent the project has made impacts on promoting local participatory decision-making and local governance;
• assess to what extent the project has or will contribute to the strengthened enabling environment for conservation;
• Assess the sustainability of project results.

The evaluation team will use a project logical framework to determine the overall contribution of project outcomes to development and global environmental goals. The evaluation team is also invited to highlight contributions which are strictly beyond the project scope.

Governance and capacity-building

The Project promotes participatory processes and behavior that affect the way land use management is done at the local and national levels. This is principally achieved through the wide participation of local communities, capacity-building, and the promotion of accountability and transparency at different levels of government. In this regard, the Evaluation will look at how the project contributed to improved governance at local and national levels, and examine how governance issues have impacted on the achievement of project goals and outputs.

One of the specific areas the evaluation team is asked to assess in this area is how and to what extent the project has built management, planning and operational capacity among the project’s stakeholders, particularly at the community levels. This should include an overview of capacity-building techniques employed by the project as well as of the monitoring mechanisms involved.

Lessons learned

The evaluation will also highlight lessons learned and best practices in addressing issues relating to relevance, performance and success.

In describing all lessons learned, an explicit distinction needs to be made between those lessons applicable only to this project.

Methodology

The evaluation methodology will be determined by the evaluation team, guided by the requirements of GEF and UNDP as articulated in various guidelines, policies and manuals on the conduct of evaluations for GEF projects as well as key project documents such as the approved GEF project brief, the final UNDP project document, the inception workshop report, the project log-frame and annual budgets and work plans, the annual Project Implementation Review, Project Board, and PMT meeting minutes as available, and other technical reports and documents as relevant. The evaluation methodology should be clearly documented in the final evaluation report including comprehensive details of the following:

- documents reviewed
- interviews conducted
- consultations held with all stakeholders
- project sites visited
- techniques and approaches used for data gathering, verification and analysis

Conduct of the evaluation

Under the leadership of the Team Leader, the Evaluation Team will work independently but will liaise closely with UNDP CO, and Executing Agency. The evaluation mission will also liaise periodically with the UNDP-GEF Regional Technical Advisor (RTA) at the UNDP Regional Centre in Bangkok to ensure that UNDP-GEF and GEF requirements are being met.

The team will visit the project site to ensure adequate consultation with all key stakeholders. Towards the end of the field evaluation, presentation will be made to all key stakeholders in country. After the presentation the team will take note of verbal and/or written responses to its presentation and consider these in preparing an interim draft evaluation report that will be provided to Executing Agency/UNDP before the team leaves for distribution to stakeholders. The executing agency and UNDP will circulate the draft report to all stakeholders requesting written feedback and finalized by the evaluators within the dates reflected in the evaluation schedule.

While the evaluation team is free to determine the actual layout of the evaluation report, this must include the minimum content requirements mentioned earlier. The Team Leader will forward the final report by e-mail to UNDP MCO and the UNDP-GEF RTA in Bangkok for onward distribution to all stakeholders. In addition the Team Leader will forward a hard copy and electronic copy saved on disk to UNDP MCO. The evaluators will be responsible for the contents, quality and veracity of the report.

Deliverables

The evaluation mission will produce the following deliverables to UNDP/GEF:

(i) Draft copy of report;
(ii) Final copy of report;

The evaluation mission will produce the following deliverables to UNDP/GEF:

(iii) Draft copy of report ;
(iv) Final copy of report;

The final TE report will include: i) findings and conclusions in relation to the issues to be addressed identified under sections 2 and 3 of this TOR; ii) assessment of gaps and/or additional measures needed that might justify future GEF investment in the country, and iii) guidance for future investments (mechanisms, scale, themes, location, etc).

The report should also include the evaluators’ independent final rating on the following:

• Sustainability;
• Achievement of objectives/outcomes (the extent to which the project's environmental and development objectives and outcomes were achieved);
• Implementation Approach;
• Stakeholder Participation/Public Involvement; and
• Monitoring & Evaluation

The rating should be within a 6-point scale as follows: Highly Satisfactory (HS), Marginally Satisfactory (MS), Satisfactory (S), Marginally Unsatisfactory (MU), Unsatisfactory (U) and Highly Unsatisfactory (HU). The final report together with the annexes shall be written in English and shall be presented in electronic form in MS Word format as well as a hard copy.

The final report together with the annexes shall be written in English and shall be presented in electronic form in MS Word format as well as a hard copy

Products expected from evaluation

The main products expected from the evaluation are:

• presentation(s) to key stakeholders;
• an interim draft report;
• a final comprehensive evaluation report

Qualifications of team leader

• Substantive experience in reviewing and evaluating similar technical assistance projects, preferably those involving UNDP/GEF or other United Nations agencies, development agencies and major donors;
• International/regional consultant with academic and/or professional background in natural resource management or related fields with experience in land management, with in-depth understanding of land issues as well as community-based natural resource management. A minimum of 10 years work experience is required;
• Experience in leading multi-disciplinary and multi-national teams to deliver quality products in high stress an short deadline situations;
• Familiar with SLM approaches in Pacific and /or developing countries either through project management and/or implementation or through consultancies in evaluation of land related projects.
• Highly knowledgeable of participatory monitoring and evaluation processes;
• Excellent English writing and communication skills.