External Evaluator

Period being evaluated: September 2017 – June 2018

Dates of the evaluation: July-August 2018

Project title: Justice & Livelihood Opportunities for all people: Enhancing access to protection and employment for the most vulnerable communities in Lebanon  

Project intervention zones: Central and North Bekaa, Akkar (Halba)

Project duration: 9.5 months

International Alert, through the funding of the Regional Development and Protection Program (RDPP) is working with two local partners and two local consultants since September 2017 in the Central and North Bekaa and Akkar’s Halba in the implementation of the ‘Justice & Livelihood Opportunities for all people’ pilot project.  The 9.5-months pilot project aims to improve access to protection and livelihoods for vulnerable Syrian refugees and Lebanese communities.  Access to protection to Syrian refugees has been challenged by the lack of legal residency, which has made most Syrian refugees vulnerable to arrests and unable to resort to official protection.  In addition, the lack of trust in the justice system makes refugees prone to exploitation by employers in the formal and/or informal sector.  Perceived competition over employment also remains a major source of tension between refugees and host communities. To address the identified problems, Alert – along with Arc en Ciel (AEC) and the Lebanese Observatory for Workers and Employees’ Rights (LOWER) – will (1) increase access to protection for vulnerable people through the engagement of trained community mediators and enhancing referral to legal aid, (2) increase opportunities for accessing the job market to vulnerable people by strengthening their employability skills, and (3) strengthen and advocate policies to improve protection and livelihoods of vulnerable people, specifically targeting businesses, important civil society stakeholders, and municipalities.

The external final evaluation of the project will help Alert and its partners in planning to develop a new proposal for a continuation of this work, and provide lessons learned for the current activities and partnerships, and guide Alert in developing projects that continue to focus on the integration of peacebuilding and conflict-sensitivity in livelihoods and protection. 

1. Objectives of the evaluation

The external evaluation aims to work on a theory-based evaluation as well as to derive lessons learned.  It aims at measuring the theory of change that 1) community mediation and social peace initiatives improve access to protection for vulnerable Lebanese and Syrian refugees (link between peace-building approaches and protection); 2) vocational training and employability trainings improve access to livelihoods; and 3) that improved access to protection and livelihoods in reducing tensions (link between protection/ livelihoods and social stability). Therefore, the evaluation aims to extract lessons learned from the project and will be used to set recommendations for future programming; thus, striving to have a learning focus.  The evaluation will be led by an external consultant, to ensure the impartiality of the process.

The objective of the evaluation is to assess and evaluate the success of the project in terms of:

• Lessons learned regarding the integration and adoption of peacebuilding and conflict-sensitive approaches in livelihood and protection programming.
• Alert partners’ view, understanding, and assessment of programming through a conflict-sensitive and peacebuilding lens. 
• The relevance and potential for success of a further project phase.

Suggested lines of inquiry include:

Evaluation Criterion

Key questions

Relevance

• Does the intervention appear to address relevant key causes and drivers of conflict and fragility in the areas of implementation: Central and North Bekaa and Halba?
• What is the relevance of the intervention as perceived by the local population, beneficiaries and Alert partners?
• Has the intervention responded flexibly to changing circumstances over time?

Effectiveness

• To what extent were the objectives achieved?
• What were the major factors influencing the achievement or non-achievement of the objectives?
• Would the same theory of change and intervention strategy be suitable for a future intervention?
• What are the, direct and indirect, positive and negative, intended and unintended, immediate and long-term results of the pilot project in terms of social stability in relation to livelihoods and protection?
• What changes can be ascertained in attitudes, behaviours, relationships or practice in target communities and in Alert’s partners?

Sustainability

• To what extent would the benefits of the project continue after donor funding ceases?
• What were the major factors which influenced/influence the achievement or non-achievement of sustainability of the access to livelihoods and protection?
• To what extent do Alert’s partners see and evaluate the added-value of the peacebuilding and conflict-sensitive perspective in the way they implement/facilitate programs?

Learning

• What were the major lessons learned from the project?
• What are the main learnings/contribution of livelihood programming to protection outcomes?
• What are the main learnings/lessons learned regarding Alert’s selection of partners for the implementation of project?
• What were the main lessons learned regarding the mixed approaches of livelihood and protection complementing one-another?
• What recommendations did the project result in that can help Alert in future proposal writing and program planning?

1. Evaluation Timing and Duration

The evaluation will take place in July-August 2018 [Note: may be August-September 2018 depending on Alert being granted a one-month no-cost extension]. It is expected to take 30 days.

The precise timing will be discussed and agreed with the consultant, but Alert envisages a calendar of work along the following lines:

• Preparation (reading, etc.): 3 days
• Initial planning and development of tools: 2 days
• Methodology and logistics planning with Alert staff and partners: 1 day
• Reflection session (with project team and partners): 1 day
• Piloting of tools: 1 day
• Data collection and analysis: 14 days
• Restitution workshop on results with staff, partners and stakeholder representatives: 2 days
• Report Writing: 5 days
• Finalisation of report after feedback: 1 day
• Dissemination of evaluation findings: Alert

Please note, Alert will review and respond to the first draft of the report within 10 working days. Based on the comments the Evaluator will amend the draft report and the final report should respond to all of this feedback.

1. Evaluation Methodology

This evaluation will adopt a qualitative and reflective approach, and utilise several data collection tools: context review, project document review, direct observation, interviews with key informants [partners, consultants, and beneficiaries], and focus group discussions.  Given the learning outcomes to be derived from this project, it is essential that the evaluation process engages Alert’s partners, the consultants, and the partners’ beneficiaries in the evaluation.

This evaluation will include the following steps:

• Preparation: Key project documentation will be provided to the consultant and will include the project proposal, logframe, donor reports, partner reports, activity reports, and monitoring data. Following this the consultant will draft a revised methodology plan, work plan and evaluation tools, which will be reviewed and finalised with the project team including methodology and planning meetings with Alert. 
• Fieldwork: to collect data, carry out interviews and focus group discussions. The fieldwork will be carried out in the partners’ areas of implementation (Taanayel in the Bekaa and Halba in Akkar).
  Alert and the partners will provide lists of the different stakeholders and beneficiaries of the project for the consultant to select from and will ensure that project staff assist and accompany the consultant throughout the evaluation.
  Interviews and/ or focus group discussions will be carried out with: project staff, other Alert staff in country, Alert Headquarters staff involved in the project, and project partners, beneficiaries and stakeholders. 
• Analysis of results: All collected data, obtained through the methodology, should form part of a comprehensive analysis that leads to lessons learned and list of recommendations.
• Restitution: At the end of the fieldwork and before submitting the draft report, the consultant will lead a restitution workshop with key project stakeholders to present their initial observations, conclusion and recommendations, with a dedicated session on ideas for developing the next phase of the project.
• A draft report responding fully to these terms of reference.
• A final report taking into account the observations and comments of Alert.
   

1. Deliverables

• Evaluation plan by 02 July 2018
• Tools for data collection by 12 July 2018
• Draft report by 27 July 2018
• Final report and summary doc by 10 August 2018
• Conducting learning and development sessions based on the evaluation findings to Alert and its partners
   
  All deliverables are to be presented in draft format and at a meeting for Alert and partner staff, before being finalised. All deliverables and data of the evaluation will be owned by Alert and may be used for both internal and external reporting and communications.

The main body of the report, in Word format, should be between 20 and 30 pages (excluding the appendices/annexes) and is to include the following elements:

• Contents table
• Executive Summary
• Introduction
• List of Acronyms
• Background (description of the project)
• Evaluation objectives and criteria
• Methodology and challenges
• Main results and analysis, clearly responding to the evaluation criteria and key questions in the TOR and referring to the project’s results and indicators
• Conclusions and recommendations, focusing on programme development for the next phase
  Annexes: Three case studies of what has worked well (written), evaluation Terms of Reference, meeting calendar, tools used in evaluation (question guides, questionnaires, observation grids), list of documents consulted.
   

1. Support and resources to be provided by Alert

Alert staff, the Senior Program Officer, and the Senior MEL Officer, will be available to provide the necessary support to the external evaluator.  In addition, the Alert Field Coordinators in the North and Bekaa will assist in coordinating meetings between consultant and beneficiaries and/or Alert partners in the field. 

The consultant is responsible for his/her own transportation and will need to have his/her own computer. 

Alert will also provide all key documents such necessary to perform the assignment, as detailed below, at least one-week prior to the start of the fieldwork, to be provided by the project officer.

1. Required skills and experience
    

• University degree in social sciences or relevant fields
• Experience of conducting external evaluations in Lebanon
• At least five years of experience with familiarity for standards of ethical conduct in research/evaluation settings 
• Has experience in evaluations/research relating to peacebuilding, protection, livelihoods, and/or economic stability
• Sound and proved experience in conducting evaluations and impact assessments, particularly utilization and learning focused evaluations.
• Expertise in qualitative and quantitative data collection techniques
• Expertise in conducting learning sessions and/or trainings
• Fluent in English and Arabic
• Strong inter-personal communication skills
• Excellent skills in research and data collection
• Ability to work both independently and as part of a team
• Demonstrated ability to handle sensitive information with discretion and professionalism
• Ability to work under pressure and meet deadlines
• Prior experience working on public consultation projects is a plus

Evaluation Criteria:

Technical Criteria (60%)

- Quality of proposal (level of effort/details put into proposal and level of understanding of assignment scope). (10%)

- Methodology (approach (mixed: qualitative and quantitative) and the extent to which the proposed sample covers all concerned stakeholders). (10%)

- Candidate profile (the strength and relevancy of the candidates’ academic background, technical and evaluation expertise). (15%)

- Delivery of Report within 30 days. (25%)

Financial Criteria (40%)

-Total Price: Bidder’s price is within budget allocated for evaluation. (40%)