Page History

Evaluation is commonly understood as the ‘systematic and objective assessment of a planned, ongoing or completed intervention, its design, implementation and results’ (OECD DAC, 2023). Within the EC, evaluation is used to assess the performance of a strategy, policy, instrument, modality, intervention or group of interventions.

Evaluation is integral to the intervention cycle and its management, providing key inputs and information at all phases of the cycle. By providing evidence of what works and what does not (and under what circumstances), evaluation helps improve engagement with partners; enhances the impact of EU development cooperation; and is critical to better programming of subsequent interventions, political dialogue and visibility of results. The benefits of the knowledge generated through evaluation extend beyond the commissioning unit or delegation to be a source of institutional learning and living memory now and in the future. A good evaluation supports better decision-making for better outcomes.

Evaluation types and timing

Evaluations can be done for different purposes and at different times to assess what works/worked and why.

• Intervention-level evaluations– previously known as project and/or programme evaluations – analyse the results of a specific intervention, or a group of logically interlinked interventions, within the frame of a wider scope of collaboration in a country or region. These evaluations are an integral part of intervention cycle management as they help enhance the programming, design, implementation, performance and achievement of results of EU interventions.
  • Intervention-level evaluations can be conducted before an intervention starts (ex ante evaluation), and/or at the midpoint (midterm evaluation) or conclusion (final evaluation) of the intervention (i.e. six months before/after the intervention’s completion date), or following implementation (ex post evaluation) (i.e. at least one year after the intervention’s completion date).
• Strategic evaluations look at the combination of the EU’s external spending and non-spending actions to review EU strategies, policies, instruments or modalities, generally over a significant period of time. A strategic evaluation has a wider scope of analysis than an intervention-level evaluation and looks deeper into the strategic dimensions of thematic areas, instruments or overall EU cooperation and partnerships in a defined geographic area. More precisely, strategic evaluations analyse EU strategies from conception to implementation at any or all of several levels – country, region, sector or financing instrument – over an extended period of time (often 7–10 years). DG INTPA strategic evaluations conducted since 2014 are available online on the DG INTPA Strategic Evaluation Reports web page.
  • Since these evaluations combine interventions and/or strategies at different degrees of implementation, they can be considered midterm or, when the work in a given sector/theme is in the process of phasing out, final.

Responsibility for EC-managed evaluations lies with the EU delegations/regional teams or headquarters units; they plan, launch, contract and manage these evaluations. Evaluations of interventions are generally the responsibility of the delegation/unit in charge of the intervention; for strategic evaluations, the responsibility generally lies with DG INTPA.D.4/FPI.4. Both types of evaluations are contracted for in OPSYS by launching a request for services under a framework contract.

Note:  In DG INTPA, evaluation of budget support can be either strategic or at the intervention level. The former are launched and managed by DG INTPA.D.4 while the latter are launched and managed by the delegations/units.

For more information see sections 1.2 and 1.3 of the Evaluation Handbook.

Evaluation Planning

An evaluation can be planned and carried out at any time of the year in line with the needs of the delegations and headquarters units in the case of intervention-level evaluations, and on the basis of the multi-annual rolling Evaluation Work Programme for strategic evaluations. 

For intervention level evaluations, EU delegations and headquarters units plan and/or revise their projected evaluations according to their priorities, needs and resource availability. The decision on what to evaluate is completely at the discretion of each delegation/unit, though by regulation, EU delegations and headquarters units are mandated to evaluate those interventions that entail significant spending. A best practice is to select a well-thought-out sample of interventions for evaluation. For more information see section 1.4 of the Evaluation Handbook. 

For strategic evaluations, DG INTPA.D.4 – in consultation with all relevant headquarters units, delegations and the European External Action Service – prepares a multi-annual Evaluation Work Programme (EWP), which is approved by DG INTPA management. The EWP establishes the list of countries, regions, instruments, themes, modalities and policies that will be subject to strategic evaluation in the upcoming period.

Contracting

• Both strategic and intervention level evaluations are undertaken by independent evaluators which are primarily contracted

The EU Better Regulation Guidelines define the overall methodological approach to evaluation across all European Commission services, but the evaluation of EU external actions also makes extensive use of the work of the OECD Development Assistance Committee. Dedicated methodologies are developed on this basis, to ascertain whether the objectives specific to EU external actions have been met and to formulate recommendations to improve future partnerships worldwide.  Evaluations occupy the two upper segments of the monitoring and evaluation pyramid; the lower levels feed into the process of the upper levels by providing lessons based on findings.

Types of evaluations

 DG INTPA manages two categories of evaluations: strategic evaluations at corporate level, centrally managed by INTPA D4 and intervention evaluations at the level of specific projects and programmes.

Strategic evaluations

Strategic evaluations assess EU strategies, policies and actions, from design to implementation. They usually cover all activities (spending and non-spending, i.e. policy dialogue and cooperation) across several years. In doing so, they involve several European services and actors, and assess the achievement of EU priorities for external actions and progress towards the UN Sustainable Development Goals. Strategic evaluations can be of several types depending on their focus: country, regional, global, sector/policy, aid modalities (e.g. budget support) and financial instruments.They benefit from a longer time span and portfolio size, which enable evaluators to draw lessons learned and recommendations at the portfolio or country/regional level.

Intervention-level evaluations

Intervention-level evaluations (also known as Project and programme evaluations) assess relevance, performance and its determining factors in view of sustainability of results, EU added value and coherence, all the while gathering lessons from each intervention. These are managed by EU Delegations or the operational units in charge of the managing the interventions.

Intervention-level evaluations require data collected through internal and external monitoring systems, which are then triangulated, Primary data are collected for these by the evaluation team through a wide range of methods and tools including key informant interviews, focus group discussions and surveys. These evaluations are conducted mid-way through implementation and/or at the end of an intervention and are an integral part of the intervention cycle.

Operational managers do not rely exclusively on evaluations for monitoring throughout implementation. They are responsible for continuous monitoring of interventions and evaluations are not continuous processes, as they take place at specific points in time and have their own objectives. Financial, administrative and operational monitoring data contribute to informing and complete the evaluators' assessment of the intervention's performance. Evaluators verify monitoring systems data and analyse why and how the reported changes happened.

Planning

• For strategic evaluations, an evaluation work programme (EWP) is available for strategic evaluations foreseen in the upcoming 5 years. It is updated on a yearly basis.
• Yearly operational evaluation plans (OEP) are also available for internal staff, for projects and programmes (intervention-level) evaluations and are also regularly updated. 

Contracting

• Intervention level  evaluations are undertaken by independent evaluators which are primarily contracted via a Framework Contract. The Framework Contract SEA 2023  (Lot 16) is valid from which entered into force on 13/02/2024 . For INTPA Staff,  you can access a guidance document for Lot 16 includes two lots specifically dedicated to evaluations, these are: 
  -    Lot 16 for Intervention level evaluations, including evaluations of budget support programmes launched by EU delegations. Dedicated ToR guidance is provided under point 2.3 .Any information on evaluations can be requested at INTPA D4 : INTPA-Evaluations@ec.europa.euof the SEA 2023 FWC IntraComm pageImage Added
  -    Lot 17 for Strategic evaluations.

Methodological Support

• For INTPA staff, for any methodological support on the preparation of your evaluationthroughout the evaluation cycle, from planning and designing, to conducting your evaluations, including its dissemination and follow-up, please contact our helpdesk for advice: INTPA-EVALUATION-SUPPORT@ec.europa.eu(for internal staff)Image Added
• For DG INTPA staff on the Knowledge extracted from Monitoring and Evaluations, please contact INTPA D4: INTPA-ICM-GUIDE@ec.europa.euImage Added

Methdological fiche(s):

...