Results & Monitoring
The EU is a major global development player implementing most of its external assistance spending through a large, decentralised network of country and regional EU Delegations. These play therefore a crucial role in translating the EU's broad international cooperation and development policy objectives into effective action and results in the field.
EU Delegations, as well as operational EC Headquarter Services, work within an overall framework to implement the EU’s international cooperation and development assistance.
It is based on the following principles underlying the monitoring, reporting and evaluation chain with respect to the implementation of EU-funded projects and programmes, requiring an organised flow of information that serves management, accountability and learning purposes:
i) Information on the performance of projects and programmes throughout their implementation and on their results at output and outcome levels through monitoring and reporting on implementation as well as more in-depth assessment of implementation issues through mid-term project evaluations where needed;
ii) In-depth assessment of results at both outcome and impact levels, and of sustainability of the projects and programmes and their value added, through individual final or ex-post project evaluations to take place at the end of or after project implementation;
iii) In-depth assessment of country, regional and thematic strategies, as well as of instruments through strategic evaluations.
Monitoring and reporting take place at different levels:
- By the Implementing Partners (IPs), through their own monitoring and reporting, that is the main source of information for the Commission’s own monitoring and reporting;
- By the Operational Managers (OMs) in EU Delegation and EC Headquarters operational services through monitoring and reporting at project level and;
- By more aggregated levels, including through the reporting, as from 2015, based on the new EU’s International Cooperation and Development Results Framework.
For Results and Monitoring, Operational Managers (OMs) assign Lead Implementing Partners (LIPs) to manage Interventions (please view What is an Intervention?). These Lead Implementing Partners can then access the Funding and Tenders Portal and grants access to the Implementing Partners (IPs) belonging to the same Consortium, so that they can manage the Logframe of the Intervention.
LIPs and IPs do not have access to Interventions and Logframes that have not been assigned to them.
For detailed information on Access and Permissions for Implementing Partners, please consult the Cooperating with Implementing Partners or Experts page.
In OPSYS, every time the user is required to perform specific operations, a new task is generated. These tasks are organised into a procedure workflow. Each task within a workflow is also reserved and assigned to specific actors, or pool of users.
This rule is extended for external partners, where access to perform a specific task or operation is not only limited to a pool of users with a specific role, but also limited per entity (e.g. Intervention).
For example, only the Operational Manager, who is in charge of the Intervention, and the Implementing Partner(s) who has been granted access to the Intervention by the OM, may update the Logframe of that Intervention.
The following four main roles are available in OPSYS for Interventions, Monitoring and Results:
Operational Manager (OM) / Operational Initiating Agent (OIA)
The Operational Manager, inside an EU Delegation or a HQ unit, is responsible for the operational follow-up of a project. They manage the operational part of the contractual relationship with the Lead Implementing Partner. They also have the responsibility to design and draft Logframes for Actions and may be responsible for the annual activity reporting, either through the Annual Activity Reports, or the External Assistance Management Report (EAMR) (depending on the DG they work for).
The Operational Managers can create, read, update and delete Interventions; and read, create, update, review, approve and delete Logframes for the Interventions they manage. The Operational Manager is responsible for adding and removing Lead Implementing Partners on the Interventions managed.
Lead implementing partner/Contractor (LIP) and Implementing Partner/Expert (IP)
The Lead Implementing Partner is the Contractor, or the signatory to the contract with the European Commission. Lead Implementing Partners/Contractors are contractually in charge of implementing activities for EC-funded projects. They have a contractual reporting obligation both on progress made towards the targets and on the financial execution. The Lead Implementing Partner/Contractor can grant or revoke access to the Implementing Partners/Experts, access the Intervention and create, update and submit the Logframe (including results, indicators and current values).
Implementing Partners/Experts (non-Lead) can be either Consortium partners or external Experts. They are contributing to the implementation of activities and reporting tasks. The Implementing Partner/Expert can access the intervention (upon receiving the access granted by the LIP), and create and update the Logframe (including results, indicators and current values), but cannot submit the Logframe or indicator values for revision. Only the Lead Implementing Partner/Contractor can submit the Logframe or indicator values for approval to the Operational Manager.
Quality Manager (QM)
Quality Managers are responsible for results methodology, guidance, support and quality control in a specific domain. They are in charge of bringing methodological support to Operational Managers.
The Quality Manager manages the library of core indicators which are proposed to Operational Managers, LIPs and IPs, when drafting Actions and Contracts, and provides support to the Operational Managers in drafting the Logframe and on the use of indicators. Quality Managers review the values encoded for progress of indicators and proceed to the reporting, including specific reporting obligation like the Annual Activity Reporting.
A Quality Manager can delegate to another Quality Manager the quality control/validation of any element that they are in charge of.
European Commission Internal Users
The staffs of DG INTPA, DG NEAR, FPI Services, and EU Delegations users with EU Login have read-only access to all the Portfolios and approved Logframes.
An Activity is what the intervention does in order to produce results (i.e. the use of resources).
Activities are actions taken or work performed, through which inputs (such as funds, technical assistance and other types of resources) are mobilized to produce specific outputs.
The Logical Framework Approach is a methodology used in EU External Action for planning, managing and evaluating Programmes and Projects or Actions, involving stakeholder analysis, problem analysis, analysis of objectives, analysis of strategies, preparation of a Logical Framework Matrix (Logframe), and activity and resource schedules.
Results-Based Management (RBM) is a management strategy focusing on performance and achievement of outputs, outcomes and impacts.
A Logical Framework is a matrix in which the intervention logic (overall objective, purpose, expected results and activities – see section below), assumptions, objectively verifiable indicators and sources of verification are represented.
The Logical Framework Matrix (LFM) involves identifying strategic elements (inputs, outputs, outcomes, impact) and their relationships, indicators, and the assumptions or risks that may influence the success or failure of a Project. It thus facilitates planning, execution and evaluation of a development intervention, and is therefore present and used in different phases of the cycle of operations.
It is presented in the form of a table summarising the key elements of a Project/Programme, which is used as a management tool, to improve the design of Interventions, and for monitoring and reporting purposes during implementation.
The Logical Framework Matrix contains:
|In OPSYS, we identify the Logical Framework Matrix (LFM) as a Logframe, which is attached to an Intervention.|
The Intervention Logic is detailed and structured narrative that explains how, in the given context, the outputs will lead to the outcome(s), and the outcome(s) to the expected impact. It describes the “vertical logic” of the Logframe by articulating the first column (results chain) and the last column (assumptions) of the Logical Framework.
According to the OECD DAC terminology, the term “Result” covers the outputs, outcomes and impact (result levels) of a development intervention. These three levels of results can be represented in a Results Chain that is reflected in the structure of the Logical Framework Matrix: Impact - Overall Objective, Outcomes - Specific Objectives, and Outputs.
An Intervention’s hierarchy of expected results (outputs, outcomes and impacts), expressed with each element or component contributing to the next, is represented as a Results Chain (i.e. a pathway of change).
The Results Chain is the causal sequence for an intervention that stipulates the necessary progression to achieve desired objectives – beginning with inputs, then moving through activities and outputs, and culminating in outcomes, impacts, and feedback. In some agencies, reach is part of the Results Chain.
Good practices in building a results chain are:
The Impact (or global objective) is a long-term effect to which the Intervention will contribute at country, region or sector level.
An impact can be a positive or negative, primary or secondary, long-term effect produced by a development intervention, whether direct or indirect, intended or unintended.
An Outcome (or specific objective) is a medium-term direct effect in the behaviour of target groups, under control of the benefitting partner.
The outcome is the likely or achieved short-term and medium-term effects of an intervention’s outputs.
Outputs (or products) are goods or services directly provided under control of the Intervention.
These outputs are products, capital goods and services that result from a development intervention, and may include changes resulting from the Intervention that are relevant to the achievement of outcomes.
An Assumption is a hypothesis on factors or risks, which could affect the progress or success of a development intervention. Assumptions are key external factors that are critical to a Intervention’s success.
Good practices when defining the assumptions are:
Risks or assumptions are two sides of the same coin - if you are not sure an assumption will hold true, then it is probably a risk for which mitigation measures should be provided.
A Risk Analysis or assessment may be performed to determine the factors (called assumptions in Logframes) that affect, or are likely to affect, the successful achievement of an Intervention’s objectives.
An Indicator is a variable used to measure and assess performance (i.e. it allows for the verification of change and the measurement of results). Indicators form the basis of the Intervention’s monitoring and evaluation system.
Indicators are quantitative or qualitative factors (or variables) that provide a simple and reliable mean of measuring achievement, to reflect the changes connected to an Intervention, or to help assess the performance of a development actor.
Indicators should always be provided with baseline (initial value) and target (intermediate or final value) values, as well as the possible source of verification for the actual values.
An indicator should:
Disaggregation is the breakdown of observations (usually within a common branch of a hierarchy) to a more detailed level, so that more detailed observations can be taken. Indicator measurements can be disaggregated (where applicable) by sex, age, urban/rural population, wealth quintile, etc.
In the case of core indicators, Quality Managers may decide, for example, that it is good practice to disaggregate the total count of a group of people into sex categories of women and men, urban and rural.
Quality Managers have the possibility to make disaggregation mandatory or optional. Configuring this parameter in the core indicator ensures that the data collected for the formulation and/or implementation will be adapted and foreseen when considering the criterion.
When indicators are being formulated, the source(s) of verification should also be specified.
The source(s) of verification should specify:
Data sources for indicators can be primary or secondary:
The performance of the Result is measured with a set of collected values (or measurements) for each indicator identified at a specific point in time (date):
The periodicity of the reporting on values to date (progress) is normally set contractually. However, the Operational Manager can also decide to modify the default settings in the system, and request a more frequent reporting from the Lead Implementing Partner.
For each value encoded, a Source of Verification (SoV) must be specified. Most indicators designed by the Quality Managers will have a suggested or mandatory SoV. If it is not the case, the Implementing Partner will need to specify where the data collected came from, and the date in which it was collected.
The Baseline is the initial value of an indicator.
Good practices when defining the baseline are:
The Target is the expected end-value of an indicator.
Good practices when defining the target are:
- Internationally agreed targets (Sustainable Development Goals), or commitments by the EU; or
- Government plans or strategies (if not final, it must be specified).
Monitoring and Reporting Results in OPSYS
Since 2021, the European Commission is using OPSYS, an IT platform, to monitor EU-funded Interventions outside the European Union and, simultaneously to report on results. The results campaign of 2022 is the first one carried out directly in OPSYS, with the active participation of Operational Managers and Implementing Partners. The objective is to bring in more efficient, transparent and results-oriented planning, management, and follow-up of EU external actions.
The logical framework approach, taking the form of a Logical Framework Matrix (Logframes), is the key internal monitoring tool used for results monitoring in OPSYS.
This logframe matrix presents the results that should be generated by an intervention at different time horizons. Result levels (outputs, outcomes and impact) are accompanied by indicators to measure (contribution to) their achievement. In OPSYS, Commission services and implementing partners access intervention logframes and use them as a basis for regular monitoring and reporting. To facilitate the design of interventions and enable reporting, a list of selected sector indicators is available.
Our Core Indicators for Design and Monitoring
Core Indicators are aligned with DG INTPA, NEAR and FPI policy priorities. They appear in OPSYS as suggested indicators when users are designing Logical Framework Matrix (logframes) for their interventions. They can be searched and filtered while creating a logframe inside the system.
We provide these indicators to enhance the quality of EC's interventions in terms of design and monitoring. The use of core indicators is not mandatory but highly recommended. We also aim to promote the use of standardized information across European Union-funded interventions in partner countries. In other words, using them supports harmonisation between the EU and Implementing Partners.