1. Introduction to Results and Monitoring

The EU is a major global development player implementing most of its external assistance spending through a large, decentralised network of country and regional EU Delegations. These play therefore a crucial role in translating the EU's broad international cooperation and development policy objectives into effective action and results in the field.

EU Delegations, as well as operational EC Headquarter Services, work within an overall framework to implement the EU’s international cooperation and development assistance.

It is based on the following principles underlying the monitoring, reporting and evaluation chain with respect to the implementation of EU-funded projects and programmes, requiring an organised flow of information that serves management, accountability and learning purposes:

i) Information on the performance of projects and programmes throughout their implementation and on their results at output and outcome levels through monitoring and reporting on implementation as well as more in-depth assessment of implementation issues through mid-term project evaluations where needed;

ii) In-depth assessment of results at both outcome and impact levels, and of sustainability of the projects and programmes and their value added, through individual final or ex-post project evaluations to take place at the end of or after project implementation;

iii) In-depth assessment of country, regional and thematic strategies, as well as of instruments through strategic evaluations.

Monitoring and reporting take place at different levels:

- By the Implementing Partners (IPs), through their own monitoring and reporting, that is the main source of information for the Commission’s own monitoring and reporting;

- By the Operational Managers (OMs) in EU Delegation and EC Headquarters operational services through monitoring and reporting at project level and;

- By more aggregated levels, including through the reporting, as from 2015, based on the new EU’s International Cooperation and Development Results Framework.

(ROM Handbook)

1.1 Access and Permissions

For Results and Monitoring, Operational Managers (OMs) assign Lead Implementing Partners (LIPs) to manage Interventions (please view What is an Intervention?). These Lead Implementing Partners can then access the Funding and Tenders Portal and grants access to the Implementing Partners (IPs) belonging to the same Consortium, so that they can manage the Logframe of the Intervention.

For detailed information on Access and Permissions for Implementing Partners, please consult the Cooperating with Implementing Partners or Experts page.

1.2 User Roles and Tasks

In OPSYS, every time the user is required to perform specific operations, a new task is generated. These tasks are organised into a procedure workflow. Each task within a workflow is also reserved and assigned to specific actors, or pool of users.

This rule is extended for external partners, where access to perform a specific task or operation is not only limited to a pool of users with a specific role, but also limited per entity (e.g. Intervention).

For example, only the Operational Manager, who is in charge of the Intervention, and the Implementing Partner(s) who has been granted access to the Intervention by the OM, may update the Logframe of that Intervention.

The following four main roles are available in OPSYS for Interventions, Monitoring and Results:

1.3 Key Elements in Results and Monitoring

1.3.1 Activity

An Activity is what the intervention does in order to produce results (i.e. the use of resources).

Activities are actions taken or work performed, through which inputs (such as funds, technical assistance and other types of resources) are mobilized to produce specific outputs.

1.3.2 Logical Framework Approach and RBM

The Logical Framework Approach is a methodology used in EU External Action for planning, managing and evaluating Programmes and Projects or Actions, involving stakeholder analysis, problem analysis, analysis of objectives, analysis of strategies, preparation of a Logical Framework Matrix (Logframe), and activity and resource schedules.

Results-Based Management (RBM) is a management strategy focusing on performance and achievement of outputs, outcomes and impacts.

1.3.3 Logical Framework Matrix - Logframe

A Logical Framework is a matrix in which the intervention logic (overall objective, purpose, expected results and activities – see section below), assumptions, objectively verifiable indicators and sources of verification are represented.

The Logical Framework Matrix (LFM) involves identifying strategic elements (inputs, outputs, outcomes, impact) and their relationships, indicators, and the assumptions or risks that may influence the success or failure of a Project. It thus facilitates planning, execution and evaluation of a development intervention, and is therefore present and used in different phases of the cycle of operations.

It is presented in the form of a table summarising the key elements of a Project/Programme, which is used as a management tool, to improve the design of Interventions, and for monitoring and reporting purposes during implementation.

The Logical Framework Matrix contains:

• The Project’s hierarchy of expected results, also called the results chain (outputs, outcomes and impacts);
• The key external factors critical to the project’s success (assumptions);
• How the Project’s achievements will be monitored and evaluated (i.e. for each expected result as part of the results chain, there should be at least one indicator, with a corresponding baseline, target and source of verification);
• The impact (or overall objective) relates to the wider objectives to which the Project will contribute;
• The outcome (or specific objective) is the sustainable flow of benefits to the Project’s target group.
• The outputs are the tangible goods and services delivered by the Project or Action.

1.3.4 Intervention Logic

The Intervention Logic is detailed and structured narrative that explains how, in the given context, the outputs will lead to the outcome(s), and the outcome(s) to the expected impact. It describes the “vertical logic” of the Logframe by articulating the first column (results chain) and the last column (assumptions) of the Logical Framework.

1.3.5 Result

According to the OECD DAC terminology, the term “Result” covers the outputs, outcomes and impact (result levels) of a development intervention. These three levels of results can be represented in a Results Chain that is reflected in the structure of the Logical Framework Matrix: Impact - Overall Objective, Outcomes - Specific Objectives, and Outputs.

1.3.6 Results Chain

An Intervention’s hierarchy of expected results (outputs, outcomes and impacts), expressed with each element or component contributing to the next, is represented as a Results Chain (i.e. a pathway of change).

The Results Chain is the causal sequence for an intervention that stipulates the necessary progression to achieve desired objectives – beginning with inputs, then moving through activities and outputs, and culminating in outcomes, impacts, and feedback. In some agencies, reach is part of the Results Chain.

Good practices in building a results chain are:

• Checking the hierarchy of expected results;
• No causal links in a single results statement.

Examples:

• "To strengthen the education system management in order to improve quality and equity [of the education system]"
• "To promote democracy through the support to electoral processes, citizen participation and transparency"

1.3.7 Impact

The Impact (or global objective) is a long-term effect to which the Intervention will contribute at country, region or sector level.

An impact can be a positive or negative, primary or secondary, long-term effect produced by a development intervention, whether direct or indirect, intended or unintended.

1.3.8 Outcome

An Outcome (or specific objective) is a medium-term direct effect in the behaviour of target groups, under control of the benefitting partner.

The outcome is the likely or achieved short-term and medium-term effects of an intervention’s outputs.

1.3.9 Output

Outputs (or products) are goods or services directly provided under control of the Intervention.

These outputs are products, capital goods and services that result from a development intervention, and may include changes resulting from the Intervention that are relevant to the achievement of outcomes.

1.3.10 Assumption

An Assumption is a hypothesis on factors or risks, which could affect the progress or success of a development intervention. Assumptions are key external factors that are critical to a Intervention’s success.

Good practices when defining the assumptions are:

• It is a condition required for the success of the Intervention;
• It should not be too critical;
• It should cover external factors not under the control of Project Management;
• It should be provided at output and outcome levels;
• It should be specific so that it can be monitored (avoid “lack of …”).

Risks or assumptions are two sides of the same coin - if you are not sure an assumption will hold true, then it is probably a risk for which mitigation measures should be provided.

A Risk Analysis or assessment may be performed to determine the factors (called assumptions in Logframes) that affect, or are likely to affect, the successful achievement of an Intervention’s objectives.

1.3.11 Indicator

An Indicator is a variable used to measure and assess performance (i.e. it allows for the verification of change and the measurement of results). Indicators form the basis of the Intervention’s monitoring and evaluation system. 

Indicators are quantitative or qualitative factors (or variables) that provide a simple and reliable mean of measuring achievement, to reflect the changes connected to an Intervention, or to help assess the performance of a development actor.

Indicators should always be provided with baseline (initial value) and target (intermediate or final value) values, as well as the possible source of verification for the actual values.

An indicator should:

• Be relevant and closely connected to the expected result (so that any change in the value can be associated/correlated to the Intervention);
• Be clear and specific – what do we measure? (e.g. “Number of…”, “Percentage of…”, “Status of…”);
• Be measurable – data is available or can be collected at reasonable cost; 
• Not include elements of the target (e.g. “increased number of…”);
• Be disaggregated by sex where applicable (or by age, urban/rural population, or by wealth quintile…).

1.3.12 Disaggregation

Disaggregation is the breakdown of observations (usually within a common branch of a hierarchy) to a more detailed level, so that more detailed observations can be taken. Indicator measurements can be disaggregated (where applicable) by sex, age, urban/rural population, wealth quintile, etc.

In the case of core indicators, Quality Managers may decide, for example, that it is good practice to disaggregate the total count of a group of people into sex categories of women and men, urban and rural.

Quality Managers have the possibility to make disaggregation mandatory or optional. Configuring this parameter in the core indicator ensures that the data collected for the formulation and/or implementation will be adapted and foreseen when considering the criterion.

1.3.13 Source of Verification

When indicators are being formulated, the source(s) of verification should also be specified.

The source(s) of verification should specify:

• How the information will be collected (e.g. from administrative records, special studies, sample surveys, observation, etc.), and/or the available documented source (e.g. progress reports, project accounts, official statistics, engineering completion certificates, etc.);
• Who will collect/provide the information (e.g. field workers, contracted survey teams, the district health office, the project management team);
• When or how regularly it will be provided (e.g. monthly, quarterly, annually, etc.).

Data sources for indicators can be primary or secondary:

• Primary data – is collected directly by the Intervention (usually by the Implementing Partner or Contractor), and may include administrative, budget, or personnel data; surveys; interviews; and direct observation. The feasibility, complexity and cost effectiveness of primary data collection should be carefully considered.
• Secondary data – that has already been collected outside the Intervention, and is readily available from other sources. Secondary data should be accessible at no or little cost to the general-public or the EU Delegation. Examples of secondary data include government reports, or existing statistics collected by international organisations. While secondary data can be more cost-efficient than primary data, their quality, availability, and reliability should be carefully considered before being used.

1.3.14 Value

The performance of the Result is measured with a set of collected values (or measurements) for each indicator identified at a specific point in time (date):

• Baseline – Generally in the past, and used as a reference point value at the beginning of the Intervention;
• Current Value – A progress measurement value to date;
• Target – Prediction of expected end or intermediate (for intermediate targets) value for future dates determined by an anniversary in the project, or at the end of the intervention;

The periodicity of the reporting on values to date (progress) is normally set contractually. However, the Operational Manager can also decide to modify the default settings in the system, and request a more frequent reporting from the Lead Implementing Partner.

For each value encoded, a Source of Verification (SoV) must be specified. Most indicators designed by the Quality Managers will have a suggested or mandatory SoV. If it is not the case, the Implementing Partner will need to specify where the data collected came from, and the date in which it was collected.

1.3.15 Baseline

The Baseline is the initial value of an indicator.

Good practices when defining the baseline are:

• To include the reference year, before the action starts or at inception;
• To use the same measurement unit as the indicator;
• If not indicated – to describe how/when/by whom it will be collected, and under which budget;
• If baseline study needed – to plan the final study/survey;
• If a new project – it could be zero (e.g. number of schools built, kilometres of roads rehabilitated);
• If relevant – it should be disaggregated by sex or other criteria.

1.3.16 Target

The Target is the expected end-value of an indicator.

Good practices when defining the target are:

• To use the same measurement unit as indicator;
• To use the same reference year - not after the end of the action (before for some outputs);
• If the baseline is available, the target must be set based on:

- Internationally agreed targets (Sustainable Development Goals), or commitments by the EU; or

- Government plans or strategies (if not final, it must be specified).

• Informed by the baseline, past trends, risks/assumptions, etc.;
• To be realistic – given resources available;
• To be achievable – within time period available;
• If relevant – it should be disaggregated by sex or other criteria.

1.4 List of Core Indicators for Design and Monitoring of EU-funded Interventions

Monitoring and Reporting Results in OPSYS

Since 2021, the European Commission is using OPSYS, an IT platform, to monitor EU-funded interventions outside the European Union and, simultaneously to report on results. The results campaign of 2022 is the first one carried out directly in OPSYS with the active participation of Operational Managers and Implementing Partners. The objective is to bring in more efficient, transparent and results-oriented planning, management, and follow-up of EU external actions.

The logical framework approach, taking the form of a Logical Framework Matrix (logframes), is the key internal monitoring tool used for results monitoring in OPSYS.

This logframe matrix presents the results that should be generated by an intervention at different time horizons. Result levels (outputs, outcomes and impact) are accompanied by indicators to measure (contribution to) their achievement. In OPSYS, Commission services and implementing partners access intervention logframes and use them as a basis for regular monitoring and reporting. To facilitate the design of interventions and enable reporting, a list of selected sector indicators is available at the right-hand top of the page.

Our Core Indicators for Design and Monitoring

Core indicators are aligned with DG INTPA, NEAR and FPI policy priorities. They appear in OPSYS as suggested indicators when users are designing Logical Framework Matrix (logframes) for their interventions. They can be searched and filtered while creating a logframe inside the system.

We provide these indicators to enhance the quality of EC's interventions in terms of design and monitoring. The use of core indicators is not mandatory but highly recommended. We also aim to promote the use of standardized information across European Union-funded interventions in partner countries. In other words, using them supports harmonisation between the EU and Implementing Partners.