Other Wikis
- EC Internal Users
- EC Internal Users
Page History
| Excerpt | |
|---|---|
|
| Div | ||
|---|---|---|
| ||
1. Introduction to Results and Monitoring
| Div | ||
|---|---|---|
| ||
Please note that this section on our EXACT - External Action Wiki is dedicated to the management of Results and Monitoring users from outside of the European Commission. For all other roles, please view on our internal RELEX Wiki (only accessible by EC internal users). |
...
The European Union (EU) implements most of its external assistance through a large network of Delegations, which translate broad cooperation and development policy objectives into effective actions. EU Delegations and Headquarter Services work under a monitoring, reporting, and evaluation framework to implement international cooperation and development assistance. The framework assesses project performance, results, and implementation issues, evaluates results and sustainability, and in-depth assesses strategies and instruments. Monitoring and reporting occur at the intervention level and is aggregated by the EU for their own legal reporting.
Reporting on Results in OPSYS
In recent years, the European Commission has widely used logical framework matrices (logframes)[1] for signed contracts. However, there has been a significant update since late 2021. Instead of just reporting logframe indicator achievements in narrative reports, implementation partners now need to report through a new IT platform, OPSYS.
OPSYS, a project management tool owned and developed by the European Commission (EC), enables logframe encoding and reporting against indicators (amongst other aspects that are purely internal to the EC).
1.1 Interventions
In OPSYS, logframes are encoded under a new item called "intervention". The Operational Managers create interventions for the relevant contracts, actions, or budgetary guarantee agreements, allowing for adequate reporting on logframe[2] level.
...
2. Accessing OPSYS logframes
Implementing partners access OPSYS through the Funding & Tender Portal (F&TP). Only the appointed Lead Implementing partner (LIP) for each intervention can submit the Logframe and its updates to the EC via the system.
After signing an agreement or a contract, the LIP cannot access the dedicated space to create and regularly update the Logframe, until the corresponding Operational Manager of the EC assigns this role in the system.
2.1 Level of Permissions
In OPSYS and the F&TP, the “Lead Implementing Partner" is responsible for reporting. The LIP is an actual person with a valid email address and EU Login, appointed by the contractual partner that signed the agreement with the European Commission.
The assigned LIP may grant access to their team members or other partners involved in the agreement. However, these contacts are referred to as Implementing Partners (IPs) in the IT system. A LIP cannot grant access to another LIP.
Lead Implementing Partner
The LIP has to be invited by the Operational Manager. They can enter the logframe framework (results and indicators) as well as current values. They submit the logframe and current values to the Operational Manager.
Additionally, LIPs can use the matching functionality in the logframes.
Implementing Partner
The so-called IPs in the system, can either be invited by a LIP or by the Operational Manager from the EC side. IPs have the same access rights as a LIP and can enter the logframe framework (result and indicators) as well as the current values. However, IPs cannot submit a logframe or the current values to the Operational Manager, and they cannot use the matching functionality.
There is no limitation to how many Lead Implementing Partners and Implementing Partners can be invited to a single intervention, but each Lead Implementing Partner can only see who they invited. Only the Operational Manager on the EU-side has a full overview. There is no “view only” role, and everyone with access rights can make changes to the logframe. |
2.2 Accessing the EU Funding and Tender Portal
Before attempting to access the EU F&TP, users need to ensure they have a valid EU Login. Additionally, to be able to access as Implementing Partner (IP or LIP), users need to first be invited as described in the 2.1 Level of Permissions section, LIP and IPs will receive a notification via the email associated to them in the system after they are invited.
...
The displayed options in the menu will depend on the permissions linked to the user’s account (see below).
2.3 Inviting an Implementing Partner
The Lead Implementing Partner (once their access has been granted by the Operational Manager) must first access the Intervention on the Funding and Tenders Portal.
...
The person assigned as Implementing Partner can now access the Intervention from "My Intervention(s)" and can enter or update the Logframe by adding or updating indicators and values.
2.4 Removing an Implementing Partner
The Lead Implementing Partner can use the same procedure as indicated in the previous section “How to add an Implementing Partner?” above to access the Implementing partners management pop-up window in the Intervention page.
...
And them click on the "Confirm" button to confirm. The IP will no longer be assigned to the Intervention and lose access immediately.
...
3. Managing logframes in OPSYS
Logframes in OPSYS must correspond with the contractually agreed logframes and serve as a means to report data collected according to the indicators of the logframes.
3.1 Accessing a logframe
To access a Logframe, the IPs will first login to the Funding & Tenders Portal, and then select My Intervention(s) from the left main menu (1).
...
- The task page accessible in My Task(s), with the relevant task for the intervention found in the list of tasks.
...
3.2 Creating a logframe
In OPSYS, logframe encoding occurs in a two-step process:
...
Generally, it is the IPs or the LIP who initiate the encoding of both the logframe and the current values. At the end of each step, only the LIP can submit the items encoded for approval of the Operational Manager.
3.2.1 Starting from scratch (NEW Logframe)
When there is no (even partial) logframe entered, then clicking the button “Create Logframe” in the intervention leads the IPs directly to the “MANAGEMENT MODE” of the logframe.
...
When the draft Logframe has been saved, the IPs can still edit the Logframe before submitting it for review to the Operational Manager via the “Manage Logframe” button on the Logframe page. But saving will close the “Management Mode”. To further edit the logframe, “Manage Logframe” in the top right must be clicked again.
...
3.3 Adding results to a logframe
To add results to a logframe, the encoder must first ensure she/he is in "MANAGEMENT MODE". If not, the encoder should navigate to this mode by clicking:
...
If the result assumptions are not yet defined, then the “To be defined” tick-box should be selected by clicking on it.
- (*) Required fields
Please note that result assumptions are not possible if the result level is set to an “overall objective – impact”.
...
The encoder should make sure to select the correct Result (if multiple) before deleting as, once deleted, the Result and Indicators (if any) can no longer be recovered. |
...
3.4 Adding an indicator to a result
In OPSYS, there are two types of indicators: predefined and customised. Predefined indicators are encoded by the EC and can be used by any user. Customised indicators are created from scratch by the user.
...
Lastly, thematic indicators are proposed such as the SDG indicators or those for specific thematic areas.
3.4.1 Searching and selecting a predefined indicator
For the relevant Result, IPs should select the “Add Indicator” button to add an indicator.
...
When the indicator being encoded is a customised indicator, the IPs can proceed to create it in OPSYS because customised indicators are not pre-encoded in the system. This is only possible for DG INTPA and NEAR, following the steps detailed in the next section.
3.4.2 Encoding a customised indicator[3]
Customised indicators are created only for interventions funded by DGs INTPA and NEAR. At design phase of the intervention, if there are no pertinent predefined indicators to measure a result in the suggested list of indicators, then a customised indicator will be formulated that will have to be created in OPSYS when encoding the logframe.
...
- Indicator name (*) – This is the actual text of the indicator.
- Indicator description – Further information that supports the understanding of the indicator.
- Indicator type (*) – “Quantitative” or “Qualitative”.
- Value type – Required if indicator type is “Quantitative”, and can be “Percentage” or “Numeric”.
- Qualitative type – If indicator type is “Qualitative”, as selected option from drop-down list.
- Unit of measure – If indicator type is “Quantitative” and value type is “Numeric”, then an option should be selected from the drop-down list (e.g. number of, kilometres, jobs, megawatts, etc.); The “number of” unit of measure has two sub-types, persons and other. Persons should be selected whenever people are counted, and then sex disaggregation becomes mandatory.
- Gender sensitive – The user must determine its sensitivity based on the specific context and application.
- Disaggregation type (e.g. sex, wealth quintile, geo-location, etc.).
- Source of data (SoD) type and sub-type describes the source of the data for reporting purposes.
- Values (Baseline and Target) – For each value, the Date (*), Total (*), Disaggregation (if relevant) and Comments (optional) are entered. Additional intermediate targets will be added by the system and can be adjusted (see section 3.5).
- (*) Required fields
Once IPs have completed the required fields and have verified the encoded data, they should click on the “Confirm” button to confirm the creation of the customised indicator and its values and the indicator it is automatically added to the Result under which it was created.
To prevent data loss, it's advisable to frequently save the logframe as a draft while encoding, especially with an unstable internet connection. When the "Save as Draft" button is pressed, the logframe is stored, mitigating the risk of data loss. To resume editing, access the logframe again through "Manage Logframe" |
3.4.3 Editing an indicator
To edit/modify an Indicator the encoder must first access and enter the “MANAGEMENT MODE” of the Logframe – as described before.
...
The indicator information (attributes) is edit as required.
- (*) Required fields
Please note that once the Logframe is approved, IPs may only edit the dates and values (baseline, intermediate and final target), and add a comment to an indicator. Further disaggregation can be added, but not removed. Neither can the text be changed or the units of measure and type of indicator.
...
When modifying an approved logframe, its status reverts to "draft" mode. Even if the "Save as Draft" button is pressed without any changes, the logframe will still enter draft mode. In draft mode, current values cannot be accessed or managed, so the logframe must be resubmitted for approval and approved by the OMs before any current value can be encoded or edited. |
3.4.4 Removing an Indicator
To delete an Indicator the IPs must first access and enter the “MANAGEMENT MODE” of the Logframe.
...
It is recommended to save the Logframe as draft after any modification to Result(s) and Indicator(s).
...
3.5 Intermediate targets
A logframe provides data on target achievement that is used by the EC internally for performance measurement. To ensure meaningful performance scores, intermediate targets (milestones) are necessary. Performance at indicator level can be assessed by comparing the current value (what we have) with the intermediate target (what we should have) for the same date.
Intermediate targets can be tedious to define and to encode. Therefore, intermediate targets are automatically generated for outputs and outcomes for all June 30 and December 31 dates between the baseline and final target dates.
3.5.1 Output indicators
For outputs, the intermediate targets will be generated by default using a linear function between baseline-date and value, and final target-date and value.
...
The graphs can be found, when clicking on “Access Indicator’s Page” in the main logframe view.
3.5.2 Outcome indicator profiles
For outcomes, which usually progress non-linearly, three non-linear profiles are suggested for automatic intermediate target generation. The outcome profile must be specified initially but can be changed later. Additionally, the automatically generated intermediate targets can be revised or removed as required.
...
a. Steady progress: The outcomes are achieved continuously throughout the second half of implementation.
This profile assumes a linear progress in the second half of implementation, whereas the target values are equal to the baseline value (here 0) in the first half of implementation.
...
Please note that these automatically generated intermediate targets can always be modified later if needed, either collectively by changing the outcome profile selection or individually by changing specific intermediate targets. |
3.5.2 Selecting an Outcome indicator Profile
The profile is now visible at the top view of the logframe.
...
Once the logframe is submitted and approved, the intermediate targets are visible in the page table.
3.5.3 Editing intermediate targets
To edit the intermediate targets, before submitting the logframe or at a later date, the Management Mode has to be opened again via “Manage Logframe”.
...
Any change to the intermediate target has to be submitted to and approved by the Operational Manager.
...
3.6 Submitting a logframe
Logframes follow a review mechanism for validation purposes, ensuring a consistent exchange, follow-up, and data-quality assurance between the Operational Managers and Implementing Partners.
...
When the logframe is submitted after revision, the logframe is resent to the Operational Manager for review.
...
3.7 Current Values
The entered current values should be substantiated by and confirmed in the narrative reports.
3.7.1 Adding Current Values
To add a current value, the logframe needs to be in “Approved” status.
...
The pop-up has various fields with some of them being mandatory (*):
- Current value date*: This is the date of the value, not the date of the encoding. It should be the same as the date of the narrative report that clarifies this value. The date has to be later than the baseline date, and before or on the final target date.
- Current value*: This is the actual current value, that needs to be reported cumulatively where applicable.
- N/A: Not applicable should be selected if no data could be collected against an indicator. It is not the same as reporting “0”, which is a value. If N/A is selected, a justification has to be provided why no data is collected.
- Disaggregation: Any disaggregation that was activated (for predefined indicators) or selected (for customised) is visible here. Disaggregation options should be filled whenever they have been selected and are required for sex disaggregation of people.
- Source(s) of data*: The options available/selected are shown here. Though, others can be added, or the existing ones can be removed, if there is change. Ideally, the location is filled with a page number of the document that is referenced for the value.
- Comment: Each current value has a comment field where further explanation can be provided, for example local geographic disaggregation options that are not available in OPSYS.
- (*) Required fields
Once all mandatory fields are filled, the user can click on “Save” in the bottom right corner of the pop-up and the value is added in “Draft” to the respective indicator.
Values need to be reported cumulatively. It means that when the logic of a quantitative numerical indicator allows for aggregation (most “number of” indicators), the baseline AND all achievement to date needs to be combined in one figure. Example: In Year 1, the current value is 150 (baseline + Year1). In Year 2, the current value is 225 (baseline + Year 1 + Year 2. |
3.7.2 Editing Current Values
Any current value can be edited at any moment, but the value will move to the status “Draft” again.
...
This "
" is the symbol to delete/remove a current value.
3.7.3 Removing Current Values
Any current value can be removed at any moment. This action cannot be undone.
...
In the table, the "
" button is clicked for the current value in question. Once confirmed the value is removed and cannot be restored.
3.7.4 Submitting Current Values
Once all indicator values are updated according to the latest narrative report and the intervention’s M&E system, the LIP needs to submit the current values to the Operational Manager for review.
...
The values have to be sent back for approval, once all required changes are made.
...
3.8 Matching Indicators
Relevant corporate indicators from the respective results frameworks (such as Global Europe Results Framework, Instrument for Pre-Accession III Results Framework, FPI Results Framework) should ideally be identified during the design phase of new interventions and then incorporated into the logframe and encoded directly (please note that corporate indicators are a subset of the predefined indicators).
...
Matching is employed to enable reporting against corporate indicators, even if these are not part of the logframe. Ideally, the existing logframe indicator should be "matched" to an appropriate corporate indicator , which generally measures a similar aspect. Units of measure may differ, but the matched indicator will be utilised to calculate the total value of the corporate indicator.
3.8.1 Adding Matching Indicator
To add a matching indicator, the logframe needs to be in “Approved” status. Only LIPs and OMs can use this functionality in OPSYS.
...
Matching indicators are not part of the approval process; thus, the LIP can add them without having to send the logframe for approval. However, their current values have to be approved, as any other current value in the logframe. |
3.8.2 Editing Matching Indicator
Matching indicators cannot be edited, as they are selected from the list of predefined indicators in OPSYS, and no baseline and target values are to be encoded.
3.8.3 Removing Matching Indicator
If there is no current value (including N/A) encoded, (1) the LIP can directly remove a match by clicking on “Delete Matching Indicator” (2) in the main logframe view.
3.8.4 Adding Current Values to Matching Indicators
Current values of matching indicators can be added by either clicking “Add Value” directly on the main logframe view, or when the user has entered an indicator’s page.
...
To submit the value, please see section 3.7.4.
3.8.5 Editing Current Value of Matching Indicator
Any current value can be edited at any moment, but the value will move to the status “Draft” again.
...
This "
" is the symbol to delete/remove a current value.
3.8.6 Removing Current Value from Matching Indicator
Any matched current value can be removed at any moment. This action cannot be undone.
...
In the table, the "
" bin button is clicked for the current value in question. Once confirmed, the value is removed and cannot be restored.
...
3.9 Error messages
There are two options for error messages when creating or updating logframes in OPSYS. One is the quality check, and the other are validation errors.
3.9.1 Quality Check errors
An automated basic Quality Check by the system is applied to the logframe and its components (results and indicators) as soon as the logframe is created. The Quality Check runs continuously as modifications or updates are applied.
...
The quality check does not check the quality of entered results or indicators. |
3.9.2 Validation errors summary
In both the main and management views, a summary of validation errors is displayed, showing any issues found in the logframe. The error summary uses a multi-layered, multi-line visualisation, with each layer/line providing additional information about the location and type of the error. Each layer can be opened to get further information and access the next layer.
First layer
The first line only provides information on the intervention the error is located. As the visualisation is within the logframe, there is no additional information in this line. Click the little arrow at the right-hand side of the line to expand the next line.
Second layer
The next layers refer to the Result within the logframe where the error is located and may show many lines (as below) if there are several issues within that result.
Third layer
The next layer shows the indicator (if the error is on indicator level). Visible also in the screenshot above.
Fourth layer
The fourth is the last layer and is shown if the error is on current (actual) value level. It then provides information for what the error is.
Common error phrases and their meaning
An error can occur during the encoding of information in OPSYS, or when the system is enhanced, leading to new rules or features being implemented causing previously entered data to become incorrect.
...
Overview
Content Tools
Apps