2. EVAL Evaluations

2.1 Find and Access an Evaluation

EVAL Contractors have access to their Evaluations (once the corresponding Evaluation Contract is signed) in statuses/steps "CONTRACTED" (if the Specific Contract is linked to the Evaluation), "INCEPTION", "INTERIM", "SYNTHESIS" and "COMPLETED".

EVAL Experts have access to Evaluations (once the corresponding Evaluation Contract is signed) for which they have been selected and in Evaluation statuses/steps "CONTRACTED" (if the Specific Contract is linked to the Evaluation), "INCEPTION", "INTERIM", "SYNTHESIS" and "COMPLETED".

EVAL Reference Group Members (external) have access to Evaluations for which they have been added as evaluation stakeholders, and in all Evaluation statuses/steps.

To find and access an Evaluation, use your EU Login to first login to the Funding and Tender Opportunities Portal.

Once logged-in, you can access OPSYS EVAL from the Manage my area > Results > Monitoring & Evaluation option (1) in the left main menu. On the Monitoring & Evaluation page, the Evaluations section (2) is accessible from the left Navigation menu.

The Evaluations section displays the list of Evaluations (3) for which you are either the Contractor, or have been assigned as Expert or external Reference Group Member. If needed, use the advanced filter features to find the Evaluation.

Click on the Evaluation ID link (4) of the Evalauation to access the Evaluation page.

2.2 Evaluation Planning Phase

The Evaluation Planning Phase consists of three statuses/steps in the system ("PLANNED", "TOR" and "CONTRACTED"), which correspond to the status of the Evaluation in the workflow. Each step is managed by the Evaluation actor(s) responsible for the specific step in the workflow, and is detailed for the phase in the sub-sections below.

2.2.1 Evaluation Planned

2.2.2 Evaluation TOR 

The Evaluation is launched when it is set to status "TOR". The contracting procedure is initiated by the Contracting Authority, and the Request for Specific Contract for the Evaluation is launched and published to Framework Contractors on the Funding and Tenders Portal.

1. Framework Contractors can respond to an invitation to participate by acknowledging the invitation and expressing willingness to participate.
2. Framework Contractors complete their submissions and submit their offers.
3. Once the Specific Contract is awarded to the EVAL Contractor, the Contract is prepared using the Terms of Reference (TOR) templates of the Framework Contract, and with the support of the methodological guidance. The approval of the TORs is requested from the external Reference Group Members of the Evaluation by the Contracting Authority.
4. The EVAL Contractor confirms that the EVAL Expert(s) are available, and that they have their EU Login details.
5. Once the Specific Contract is prepared and approved by the Contracting Authority, the EVAL Contractor signs the Contract, followed by the final signature of the Contract by the Contracting Authority.

There is no action required in OPSYS EVAL from external actors for the Evaluation in status "TOR".

Once the above-listed steps are performed and the Specific Contract is signed by both parties, the Evaluation Manager will submit the Evaluation in OPSYS EVAL to the next status/step "CONTRACTED".

If you need methodological support for finalising the ToRs, please contact INTPA EVALUATION SUPPORT (for evaluations managed by DG INTPA) or NEAR EVALUATION HELP DESK (for evaluations managed by DG NEAR).

2.2.3 Evaluation Contracted

After the signing of the Specific Contract by both parties, the EVAL Contractor can access and download the signed Specific Contract from the Funding and Tenders Portal.

If requested, the bank guarantee is provided by the EVAL Contractor.

If applicable, Pre-financing Payment process is initiated automatically by the Contracting Authority.

The Evaluation Manager will verify the stakeholders' information (including the EVAL Reference Group Members) for the Evaluation, and update if needed.

The Evaluation Manager will then link the Specific Contract to the Evaluation.

If defined in the Specific Contract, the EVAL Contractor and EVAL Expert(s) are automatically retrieved from the Specific Contract to the Evaluation.

When the Specific Contract is linked to the Evaluation, the Evaluation Manager will link the required Contractual deliverables to their associated Evaluation "INCEPTION", "INTERIM" and "SYNTHESIS" statuses/steps.

There is no action required in OPSYS EVAL from external actors for the Evaluation in status "CONTRACTED".

Once the Specific Contract is linked to the Evaluation and all the above steps are completed internally, the Evaluation Manager will submit the Evaluation in OPSYS EVAL to the next status/step "INCEPTION" and the Evaluation Planning Phase is complete.

2.3 Evaluation Implementation Phase

The Evaluation Implementation Phase consist of three statuses/steps in the system ("INCEPTION", "INTERIM" and "SYNTHESIS"), which correspond to the status of the Evaluation in the workflow. Each step is managed by the Evaluation actor(s) responsible for the specific step in the workflow, and is detailed for the phase in the sub-sections below.

2.3.1 Evaluation Inception

The Evaluation is now accessible to the EVAL Contractor and EVAL Expert(s) from OPSYS EVAL (via Results > Monitoring & Evaluation > Evaluations section of the Funding and Tenders Portal).

The management of Contract deliverables by the EVAL Contractor is performed from the Specific Contract. During the "INCEPTION" status/step of the Evaluation, the EVAL Contractor uploads and submits an Inception Report deliverable before the set deadline. Once the deliverables are submitted by the EVAL Contractor, a notification is sent to the Contracting Authority to review the deliverables. The Contracting Authority can add comments to the deliverable, and either send the deliverable back to the EVAL Contractor for revision, or approve the deliverable. When the deliverable is approved, the Evaluation Manager selects the relevant deliverable for the Evaluation to indicate that the deliverable has been correctly submitted, and the Evaluation can proceed to the next step.

There is no action required in OPSYS EVAL from external actors for the Evaluation in status "INCEPTION".

After successfully selecting the Inception Report (and other(s) if applicable) deliverable in the Evaluation, the Evaluation Manager will submit the Evaluation in OPSYS EVAL to the next status/step "INTERIM".

2.3.2 Evaluation Interim

The management of Contract deliverables by the EVAL Contractor is performed from the Specific Contract. During the "INTERIM" status/step of the Evaluation, the EVAL Contractor uploads and submits an Interim Report deliverable (and any other required deliverable for this Evaluation status/step) before the set deadline. Once the deliverables are submitted by the EVAL Contractor, a notification is sent to the Contracting Authority to review the deliverables. The Contracting Authority can add comments to the deliverable, and either send the deliverable back to the EVAL Contractor for revision, or approve the deliverable. When the deliverable is approved, the Evaluation Manager selects the relevant deliverable for the Evaluation to indicate that the deliverable has been correctly submitted, and the Evaluation can proceed to the next step.

There is no action required in OPSYS EVAL from external actors for the Evaluation in status "INCEPTION".

After successfully selecting the Interim Report (and other(s) if applicable) deliverable in the Evaluation, the Evaluation Manager will submit the Evaluation in OPSYS EVAL to the next status/step "SYNTHESIS". The Contracting Authority manages the interim payment(s).

2.3.3 Evaluation Synthesis

The management of Contract deliverables by the EVAL Contractor is performed from the Specific Contract. During the "Synthesis" status/step of the Evaluation, the EVAL Contractor uploads and submits a Final Report deliverable (and any other required deliverable for this Evaluation status/step) before the set deadline. Once the deliverables are submitted by the EVAL Contractor, a notification is sent to the Contracting Authority to review the deliverables. The Contracting Authority can add comments to the deliverable, and either send the deliverable back to the EVAL Contractor for revision, or approve the deliverable. When the deliverable is approved, the Evaluation Manager selects the relevant deliverable for the Evaluation to indicate that the deliverable has been correctly submitted.

For the Evaluation in OPSYS EVAL, a draft Conclusion, lessons learned and recommendations template becomes downloadable and must be completed by the EVAL Contractor or EVAL Expert(s). Alternatively, the Draft conclusion, lessons learned and recommendations form can be completed online (see below for details). For both the template and online form, conclusions, lessons learned and recommendations can be grouped and prioritised. The EVAL Contractor confirms that the Conclusion, lessons learned and recommendations template or form is set as final for the Evaluation.  

After successfully selecting the Final Report (and other(s) if applicable) deliverable in the Evaluation, and the Conclusions, lessons learned and recommendations are submitted by the  EVAL Contractor or EVAL Expert(s) and reviewed by the Evaluation Manager, the Evaluation Manager will submit the Evaluation in OPSYS EVAL to the next status/step "COMPLETED". The Contracting Authority manages the final payment and clearing of the pre-financing (if applicable).

Fill-in the Conclusions, Lessons Learned and Recommendations 

To fill-in the Conclusions, lessons learned and recommendations, as EVAL Contractor or EVAL Expert(s), first find and access the Evaluation in OPSYS EVAL.

On the Evaluation page:

• (Recommended) If you want to Fill Conclusions, lessons learned and recommendations via the online form, choose the “Fill Conclusions, lessons learned and recommendations” option (1) and complete the online form; or
• If you want to complete the Conclusions, lessons learned and recommendations via the downloadable template (editable PDF file), choose the “Download Conclusions. lessons learned and recommendations template” option (2) to download the PDF file to your computer and complete the file.

Via the online form

On the Evaluation Online Form page, select the Evaluation forms type “With Clusters”, and the form displays specific text boxes for the cluster(s):

For the Cluster 1 text box, enter the required information. If you wish to add more clusters, click on the add icon located on the right side of the box, and then at the bottom of the page the Cluster 2 panel is displayed.

Click on the down arrow of the Cluster 2 panel to expand the panel and enter the required information. If you wish to delete a cluster, click on the delete icon of the cluster to delete.

If you delete a cluster, click on the "Yes" button in the Confirmaton pop-up message to confirm that the chosen cluster can be deleted from the form. This action cannot be undone!

Alternatively, click on the “No” button to cancel the deletion.

How to enter the Conclusions, Lessons Learned and Recommendations

For each required cluster panel, click on the down arrow of the cluster panel to expand the panel and enter the description, conclusions, lessons learned and recommendations in the corresponding tabs and text boxes.

In the Conclusions tab, enter the conclusions, and if needed, use the add and delete icons to add or remove conclusions.

In the Lessons Learned tab, enter the lessons learned, and if needed, use the add and delete icons to add or remove lessons learned.

In the Recommendations tab, enter the recommendations, and if needed, use the add and delete icons to add or remove recommendations.

For each recommendation:

• Select the Main implantation responsibility of the recommendation, and add a description;
• Select the Main associated actors of the recommendation, and add a description;
• For Linked Conclusion(s), link the recommandation to the corresponding conclusion(s) by indicating to which conclusion(s) the recommentation should be assigned (e.g. "C1" corresponds to Conclusion 1, ...). A recommendation can be linked to one or multiple conclusions;
• Select the Priotity of the recommendation;
• Select the Importance of the recommendation.

At any moment while completing the form, you can click on the "Save" button at the top right of the Evaluation Online Form page to save the form.

How to set the status of the Conclusions, Lessons Learned and Recommendations as Final

When the required conclusions, lessons learned and recommendations are all completed and reviewed, click on the "Set as Final" button at the top right of the Evaluation Online Form page to confirm the completion of the form.

Click on the "Yes" button in the Confirmaton pop-up message to confirm the completion of the Conclusions, lessons learned and recommendations form.

Alternatively, click on the “No” button to cancel the confirmed completion of the form.

If the Conclusions, lessons learned and recommendations form is successfully confirmed as completed, a green Success! message is displayed at the bottom right of the page, and:

• The status of the Evaluation Online Form is updated to "Final".
• The Conclusions, lessons learned and recommendations are registered and set to read-only.

Click on the “Back to Evaluation” button at the top of the Evaluation Online Form page to be re-directed back to Evaluation page.

Via the downloadable PDF template

Select the "With Clusters" option in the pop-up message when downloading the Conclusions, lessons learned and recommendations template.

1. Open the downloaded PDF template file.

2. Complete the required fields in the template file (using the same process as described above for the online form).

3. Once completed, save the template file to your computer.

4. Once the template file is saved, return to the Evaluation page in the system, click on the “Actions” button at the top right of the page and choose the “Upload Conclusions, lessons learned and recommendations template” option to upload the completed PDF file template from your computer to the system.

5. Once to have uploaded the completed template file, click on the "Submit" button to submit the template file.

6. Confirm the submission by clicking on the "Continue" button. Warning: The submission of the template file will overide the information on the online form!

If the Conclusions, lessons learned and recommendations template file was successfully uploaded, a green confirmation message will be displayed at the bottom right of the page.

2.4 Evaluation Completed

Following the submission and approval of the Final Report, the Conclusions, lessons learned and recommendations are submitted, and all the required deliverables are submitted, the Evaluation Manager sets the Evaluation in OPSYS EVAL to status/step “COMPLETED”. The Contracting Authority then manages the final payment and clearing of the pre-financing (if applicable).

2.5 Response of Service and Follow-up Phases

For the Response of Service Phase, inputs provided in the Evaluation are disseminated to policy makers, concerned services and partners for future reference. The Evaluation Manager fills in the online template Response of the services Report with the resposes/feedback of the services involved.

If this is the last step foreseen for the Evaluation (no Follow-up one year later is planned), the Evaluation is finalised.

The Contracting Authority closes the Specific Contract.

For the Follow-up Phase (if applicable), the follow-up of the Evaluation is carried out one year later to monitor the impact of the inputs disseminated across the stakeholders. The Evaluation Manager fill in the online template Follow-up on year later Report for the Evaluation with the input of the services and other actors involved.

The Evaluation is concluded.

The Response of Service and Follow-up phases are managed internally, so no action is required from external actors for these phases.