Evaluation methodology

OVERVIEW

INTPA is presently revising its evaluation methodology, which was originally developed in 2004 and published in 2006. This fiche presents the key elements that are driving its revision; it will be replaced once the methodology is released.

What is it?

The Evaluation Methodology will be an online, interactive and comprehensive guide built around the processes guiding evaluation in INTPA (both at the intervention level and for strategic evaluations). Users will have the possibility to download and print some parts of it and a printed version may be available at a later stage, as well.By navigating and understanding the different phases of the evaluation processes in INTPA, the user will have the possibility to access different elements of the evaluation theory as well as support and guidance responding to their specific needs.The methodology aims at treating complex concepts and processes with clarity and brevity while offering the reader the possibility to deepen her/his knowledge with additional readings.While most of the content of the manual will reflect approaches that the evaluation community have consolidated by practice over the course of the years, a consistent space will be devoted also to innovative methods and tools still of limited use in INTPA but increasingly common and accepted.The European Commission approach to evaluate its external assistance focusses on results and impacts, in line with internationally recognised best practices. This revised methodology will provide an important tool to operationalise this philosophy.Being this primarily an online tool, by design it will be easily adaptable and updatable as to reflect new and emerging techniques and methods, and to accompany the natural evolution of processes.

What can it be used for?

The evaluation methodology will be an important support to accompany evaluation managers and evaluators from planning, all the way through their evaluations until dissemination of results, by offering a wide range of different methods, approaches and tools. Evaluators and partners will find in the methodology a clear representation to understand how DG INTPA understands this function and the value it attributes to it. The ultimate goal of the evaluation function, and therefore of the methodology, is the improvement of interventions and long-term policies and strategies in its partner countries and regions. Evaluation supports the strengthening of policies by analysing their results and this for Learning and Accountability purposes. The methodology will be used for enhancing the quality of evaluations in INTPA.

When can it be used?

The methodology will provide a clear and systematic guidance to support the participation of managers in each phase of the evaluation process. In addition, the methodology can be used at any time during the cycle of interventions as a valid source of references for key evaluation concepts and theories, thus helping planning interventions with evaluation in mind.   

Who can use it?

The methodology will be an open, balanced and public guide, primarily destined to INTPA staff in the EU Delegations and at the Headquarters and to evaluators.

Other stakeholders who in various ways are interested by evaluation and by the EU international partnership efforts may find the guide useful as well; in particular, partner institutions, beneficiaries of EU interventions, consulting companies and researchers.  

What are its strengths?

• User-friendly evaluation methodology.
• Digital format to facilitate navigation through the evaluation processes.
• Link theory with practice. Guiding the processes by understanding theory that lies behind the processes, allowing the user to own the exercise and to understand why, for what and how to conduct / execute / accompany an evaluation.
• Include new concepts that until now, although already commonly used by other organizations, are not yet in common use in INTPA, such as new technology-related evaluation tools, new approaches or evaluation methods.
• Gathering evaluation thought leaders in order to get a broader perspective.

What are its limitations?

• The methodology cannot be a tick the box exercise, it is not possible to give everything cooked to the user, the theory behind the process is important and the decision of how, when and why has to be taken by the user after a reflection with all the elements that the methodology can offer.

PRACTICAL APPLICATION

Key elements

The methodology will be composed of seven sections (or blocks) accessible either in sequential order or through direct navigation.

Further sections may be added later.

Proposed Structure:

The INTPA evaluation processes (i.e., Section-6) will be the core of the methodology guidance while different sections discussed above will be plugged in into the ‘processes’ to give the users a detailed understanding of the theory that lies behind them.

Requirements

Data/information

Time

Skills

Facilities and materials

Financial costs and sources

Tips and tricks

RESOURCES

Where to find it

The link will be available once the methodology is released.

Complementary guides, methodologies and tools

The European Commission (EC), DG DEVCO, Evaluation methodological approach (Previous methodology): https://europa.eu/capacity4dev/evaluation_guidelines

The European Commission (EC), DG INTPA/ESS Evaluation in hard to reach areas   

Exact External Wiki: https://wikis.ec.europa.eu/display/ExactExternalWiki/EVAL+-+e-Evaluation+Tool+and+Library

Capacity4Dev: INTPA/ESS initiatives

Capacity4Dev: Monitoring & Evaluation focal points

My INTPA: INTPA.D.4 Performance, Results and Evaluation; Internal Communication, Knowledge Management and Collaborative Methods – Evaluation Section

INTPA Academy: search "evaluation"

EU Learn: search "evaluation"  

The European Commission (EC), DG NEAR: Programming, M&E WIKI  

The Organisation for Economic Development and Cooperation, Development Assistance Committe (OECD DAC): Evaluation Criteria