Literature review

OVERVIEW

What is it?

A literature review compiles and assesses existing research and evidence on a topic of interest [The International Bank for Reconstruction and Development / The World Bank, 2007 Tools for Institutional, Political, and Social Analysis of Policy Reform. Literature Review (Pages 185-188)], drawing on secondary data such as programme reports and other organisational analyses. As a method of summarising available information on a specific topic, which allows for the establishment of a theoretical framework, it is distinct from systematic review, which implies a standard scientific protocol and is considered a research technique.

What can it be used for?

A literature review can be used to:

• establish gain an overview of existing key findings on a particular topic within a country or sector;
• compile evidence (through concepts, topics or ideas) which will sustain decision-making (e.g. the impact of previous reform efforts in a given country, evidence of similar reforms in comparable countries, social composition of different sectors).

When can it be used?

A literature review is an essential tool at any phase of the intervention cycle, especially the design phase (identification and formulation), since it can provide context information, baseline data and targets.

Who can use it?

• EU staff
• Implementing partners
• Key stakeholders

What are its strengths?

• Cost-efficient tool that summarises key findings on concrete topics or questions.

What are its limitations?

• Since it relies on secondary sources, the reliability and validity of data are not always ensured. Attention should be paid to the data sources and their possible bias.

PRACTICAL APPLICATION

Key elements

Follow these steps in preparing and conducting a literature review:

Step 1: Develop and address explicitly formulated questions. A central question should be proposed that addresses the intervention about which evidence is being gathered. The central question can be then broken down into sub-questions or topics.

Step 2: Develop a methodological approach. This approach should be based on the central question identified and available resources (e.g. translation of documents in local languages).

Step 3: Apply a systematic framework for identifying and selecting the literature of interest. Evidence can be provided by different sources (electronic/Internet sources, print sources or 'grey sources' such as unpublished material).

Step 4: Apply a systematic framework for interpreting and analysing the data. In order to distinguish what evidence shall be retained, selection criteria must be defined based on quality and credibility standards. These criteria should relate to:

• the questions, stakeholders and objectives of the intervention being addressed;
• the assessment of biaise in primary studies.

Step 5: Produce a summary of findings from the existing evidence. Once the findings have been interpreted, the key messages and implications for policymaking or implementation should be summarised clearly and succinctly. Important messages regarding future research needs or research gaps encountered in the area covered by the literature review should also be summarised.

Step 6: Draft the final literature review report. This will include:

• introduction/overview of the topic under review;
• description of methodological approach adopted;
• main findings by question or topic;
• research gaps identified;
• recommendations (if requested);
• selected bibliography.

Requirements

Data/information. A literature review relies entirely on access to data and background information on the intervention under review. A deep knowledge of the topic area by the reviewer is also required.

Time. Literature reviews are usually time-intensive exercises. The duration of the review will depend on its scope and rigor. An average of four to six weeks will likely be required.

Skills. Analytical skills are needed together with the necessary language skills. The greatest bias in traditional reviews stems from a reliance on studies written in English and published in international journals.

Facilities and materials. N/A

Financial costs and sources. If external expertise is required to carry out the review, funds should be made available to cover the costs associated with fees, travel expenses and logistics. These funds may come from the project itself or through other EC instruments such as a framework contract or a technical cooperation facility.

Tips and tricks
Different types of sources (formal/informal, published/unpublished, etc.) may be relevant at different phases of the intervention cycle; these should be integrated and triangulated to the extent possible.
Examples of sources:

• bilateral and multilateral partnership agreements
• sector-based/thematic agreements (trade, investment, science and technology, etc.); co-operation, partnership and association agreements; conclusions of bilateral and multilateral conferences
• country/regional strategy papers (CSPs and RSPs) and national/regional indicative papers (NIPs and RIPs);
• country poverty reduction strategy papers (PRSP) and available sector-based national strategies
• other multilateral and bilateral cooperation agency reports, such as public expenditure and financial accountability (PEFA), UN Universal Periodic Review (UNCPR), etc.
• available strategic thematic or sector-based project and programme evaluations
• relevant policy documents – environment and climate change:
• nationally determined contributions (NDCs), national biodiversity strategy and action plans (NBSAPs), environmental policies and strategies, sector strategies
• World Bank country environmental analyses or UN environmental profiles
• environment- and climate-related legislation
• existing environmental and climate change screenings, strategic environmental assessments (SEAs), environmental impact assessments (EIAs) and/or climate risk assessments (CRAs)
• Relevant policy documents – gender:
• EU Gender Action Plan II
• Database Knowledge Bank GFP available in the EU Resource Package for Gender Mainstreaming in Development Cooperation International Training Centre of ILO, UN Women and European Commission (2015): Gender Mainstreaming in EU Dev.Coop.
• Gender Equality Index – Country Reports of the European Institute for Gender Equality
• Organisation for Economic Co-operation and Development (OECD) Development Centre's Social Institutions and Gender Index
• existing gender analyses
• Relevant policy documents – human rights and democratic governance:
• country reports of the UN Human Rights Committee
• Universal periodic review recommendations, UN Special Procedures recommendations and treaty body reports
• Human Rights Guide to the Sustainable Development Goals
• Relevant policy documents – fragility and conflict situations:
• OECD States of Fragility Report, fragility framework
• national risk management frameworks or other type of risk assessments (e.g. conflict analyses, situational risk analysis, fragility assessments, vulnerability assessments, risk and resilience assessments, post-disaster needs assessments, recovery and peacebuilding assessments, etc.)
• Monitoring reports on projects and programmes in progress.

RESOURCES

Where to find it

N/A

Complementary guides, methodologies and tools

The International Bank for Reconstruction and Development / The World Bank, 2007. Tools for Institutional, Political, and Social Analysis of Policy Reform. Literature Review (Pages 185-188).

*Relevant for Evaluation: European Commission. Directorate General External Relations. Directorate General Development. EuropeAid Co-operation Office. Joint Evaluation Unit. 2006. Evaluation methods for the European Union's External Assistance. Evaluation Tools. Volume 4. (Summary available at Capacity4dev. Evaluation methodological approach).