Since October 2023, a new URL for files retrieval from Eurostat API is available (https://ec.europa.eu/eurostat/api/dissemination/files) and phasing out of the Bulk Download Listing (https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing) is planned in the coming weeks.
Currently both URL endpoints remains available and distribute the same content without change.
At the end of the transition phase :
Migration path is different for
The Downloads section of Databrowser is listing the content available via Eurostat Dissemination Chain and allow to download one element, a batch of selected elements (max 50) or retrieve an inventory with download links for use in scripts
Option 1 (SDMX) : Retrieve all dataflows "stubs"(only references and title) in XML https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/1.0?detail=allstubs
Option 2 (TXT) : Retrieve the inventory listing of data https://ec.europa.eu/eurostat/api/dissemination/files/inventory?type=data
Transitioning from Eurostat Bulk Download to API introduces some structural and data changes in the TSV and SDMX file formats.
They are presented in detail in the following documents:
For TSV: Migrating_to_API_TSV.pdf
For SDMX: Migrating_to_API_SDMX.pdf
Option 1 : Retrieve all code lists "stubs"(only references and title) in XML https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/codelist/ESTAT/all/latest?detail=allstubs
Option 2 : Retrieve the inventory listing of code lists https://ec.europa.eu/eurostat/api/dissemination/files/inventory?type=codelist
For a known code list code the download URL must be changed to
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/codelist/ESTAT/<CODELIST_CODE>/latest?format=TSV&lang=en |
Examples :
Option 1 : New URL https://ec.europa.eu/eurostat/api/dissemination/catalogue/metabase.txt.gz. Please consult API - Detailed guidelines - Catalogue API - Metabase for further details
Option 2 : Migrating_to_API_data_constraints.pdf explains how to get similar information per dataset from SDMX artefacts.
Option 1 : Retrieve all codelists in XML https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/codelist/ESTAT/all/latest
Option 2 : Retrieve the inventory listing of code lists https://ec.europa.eu/eurostat/api/dissemination/files/inventory?type=codelist and request the TSV content for each code list as explained above in Downloading a code list or in API - Examples - periodic data download
Code lists are versioned SDMX artefacts, in case the version mentioned in the inventory listing did not change, there is no need to re-download the content as it would be identical. |
Option 3 : (to be validated with API) https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/codelist/ESTAT/all/latest?format=TSV is working but is the column format and the order final and considered usable ? |
As mentioned in above section in for "Migrating from /data content to API"
Option 1 in XML : Retrieve all dataflows "stubs"(only references and title) in XML https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/1.0?detail=allstubs
Option 2 in TSV : Retrieve the inventory listing of data https://ec.europa.eu/eurostat/api/dissemination/files/inventory?type=data
As in Eurobase, one code list is defined for each dimension the "dimension list" is equivalent to the list of code lists.
Option 1 : Retrieve all code lists in XML https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/codelist/ESTAT/all/latest
Option 2 : Retrieve the inventory listing of codelists https://ec.europa.eu/eurostat/api/dissemination/files/inventory?type=codelist
In case table of contents was mainly used to check for updates on a specific list of datasets, please consider looking up directly the dataflow for each dataset where the information is available in SDMX Annotations Example: https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/DEMO_GIND/latest If necessary the listing for ALL dataflows can also be retrieved in SDMX-ML format https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/all/latest |
Option 1 : Retrieve new XML table of contents from API catalogue https://webgate.acceptance.ec.europa.eu/eurostat/api/dissemination/catalogue/toc/xml. Please consult API - Getting started with catalogue API for further details
Option 2 : Retrieve backward compatible XML table of contents from API catalogue https://webgate.acceptance.ec.europa.eu/eurostat/api/dissemination/catalogue/toc/legacy_xml.
The file download keeps the same basis as previously with specifying the relative path to content as a query parameter named file.
https://ec.europa.eu/eurostat/api/dissemination/files?file=<relative-path-to-content> |
In case a file does not exists HTTP response will be a 404 Not found response
Examples:
Other parameters are safely ignored so in case parameters from previous Bulk Download Listing such as sort changing the base URL to would work. |
Listing of a directory content in CSV
https://ec.europa.eu/eurostat/api/dissemination/files?format=csv&dir=
Additional options
When hierarchy=true, sort/order will be ignored |
The CSV will have the following columns :
Column name | Description |
---|---|
NAME | Directory or file name when hierarchy=true, it is prefixed with the relative path to the dir requested |
SIZE | Not specified for folders Formatted according to sizeFormat parameter NONE : raw number of bytes, easier for handling by clients (sorting READABLE: formatted with unit (13.06 KB or 9.74 MB) |
TYPE | DIR for directory Extension from filename (TXT,DOC...) |
DATE | Last modified date from filesystem Formatted according to dateFormat parameter ISO: ISO8601 format (2023-08-02T04:45:34+00:00) UNIX: number of seconds since EPOCH TIME (JAN 01 1970) READABLE: Eurostat date time format ( 01/08/2023 22:50:05) |
A download URL can be computed with the file download base URL + the current folder listed + the content of the NAME column
https://ec.europa.eu/eurostat/api/dissemination/files?file=
Example for COMEXT_OTHER_DATA with hierarchy=false, sizeFormat=READABLE, dateFormat=READABLE
https://ec.europa.eu/eurostat/api/dissemination/files?format=csv&dir=comext&hierarchy=false&sizeFormat=READABLE&dateFormat=READABLE
NAME,SIZE,TYPE,DATE EUROPROMS,,DIR,24/11/2017 12:09:15 NON_EU_DATASETS,,DIR,30/05/2022 11:06:16 TEC exercise for CA 2014.zip,122.52 KB,zip,18/07/2019 13:05:02 TEC exercise for VN and JP 2017.zip,1.54 MB,zip,16/07/2018 10:11:15 |
So the download URL for TEC exercise for CA 2014.zip would be https://ec.europa.eu/eurostat/api/dissemination/files?file=comext/TEC exercise for CA 2014.zip
Example for COMEXT_OTHER_DATA with hierarchy=true
https://ec.europa.eu/eurostat/api/dissemination/files?format=csv&dir=comext&hierarchy=true&sizeFormat=READABLE&dateFormat=READABLE
NAME,SIZE,TYPE,DATE |
So the download URL for EUROPROMS/epannsold.zip would be https://ec.europa.eu/eurostat/api/dissemination/files?file=comext/EUROPROMS/epannsold.zip
For backward compatibility with web-crawlers on /comext and /metadata, API Files will continue to serve HTML content (when format=html is specified as query parameter)