Child pages
  • API - Getting started with SDMX2.1 API

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The operations in this API supports SMDX-2.1 artefacts and implements the 2.1 version of SDMX Guidelines for the use of Web Services formalized as the REST API specification v1.5.

To make the most of this guide, a basic knowledge of XML and REST webservices web services is required.

The main elements are refered referred to as SDMX artefacts.  Below are Here is a short definition of some terms that are used in SDMX and their definitions:

  • Dataset: a collection of related observations, organized organised according to a predefined structure

  • Data Structure Definition (DSD): metadata describing the structure and organization organisation of a dataset, the statistical concepts and attached to them code lists used within the dataset

  • Dimensions: concepts that determine the dataset’s "physical" structure

  • Codelist: a code list is a predefined list from which some statistical coded concepts take their values. Each code list has the following properties:

    • identifier (it provides a unique identification within the set of code lists specified by a structural definitions maintenance agency);

    • name (also unique);

    • description (a description of the purpose of the code list); and

    • code value length (either an exact or a maximum number of characters and a type, i.e. numeric or alphanumeric).

  • Attributes: give additional information about the concepts used and do not affect the dataset structure itself

  • Dataflow: a structure which describes, categorizes categorises and constrains the allowable content of a dataset that providers supply for different reference periods

  • Concept scheme: the descriptive information for an arrangement or division of concepts into groups based on characteristics, which the objects have in common. A concept scheme is a maintained list of concepts that are used in key family and metadata structure definitions (Definitions from EUROSTAT SDMX info space and OECD Glossary of statistical terms)

...

About versioned artefacts

...

While all SDMX artefacts could be versioned

...

, currently only the following structural artefacts are versioned: Code lists (CL), Concept Schemes (CS), Data structure definitions (DSD).

It means that such artefacts

...

are final and identified by a version number and safe to be copied/cached for further reference.

Other artefacts : Dataflow, ContentConstraint will always have the default version '1.0' and need to be requested again for updates.

...

Additionally a set of annotations would provide additional information (omitted in previous example, please expand full XML below ) would provide additional informationto see them) 

Annotation type 

Description

Value(s) (in AnnotationTitle

or multi-lingual AnnotationText)

OBS_COUNTNumber of statisticals statistical observations in the dataset95814
OBS_PERIOD_OVERALL_OLDESTOldest TIME position reported in an observation2002
OBS_PERIOD_OVERALL_LATESTLatest TIME position reported in an observation2014
UPDATE_STRUCTURE

Timestamp when the dataset structure last changed

  • structural change to the list of dimensions
  • change in list of dimension positions
2021-02-08T23:00:00+0100
UPDATE_DATATimestamp when the dataset data last changed2023-05-10T11:00:00+0200
ESMS_HTMLLink to Reference Metadata pagehttps://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm
ESMS_SDMXLink to Reference Metadata archivehttps://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip
SOURCE_INSTITUTIONSSource institutionEurostat

...

(warning) These resources are versioned, so version present in the reference must be used to ensure consistency.

This definition is informing about the providing list of dimensions used in the definition of the time-series of the dataset.

...

For each dimension a reference is provided

  1. to the to the concept holding the dimension label, the concept is one item of a concept scheme.

    Tip
    iconfalse

    In current Eurostat Dissemination Chain, there is one DSD and one ConceptScheme generated for each dataset with identical identifier (but potentially different version).

  2. to the code lists holding the code and labels for the dimension positions

    These code lists are reference metadata and may contains more code and labels that than the one used by a specific dataset.

    To known the list of positions present in the dataset, please refer to the Content Constraint artefact (next section).

Additionally the DSD defines

...

  • provides annual data ( freq = A)
  • provides data for 14 indicators 
  • provides data in 2 units
  • provides 17 breakdowns and a breakdown TOTAL on hhtyp ( "Type of Household" )[ hhtyp ]
  • provides data for EU aggregates and member states + other countries
  • provides data from 2002 to 2010 and plus 2014
Code Block
languagexml
titleContent Constraint SDMX XML
linenumberstrue
collapsetrue
<?xml version='1.0' encoding='UTF-8'?>
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common">
	<m:Header>
		<m:ID>DS-ISOC_CI_ID_H1683705333706</m:ID>
		<m:Test>false</m:Test>
		<m:Prepared>2023-05-10T07:55:33.706Z</m:Prepared>
		<m:Sender id="ESTAT">
			<c:Name xml:lang="de">Statistische Amt der Europäischen Union (Eurostat)</c:Name>
			<c:Name xml:lang="en">Statistical Office of the European Union (Eurostat)</c:Name>
			<c:Name xml:lang="fr">Office de statistique de l'Union européenne (Eurostat)</c:Name>
		</m:Sender>
		<m:Receiver id="unknown"/>
	</m:Header>
	<m:Structures>
		<s:Constraints>
			<s:ContentConstraint agencyID="ESTAT" id="ISOC_CI_ID_H" isFinal="false" type="Actual" urn="urn:sdmx:org.sdmx.infomodel.registry.ContentConstraint=ESTAT:ISOC_CI_ID_H(1.0)" version="1.0">
				<c:Name xml:lang="en">Cube description for dataflow ISOC_CI_ID_H</c:Name>
				<s:ConstraintAttachment>
					<s:Dataflow>
						<Ref agencyID="ESTAT" class="Dataflow" id="ISOC_CI_ID_H" package="datastructure" version="1.0"/>
					</s:Dataflow>
				</s:ConstraintAttachment>
				<s:CubeRegion include="true">
					<c:KeyValue id="freq">
						<c:Value>A</c:Value>
					</c:KeyValue>
					<c:KeyValue id="indic_is">
						<c:Value>H_IPC</c:Value>
						<c:Value>H_ITV</c:Value>
						<c:Value>H_IPALM</c:Value>
						<c:Value>H_IMPH</c:Value>
						<c:Value>H_IGAME</c:Value>
						<c:Value>H_IPCQ</c:Value>
						<c:Value>H_ITVQ</c:Value>
						<c:Value>H_IPALMQ</c:Value>
						<c:Value>H_IMPHQ</c:Value>
						<c:Value>H_IGAMEQ</c:Value>
						<c:Value>H_IOTHDV</c:Value>
						<c:Value>H_IDKPC</c:Value>
						<c:Value>H_IPORT</c:Value>
						<c:Value>H_ITV2</c:Value>
					</c:KeyValue>
					<c:KeyValue id="unit">
						<c:Value>PC_HH</c:Value>
						<c:Value>PC_HH_IACC</c:Value>
					</c:KeyValue>
					<c:KeyValue id="hhtyp">
						<c:Value>TOTAL</c:Value>
						<c:Value>A1</c:Value>
						<c:Value>A1_DCH</c:Value>
						<c:Value>A2</c:Value>
						<c:Value>A2_DCH</c:Value>
						<c:Value>A_GE3</c:Value>
						<c:Value>A_GE3_DCH</c:Value>
						<c:Value>ALL_NDCH</c:Value>
						<c:Value>ALL_DCH</c:Value>
						<c:Value>HH_O1</c:Value>
						<c:Value>HH_NO1</c:Value>
						<c:Value>HH_DEG1</c:Value>
						<c:Value>HH_DEG2</c:Value>
						<c:Value>HH_DEG3</c:Value>
						<c:Value>HHI_Q1</c:Value>
						<c:Value>HHI_Q2</c:Value>
						<c:Value>HHI_Q3</c:Value>
						<c:Value>HHI_Q4</c:Value>
					</c:KeyValue>
					<c:KeyValue id="geo">
						<c:Value>EU27_2020</c:Value>
						<c:Value>EU28</c:Value>
						<c:Value>EU27_2007</c:Value>
						<c:Value>EU25</c:Value>
						<c:Value>EU15</c:Value>
						<c:Value>EA</c:Value>
						<c:Value>BE</c:Value>
						<c:Value>BG</c:Value>
						<c:Value>CZ</c:Value>
						<c:Value>DK</c:Value>
						<c:Value>DE</c:Value>
						<c:Value>EE</c:Value>
						<c:Value>IE</c:Value>
						<c:Value>EL</c:Value>
						<c:Value>ES</c:Value>
						<c:Value>FR</c:Value>
						<c:Value>HR</c:Value>
						<c:Value>IT</c:Value>
						<c:Value>CY</c:Value>
						<c:Value>LV</c:Value>
						<c:Value>LT</c:Value>
						<c:Value>LU</c:Value>
						<c:Value>HU</c:Value>
						<c:Value>MT</c:Value>
						<c:Value>NL</c:Value>
						<c:Value>AT</c:Value>
						<c:Value>PL</c:Value>
						<c:Value>PT</c:Value>
						<c:Value>RO</c:Value>
						<c:Value>SI</c:Value>
						<c:Value>SK</c:Value>
						<c:Value>FI</c:Value>
						<c:Value>SE</c:Value>
						<c:Value>IS</c:Value>
						<c:Value>NO</c:Value>
						<c:Value>CH</c:Value>
						<c:Value>UK</c:Value>
						<c:Value>MK</c:Value>
						<c:Value>RS</c:Value>
						<c:Value>TR</c:Value>
					</c:KeyValue>
					<c:KeyValue id="TIME_PERIOD">
						<c:Value>2002</c:Value>
						<c:Value>2003</c:Value>
						<c:Value>2004</c:Value>
						<c:Value>2005</c:Value>
						<c:Value>2006</c:Value>
						<c:Value>2007</c:Value>
						<c:Value>2008</c:Value>
						<c:Value>2009</c:Value>
						<c:Value>2010</c:Value>
						<c:Value>2014</c:Value>
					</c:KeyValue>
				</s:CubeRegion>
			</s:ContentConstraint>
		</s:Constraints>
	</m:Structures>
</m:Structure>

...

(info) It is not necessary to do these call calls one by one.

Starting back from the Dataflow it is possible to include the referenced artefacts, at two different level

Also form the DSD, it is possible to include the referenced artefacts:

Code Block
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/datastructure/ESTAT/ISOC_CI_ID_H/28.0?references=children

Special case of Dataset listing

Instead of specifying a dataset code in the dataflow request the ALL keyword can be used to retrieve a list of all Eurostat datasets in one request

(warning) It is recommended to retrieve the compressed version 

...

Scope

Details on the series-keys filterLink

Single time-series

fully specified


DimensionVersionValue
FREQA
INDIC_ISH_IPC
UNITPC_HH_IACC
HHTYPTOTAL
GEOEA
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/A.H_IPC.PC_HH_IACC.TOTAL.EA 

EU27 and EA data 

As the the GEO dimension is the last, previous dimension must be left blank

Code Block
....EU27_2020+EA
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/....EU27_2020+EA 

...

Filtering the observations to be returned based on their TIME_PERIOD value is controller via a FROM-TO filter with the query parameter startPeriod and endPeriod ((warning) case sensitive)

Reusing above single time-series example, it can be restricted to  2008 to 2010 as follow

Code Block
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/A.H_IPC.PC_HH_IACC.TOTAL.EU27_2020?startPeriod=2008&endPeriod=2010

...

Retrieving Navigation artefacts

It is worth to mention that secondary artefacts exists to that represent as SDMX artefacts a classification of dataset in categories (also refered refrered as "Navigation Tree in Eurostat")

  • Category Scheme : Hierarchy of categories 
  • Categorisation one one categorisation is referencing one dataset into a category of a Category Scheme

...