Child pages
  • API - Getting started with SDMX3.0 API

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »

Introduction

The SDMX initiative sets standards to facilitate the exchange of statistical data and metadata using modern information technology. SDMX has also been published as an ISO International Standard (ISO 17369).

The operations in this API supports SMDX-3.0 artefacts and implements the SDMX REST API specification v2.0.

To make the most of this guide, a basic knowledge of XML and REST webservices is required.

The main elements are referred to as SDMX artefacts. Please find below a short definition of some terms used in SDMX:

  • Dataset: a collection of related observations, organised according to a predefined structure

  • Data Structure Definition (DSD): metadata describing the structure and organisation of a dataset, the statistical concepts and attached to them code lists used within the dataset

  • Dimensions: concepts that determine the dataset’s "physical" structure

  • Codelist: a code list is a predefined list from which some statistical coded concepts take their values. Each code list has the following properties:

    • identifier (it provides a unique identification within the set of code lists specified by a structural definitions maintenance agency);

    • name (also unique);

    • description (a description of the purpose of the code list); and

    • code value length (either an exact or a maximum number of characters and a type, i.e. numeric or alphanumeric).

  • Attributes: give additional information about the concepts used and do not affect the dataset structure itself

  • Dataflow: a structure which describes, categorises and constrains the allowable content of a dataset that providers supply for different reference periods

  • Concept scheme: the descriptive information for an arrangement or division of concepts into groups based on characteristics, which the objects have in common. A concept scheme is a maintained list of concepts that are used in key family and metadata structure definitions (Definitions from EUROSTAT SDMX info space and OECD Glossary of statistical terms)

For in depth details, check as well the learning section of SDMX.org or the  formal definition of the SDMX information model.

About versioned artefacts

While all SDMX artefacts could be versioned, currently only the following structural artefacts are versioned: Code lists (CL), Concept Schemes (CS), Data structure definitions (DSD).

It means that such artefacts are final and identified by a version number and safe to be copied/cached for further reference.

Other artefacts : Dataflow, ContentConstraint will always have the default version '1.0' and need to be requested again for updates.

Retrieving Structural metadata artefacts

Starting from the online data code of a dataset of choice, it is possible to query the API for detailed metadata on this data

Call in this guide, explicitly request not compressed response but it is recommended to use compression in automated or recurrent calls.

Looking into the metadata of a dataset

in the SDMX Dataflow

Taking the dataset ISOC_CI_ID_H as example, its main information are available in its Dataflow SDMX artefact

(warning) This resource is not versioned, so 1.0 or + (meaning latest) can be used interchangeably 

The minimal response would always contains the dataset label and the reference to the versioned DSD currently used by the dataset

<s:Dataflow id="ISOC_CI_ID_H" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ESTAT:ISOC_CI_ID_H(1.0)" agencyID="ESTAT" version="1.0">
  <c:Name xml:lang="de">Haushalte - Verfügbarkeit von Internet-Geräten</c:Name>
  <c:Name xml:lang="en">Households - devices to access the internet</c:Name>
  <c:Name xml:lang="fr">Ménages - dispositifs pour accéder à l'internet</c:Name>
  <s:Structure>urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=ESTAT:ISOC_CI_ID_H(28.0)</s:Structure>
</s:Dataflow>

Additionally a set of annotations would provide additional information (omitted in previous example, please expand full XML below to see them) 

Annotation type 

Description

Value(s) (in AnnotationTitle

or multi-lingual AnnotationText)

OBS_COUNTNumber of statistical observations in the dataset95814
OBS_PERIOD_OVERALL_OLDESTOldest TIME position reported in an observation2002
OBS_PERIOD_OVERALL_LATESTLatest TIME position reported in an observation2014
UPDATE_STRUCTURE

Timestamp when the dataset structure last changed

  • structural change to the list of dimensions
  • change in list of dimension positions
2021-02-08T23:00:00+0100
UPDATE_DATATimestamp when the dataset data last changed2023-05-10T11:00:00+0200
ESMS_HTMLLink to Reference Metadata pagehttps://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm
ESMS_SDMXLink to Reference Metadata archivehttps://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip
SOURCE_INSTITUTIONSSource institutionEurostat
ISOC_CI_ID_H Dataflow XML
<?xml version='1.0' encoding='UTF-8'?>
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/common" capture-installed="true">
	<m:Header>
		<m:ID>0A6990803FDB4C93B9A98B0BAC50CE2A</m:ID>
		<m:Test>false</m:Test>
		<m:Prepared>2023-06-08T12:32:32.883Z</m:Prepared>
		<m:Sender id="ESTAT"/>
	</m:Header>
	<m:Structures>
		<s:Dataflows>
			<s:Dataflow id="ISOC_CI_ID_H" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ESTAT:ISOC_CI_ID_H(1.0)" agencyID="ESTAT" version="1.0">
				<c:Annotations>
					<c:Annotation>
						<c:AnnotationTitle>DATASET</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_OBJECT_TYPE</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>95814</c:AnnotationTitle>
						<c:AnnotationType>OBS_COUNT</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2002</c:AnnotationTitle>
						<c:AnnotationType>OBS_PERIOD_OVERALL_OLDEST</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2014</c:AnnotationTitle>
						<c:AnnotationType>OBS_PERIOD_OVERALL_LATEST</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2021-02-08T23:00:00+0100</c:AnnotationTitle>
						<c:AnnotationType>UPDATE_STRUCTURE</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2023-05-10T11:00:00+0200</c:AnnotationTitle>
						<c:AnnotationType>UPDATE_DATA</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>Explanatory texts (metadata)</c:AnnotationTitle>
						<c:AnnotationType>ESMS_HTML</c:AnnotationType>
						<c:AnnotationURL>https://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm</c:AnnotationURL>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>Explanatory texts (metadata)</c:AnnotationTitle>
						<c:AnnotationType>ESMS_SDMX</c:AnnotationType>
						<c:AnnotationURL>https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip</c:AnnotationURL>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationType>SOURCE_INSTITUTIONS</c:AnnotationType>
						<c:AnnotationText xml:lang="fr">Eurostat</c:AnnotationText>
						<c:AnnotationText xml:lang="en">Eurostat</c:AnnotationText>
						<c:AnnotationText xml:lang="de">Eurostat</c:AnnotationText>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>
							<adms:identifier xmlns:adms="http://www.w3.org/ns/adms#" xmlns:skos="http://www.w3.org/2004/02/skos/core.html" xmlns:dct="http://purl.org/dc/terms/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
								<adms:Identifier rdf:about="https://doi.org/10.2908/ISOC_CI_ID_H">
									<skos:notation rdf:datatype="http://purl.org/spar/datacite/doi">10.2908/ISOC_CI_ID_H</skos:notation>
									<dct:creator rdf:resource="http://publications.europa.eu/resource/authority/corporate-body/ESTAT"/>
									<dct:issued rdf:datatype="http://www.w3.org/2001/XMLSchema#date">2023-01-19T12:50:39</dct:issued>
								</adms:Identifier>
							</adms:identifier>
						</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_DOI_XML</c:AnnotationType>
					</c:Annotation>
				</c:Annotations>
				<c:Name xml:lang="de">Haushalte - Verfügbarkeit von Internet-Geräten</c:Name>
				<c:Name xml:lang="en">Households - devices to access the internet</c:Name>
				<c:Name xml:lang="fr">Ménages - dispositifs pour accéder à l'internet</c:Name>
				<s:Structure>urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=ESTAT:ISOC_CI_ID_H(28.0)</s:Structure>
			</s:Dataflow>
		</s:Dataflows>
	</m:Structures>
</m:Structure>

in the referenced data structure definition (DSD)

From the reference present in the dataflow, it is possible to query for the corresponding SDMX Data Structure Definition

(warning) These resources are versioned, so version present in the reference must be used to ensure consistency.

This definition is providing the list of dimensions used in the definition of the time-series of the dataset.

The order of dimensions will help build key filtering in the data query later

For each dimension a reference is provided

  1. to the concept holding the dimension label, the concept is one item of a concept scheme.

    In current Eurostat Dissemination Chain, there is one DSD and one Concept Scheme generated for each dataset with identical identifier (but potentially different version).

  2. to the code list holding the code and labels for the dimension positions

    These code lists are reference metadata and may contains more code and labels than the one used by a specific dataset.

    To known the list of positions present in the dataset, please refer to the Content Constraint artefact (next section).


Additionally the DSD defines

ISOC_CI_ID_H DSD XML
<?xml version='1.0' encoding='UTF-8'?>
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/common">
	<m:Header>
		<m:ID>0CF4F04F13CB48928628592AC7E846E7</m:ID>
		<m:Test>false</m:Test>
		<m:Prepared>2023-05-10T09:14:15.659Z</m:Prepared>
		<m:Sender id="ESTAT">
			<c:Name xml:lang="de">Statistische Amt der Europäischen Union (Eurostat)</c:Name>
			<c:Name xml:lang="en">Statistical Office of the European Union (Eurostat)</c:Name>
			<c:Name xml:lang="fr">Office de statistique de l'Union européenne (Eurostat)</c:Name>
		</m:Sender>
		<m:Receiver id="unknown"/>
	</m:Header>
	<m:Structures>
		<s:DataStructures>
			<s:DataStructure agencyID="ESTAT" id="ISOC_CI_ID_H" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=ESTAT:ISOC_CI_ID_H(28.0)" version="28.0">
				<c:Annotations>
					<c:Annotation>
						<c:AnnotationTitle>OBS_FLAG</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_FLAG_SETTINGS</c:AnnotationType>
						<c:AnnotationURL/>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>time</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_TIME_DIMENSION_CODE</c:AnnotationType>
						<c:AnnotationURL/>
					</c:Annotation>
				</c:Annotations>
				<c:Name xml:lang="en">ISOC_CI_ID_H data structure</c:Name>
				<s:DataStructureComponents>
					<s:DimensionList id="DimensionDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DimensionDescriptor=ESTAT:ISOC_CI_ID_H(28.0).DimensionDescriptor">
						<s:Dimension id="freq" position="1" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).freq">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).freq</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=ESTAT:FREQ(1.31)</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="indic_is" position="2" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).indic_is">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).indic_is</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=ESTAT:INDIC_IS(9.0)</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="unit" position="3" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).unit">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).unit</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=ESTAT:UNIT(16.0)</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="hhtyp" position="4" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).hhtyp">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).hhtyp</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=ESTAT:HHTYP(2.0)</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="geo" position="5" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).geo">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).geo</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=ESTAT:GEO(14.0)</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:TimeDimension id="TIME_PERIOD" urn="urn:sdmx:org.sdmx.infomodel.datastructure.TimeDimension=ESTAT:ISOC_CI_ID_H(28.0).TIME_PERIOD">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).TIME_PERIOD</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:TextFormat textType="ObservationalTimePeriod"/>
							</s:LocalRepresentation>
						</s:TimeDimension>
					</s:DimensionList>
					<s:AttributeList id="AttributeDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.AttributeDescriptor=ESTAT:ISOC_CI_ID_H(28.0).AttributeDescriptor">
						<s:Attribute id="OBS_FLAG" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DataAttribute=ESTAT:ISOC_CI_ID_H(28.0).OBS_FLAG">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).OBS_FLAG</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>urn:sdmx:org.sdmx.infomodel.codelist.Codelist=ESTAT:OBS_FLAG(1.31)</s:Enumeration>
							</s:LocalRepresentation>
							<s:AttributeRelationship>
								<s:Observation/>
							</s:AttributeRelationship>
						</s:Attribute>
					</s:AttributeList>
					<s:MeasureList id="MeasureDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.MeasureDescriptor=ESTAT:ISOC_CI_ID_H(28.0).MeasureDescriptor">
						<s:Measure id="OBS_VALUE" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Measure=ESTAT:ISOC_CI_ID_H(28.0).OBS_VALUE">
							<s:ConceptIdentity>urn:sdmx:org.sdmx.infomodel.conceptscheme.Concept=ESTAT:ISOC_CI_ID_H(28.0).OBS_VALUE</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:TextFormat textType="Double"/>
							</s:LocalRepresentation>
						</s:Measure>
					</s:MeasureList>
				</s:DataStructureComponents>
			</s:DataStructure>
		</s:DataStructures>
	</m:Structures>
</m:Structure>

in the SDMX Data Constraint

In the API, SDMX Data Constraints artefacts are used to define available positions, it lists for each dimension code the list of position code that are used at least once to refer to a statistical observation.

From example below we can see that ISOC_CI_ID_H dataset 

  • provides annual data ( freq = A)
  • provides data for 14 indicators 
  • provides data in 2 units
  • provides 17 breakdown and a TOTAL on "Type of Household" [ hhtyp ]
  • provides data for EU aggregates and member states + other countries
  • provides data from 2002 to 2010 plus 2014
Data Constraint SDMX XML
<?xml version='1.0' encoding='UTF-8'?>
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v3_0/common">
	<m:Header>
		<m:ID>BA8A9F67ED104DEEB816A3660F8170BE</m:ID>
		<m:Test>false</m:Test>
		<m:Prepared>2023-05-10T09:10:50.349Z</m:Prepared>
		<m:Sender id="ESTAT">
			<c:Name xml:lang="de">Statistische Amt der Europäischen Union (Eurostat)</c:Name>
			<c:Name xml:lang="en">Statistical Office of the European Union (Eurostat)</c:Name>
			<c:Name xml:lang="fr">Office de statistique de l'Union européenne (Eurostat)</c:Name>
		</m:Sender>
		<m:Receiver id="unknown"/>
	</m:Header>
	<m:Structures>
		<s:DataConstraints>
			<s:DataConstraint agencyID="ESTAT" id="ISOC_CI_ID_H" version="1.0" urn="urn:sdmx:org.sdmx.infomodel.registry.DataConstraint=ESTAT:ISOC_CI_ID_H(1.0)" role="Actual">
				<c:Annotations>
					<c:Annotation>
						<c:AnnotationTitle>3</c:AnnotationTitle>
						<c:AnnotationType>TABLE_COMPLEXITY</c:AnnotationType>
					</c:Annotation>
				</c:Annotations>
				<c:Name xml:lang="en">Cube description for dataflow ISOC_CI_ID_H</c:Name>
				<s:ConstraintAttachment>
					<s:Dataflow>urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ESTAT:ISOC_CI_ID_H(1.0)</s:Dataflow>
				</s:ConstraintAttachment>
				<s:CubeRegion include="true">
					<s:KeyValue id="freq">
						<s:Value>A</s:Value>
					</s:KeyValue>
					<s:KeyValue id="indic_is">
						<s:Value>H_IPC</s:Value>
						<s:Value>H_ITV</s:Value>
						<s:Value>H_IPALM</s:Value>
						<s:Value>H_IMPH</s:Value>
						<s:Value>H_IGAME</s:Value>
						<s:Value>H_IPCQ</s:Value>
						<s:Value>H_ITVQ</s:Value>
						<s:Value>H_IPALMQ</s:Value>
						<s:Value>H_IMPHQ</s:Value>
						<s:Value>H_IGAMEQ</s:Value>
						<s:Value>H_IOTHDV</s:Value>
						<s:Value>H_IDKPC</s:Value>
						<s:Value>H_IPORT</s:Value>
						<s:Value>H_ITV2</s:Value>
					</s:KeyValue>
					<s:KeyValue id="unit">
						<s:Value>PC_HH</s:Value>
						<s:Value>PC_HH_IACC</s:Value>
					</s:KeyValue>
					<s:KeyValue id="hhtyp">
						<s:Value>TOTAL</s:Value>
						<s:Value>A1</s:Value>
						<s:Value>A1_DCH</s:Value>
						<s:Value>A2</s:Value>
						<s:Value>A2_DCH</s:Value>
						<s:Value>A_GE3</s:Value>
						<s:Value>A_GE3_DCH</s:Value>
						<s:Value>ALL_NDCH</s:Value>
						<s:Value>ALL_DCH</s:Value>
						<s:Value>HH_O1</s:Value>
						<s:Value>HH_NO1</s:Value>
						<s:Value>HH_DEG1</s:Value>
						<s:Value>HH_DEG2</s:Value>
						<s:Value>HH_DEG3</s:Value>
						<s:Value>HHI_Q1</s:Value>
						<s:Value>HHI_Q2</s:Value>
						<s:Value>HHI_Q3</s:Value>
						<s:Value>HHI_Q4</s:Value>
					</s:KeyValue>
					<s:KeyValue id="geo">
						<s:Value>EU27_2020</s:Value>
						<s:Value>EU28</s:Value>
						<s:Value>EU27_2007</s:Value>
						<s:Value>EU25</s:Value>
						<s:Value>EU15</s:Value>
						<s:Value>EA</s:Value>
						<s:Value>BE</s:Value>
						<s:Value>BG</s:Value>
						<s:Value>CZ</s:Value>
						<s:Value>DK</s:Value>
						<s:Value>DE</s:Value>
						<s:Value>EE</s:Value>
						<s:Value>IE</s:Value>
						<s:Value>EL</s:Value>
						<s:Value>ES</s:Value>
						<s:Value>FR</s:Value>
						<s:Value>HR</s:Value>
						<s:Value>IT</s:Value>
						<s:Value>CY</s:Value>
						<s:Value>LV</s:Value>
						<s:Value>LT</s:Value>
						<s:Value>LU</s:Value>
						<s:Value>HU</s:Value>
						<s:Value>MT</s:Value>
						<s:Value>NL</s:Value>
						<s:Value>AT</s:Value>
						<s:Value>PL</s:Value>
						<s:Value>PT</s:Value>
						<s:Value>RO</s:Value>
						<s:Value>SI</s:Value>
						<s:Value>SK</s:Value>
						<s:Value>FI</s:Value>
						<s:Value>SE</s:Value>
						<s:Value>IS</s:Value>
						<s:Value>NO</s:Value>
						<s:Value>CH</s:Value>
						<s:Value>UK</s:Value>
						<s:Value>MK</s:Value>
						<s:Value>RS</s:Value>
						<s:Value>TR</s:Value>
					</s:KeyValue>
					<s:KeyValue id="TIME_PERIOD">
						<s:Value>2002</s:Value>
						<s:Value>2003</s:Value>
						<s:Value>2004</s:Value>
						<s:Value>2005</s:Value>
						<s:Value>2006</s:Value>
						<s:Value>2007</s:Value>
						<s:Value>2008</s:Value>
						<s:Value>2009</s:Value>
						<s:Value>2010</s:Value>
						<s:Value>2014</s:Value>
					</s:KeyValue>
				</s:CubeRegion>
			</s:DataConstraint>
		</s:DataConstraints>
	</m:Structures>
</m:Structure>

retrieve several artefacts in a single response

(info) It is not necessary to do these calls one by one.

Starting back from the Dataflow it is possible to include the referenced artefacts, at two different level

Also from the DSD, it is possible to include the referenced artefacts :

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/structure/datastructure/ESTAT/ISOC_CI_ID_H/28.0?compress=false&references=children 

Special case of Dataset listing

Instead of specifying a dataset code in the dataflow request the special character * can be used to mean ALL and retrieve a list of all Eurostat datasets in one request

(warning) It is recommended to retrieve the compressed version 

Special case of Metadata harvesting

Similarly to the request on ALL dataflows it is possible to get the latest version for all artefacts for a specified type

(warning) It is recommended to retrieve the compressed version 

Data query

in SDMX 3.0, the data query requires the dataflow complete identifier : type/agency/ID/version

Above link retrieves the complete dataset in SDMX-ML 3.0 Structure Specific Data, that is the default format as Generic Data is removed from SDMX 3.0

The data file is composed of a set of time-series identified by its series-keys containing Observation as the one show below

SDMX Generic Time Series XML Excerpt
 		<Series freq="A" geo="EA" hhtyp="TOTAL" indic_is="H_IPC" unit="PC_HH_IACC">
			<Obs TIME_PERIOD="2003" OBS_VALUE="96.27"/>
			<Obs TIME_PERIOD="2004" OBS_VALUE="96.02"/>
			<Obs TIME_PERIOD="2005" OBS_VALUE="96.27"/>
			<Obs TIME_PERIOD="2006" OBS_VALUE="96.23"/>
			<Obs TIME_PERIOD="2007" OBS_VALUE="96.59"/>
			<Obs TIME_PERIOD="2008" OBS_VALUE="84.89"/>
			<Obs TIME_PERIOD="2009" OBS_VALUE="96.81"/>
			<Obs TIME_PERIOD="2010" OBS_VALUE="97.16"/>
			<Obs TIME_PERIOD="2014" OBS_VALUE="95.63"/>
		</Series>

It is possible to further customise the query to retrieve only the needed data or to request a different output format.

Filtering data

In SDMX 3.0 REST API, there are two kind of filtering mechanism

  • Filtering on series-keys : it is possible to define for each dimension one value or a wildcard to scope the data extraction ; (warning) this filter must follow dimensions order
  • Filter by component value : it is possible to define for each dimension a list of value to filter on ; TIME_PERIOD dimension allow range ; this filter is not impacted by dimensions order

These two filtering mechanisms can be combined with the limitation that a dimension specified on the series-keys filter should not be repeated in the component value filter.

While all filtering could be done on component value filter, using series-keys filter can ease readability of the data slices targeted by a data query.

Filtering on series-keys

First level of filtering is about selecting how the dataset could be sliced. It is done by filtering on the series-keys following the dimension order as specified in the DSD

In the example of ISOC_CI_ID_H, the series-keys template is as follow

FREQ.INDIC_IS.UNIT.HHTYP.GEO

with the following syntax:

  • '*' is a wildcard for ALL values meaning no filtering is needed for this dimension 
  • A specific value can be set for a dimension
  • Partial key is supported and assume '*' wildcard value for remaining dimensions, see example below

Scope

Details on the series-keys filterLink

Single time-series

fully specified


DimensionVersion
FREQA
INDIC_ISH_IPC
UNITPC_HH_IACC
HHTYPTOTAL
GEOEA
https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?compress=false  

Single indicator

+ Single Unit

All Type and countries


DimensionVersion
FREQA
INDIC_ISH_IPC
UNITPC_HH_IACC
HHTYP*
GEO*

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.*.*?compress=false


(lightbulb) Using partial key support means that the remaining dimension .*.* can be omitted from the URL so it would be equivalent to 

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC?compress=false

This can prove useful on FREQ dimension, in dataset holding data for multiple frequency , to ensure to retrieve a single type of data (annual, quarterly, monthly...) per request as FREQ is usually the first dimension of a dataset.

Single GEO data  

As the GEO dimension is the last, previous dimension must be wildcarded

*.*.*.*.EU27_2020

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/*.*.*.*.EU27_2020?compress=false


Filtering on component value

Component value filtering has the following syntax c[dim_id]=value:

  • dim_id is the dimension id as found in the DSD 
  • value can list several positions separated by a comma
  • both dim_id and value are case insensitive
ScopeDescriptionLink

Single time-series

fully specified

Alternative query equivalent to series-keys filtering above

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0?c[freq]=A&c[indic_is]=H_IPC&c[UNIT]=PC_HH_IACC&c[hhtyp]=TOTAL&c[geo]=EA&compress=false 

Single indicator

+ Single Unit

All Type and countries

Alternative query equivalent to series-keys filtering above

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0?c[freq]=A&c[indic_is]=H_IPC&c[UNIT]=PC_HH_IACC&compress=false 

Single GEO data 

Alternative query equivalent to series-keys filtering above

Not needing to know of the dimension list and order

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0?c[geo]=EU27_2020&compress=false 


All indicators and unit

TOTAL type

Only EU+EA

The GEO list of value can only be expressed as component value filtering

Single position HHTYP = TOTAL can be expressed either way

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0?c[hhtyp]=TOTAL&c[geo]=EU27_2020,EA&compress=false


It could be expressed alternatively In combination with series-keys filter for hhtyp dimension

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/*.*.*.TOTAL.*?c[geo]=EU27_2020,EA&compress=false 

Filtering on time period

Filtering the observations to be returned based on their TIME_PERIOD value is also controlled via component value filtering as other dimension but additionally it supports value range definition prefixing the value by a short operator and a colon ': '

Reusing above single time-series example, it can be restricted on TIME_PERIOD

ScopeDescriptionLink
Single time valueGet value for 2010 only

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=2010&compress=false    

or to align with other examples below by specifying the equals operator eq

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=eq:2010&compress=false    

Rolling time

Get value after 2010


Inclusive operator  : "greater than or equals" : ge
https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=ge:2010&compress=false    
Exclusive operator : "greater than" gt

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=gt:2010&compress=false   

Until time

Get value before 2010


Inclusive operator  : "lower than or equals" : le
https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=le:2010&compress=false    
Exclusive operator : "lower than" lt

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=lt:2010&compress=false   

From - to

By combining previous operator with a '+'

it is possible to specify a time range to get data between 2008 and 2010

c[TIME_PERIOD]=ge:2008+le:2010

https://ec.europa.eu/eurostat/api/dissemination/sdmx/3.0/data/dataflow/ESTAT/ISOC_CI_ID_H/1.0/A.H_IPC.PC_HH_IACC.TOTAL.EA?c[TIME_PERIOD]=ge:2008+le:2010&compress=false    

Supported format 

An additional format parameter allows to request a response in a different semantic format

Compression

An additional compress parameter allows to disable compression. It is recommended to use it only if necessary.

Retrieving Navigation artefacts

It is worth to mention secondary navigation artefacts that represent as SDMX artefacts a classification of dataset in categories (also referred as "Navigation Tree")

  • Category Scheme : Hierarchy of categories 
  • Categorisation :  one categorisation is referencing one dataset into a category of a Category Scheme



  • No labels
_