Child pages
  • API - Getting started with SDMX2.1 API

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Taking the dataset ISOC_CI_ID_H as example, its main information are available in its Dataflow SDMX artefact

(warning) This resource is not versioned, so 1.0 and latest can be used interchangeably 

...

Additionally a set of annotations (omitted in previous example, please expand full XML below) would provide additional information

Annotation type 

Description

Value(s) (in AnnotationTitle

or multi-lingual AnnotationText)

OBS_COUNTNumber of statisticals observations in the dataset95814
OBS_PERIOD_OVERALL_OLDESTOldest TIME position reported in an observation2002
OBS_PERIOD_OVERALL_LATESTLatest TIME position reported in an observation2014
UPDATE_STRUCTURE

Timestamp when the dataset structure last changed

  • structural change to the list of dimensions
  • change in list of dimension positions
2021-02-08T23:00:00+0100
UPDATE_DATATimestamp when the dataset data last changed2023-05-10T11:00:00+0200
ESMS_HTMLLink to Reference Metadata pagehttps://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm
ESMS_SDMXLink to Reference Metadata archivehttps://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip
SOURCE_INSTITUTIONSSource institutionEurostat
Code Block
languagexml
titleISOC_CI_ID_H Dataflow XML
linenumberstrue
collapsetrue
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common">
	<m:Header>
		<m:ID>DF1684040131</m:ID>
		<m:Test>false</m:Test>
		<m:Prepared>2023-05-14T06:55:31.654+02:00</m:Prepared>
		<m:Sender id="ESTAT"/>
	</m:Header>
	<m:Structures>
		<s:Dataflows>
			<s:Dataflow id="ISOC_CI_ID_H" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ESTAT:ISOC_CI_ID_H(1.0)" agencyID="ESTAT" version="1.0" isFinal="false">
				<c:Annotations>
					<c:Annotation>
						<c:AnnotationTitle>DATASET</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_OBJECT_TYPE</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>95814</c:AnnotationTitle>
						<c:AnnotationType>OBS_COUNT</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2002</c:AnnotationTitle>
						<c:AnnotationType>OBS_PERIOD_OVERALL_OLDEST</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2014</c:AnnotationTitle>
						<c:AnnotationType>OBS_PERIOD_OVERALL_LATEST</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2021-02-08T23:00:00+0100</c:AnnotationTitle>
						<c:AnnotationType>UPDATE_STRUCTURE</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>2023-05-10T11:00:00+0200</c:AnnotationTitle>
						<c:AnnotationType>UPDATE_DATA</c:AnnotationType>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>Explanatory texts (metadata)</c:AnnotationTitle>
						<c:AnnotationType>ESMS_HTML</c:AnnotationType>
						<c:AnnotationURL>https://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm</c:AnnotationURL>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>Explanatory texts (metadata)</c:AnnotationTitle>
						<c:AnnotationType>ESMS_SDMX</c:AnnotationType>
						<c:AnnotationURL>https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip</c:AnnotationURL>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationType>SOURCE_INSTITUTIONS</c:AnnotationType>
						<c:AnnotationText xml:lang="fr">Eurostat</c:AnnotationText>
						<c:AnnotationText xml:lang="en">Eurostat</c:AnnotationText>
						<c:AnnotationText xml:lang="de">Eurostat</c:AnnotationText>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>
							<adms:identifier xmlns:adms="http://www.w3.org/ns/adms#" xmlns:skos="http://www.w3.org/2004/02/skos/core.html" xmlns:dct="http://purl.org/dc/terms/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
								<adms:Identifier rdf:about="https://doi.org/10.2908/ISOC_CI_ID_H">
									<skos:notation rdf:datatype="http://purl.org/spar/datacite/doi">10.2908/ISOC_CI_ID_H</skos:notation>
									<dct:creator rdf:resource="http://publications.europa.eu/resource/authority/corporate-body/ESTAT"/>
									<dct:issued rdf:datatype="http://www.w3.org/2001/XMLSchema#date">2023-01-19T12:50:39</dct:issued>
								</adms:Identifier>
							</adms:identifier>
						</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_DOI_XML</c:AnnotationType>
					</c:Annotation>
				</c:Annotations>
				<c:Name xml:lang="de">Haushalte - Verfügbarkeit von Internet-Geräten</c:Name>
				<c:Name xml:lang="en">Households - devices to access the internet</c:Name>
				<c:Name xml:lang="fr">Ménages - dispositifs pour accéder à l'internet</c:Name>
				<s:Structure>
					<Ref id="ISOC_CI_ID_H" version="28.0" agencyID="ESTAT" package="datastructure" class="DataStructure"/>
				</s:Structure>
			</s:Dataflow>
		</s:Dataflows>
	</m:Structures>
</m:Structure>

...

From the reference present in the dataflow, it is possible to query for the corresponding SDMX Data Structure Definition

(warning) These resources are versioned, so version present in the reference must be used to ensure consistency.

...

  1. to the to the concept holding the dimension label

    Tip
    iconfalse

    In current Eurostat Dissemination Chain, there is one DSD and one ConceptScheme generated for each dataset with identical identifier (but potentially different version).

  2. to the code lists holding the code and labels for the dimension positions


    These code lists are reference metadata and may contains more code and labels that the one used by a specific dataset.

    To known the list of positions present in the dataset, please refer to the Content Constraint artefact (next section).


...

Code Block
languagexml
titleISOC_CI_ID_H DSD XML
linenumberstrue
collapsetrue
<?xml version='1.0' encoding='UTF-8'?>
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common">
	<m:Header>
		<m:ID>DS-ISOC_CI_ID_H1683705333393</m:ID>
		<m:Test>false</m:Test>
		<m:Prepared>2023-05-10T07:55:33.393Z</m:Prepared>
		<m:Sender id="ESTAT">
			<c:Name xml:lang="de">Statistische Amt der Europäischen Union (Eurostat)</c:Name>
			<c:Name xml:lang="en">Statistical Office of the European Union (Eurostat)</c:Name>
			<c:Name xml:lang="fr">Office de statistique de l'Union européenne (Eurostat)</c:Name>
		</m:Sender>
		<m:Receiver id="unknown"/>
	</m:Header>
	<m:Structures>
		<s:DataStructures>
			<s:DataStructure agencyID="ESTAT" id="ISOC_CI_ID_H" isFinal="true" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=ESTAT:ISOC_CI_ID_H(28.0)" version="28.0">
				<c:Annotations>
					<c:Annotation>
						<c:AnnotationTitle>OBS_FLAG</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_FLAG_SETTINGS</c:AnnotationType>
						<c:AnnotationURL/>
					</c:Annotation>
					<c:Annotation>
						<c:AnnotationTitle>time</c:AnnotationTitle>
						<c:AnnotationType>DISSEMINATION_TIME_DIMENSION_CODE</c:AnnotationType>
						<c:AnnotationURL/>
					</c:Annotation>
				</c:Annotations>
				<c:Name xml:lang="en">ISOC_CI_ID_H data structure</c:Name>
				<s:DataStructureComponents>
					<s:DimensionList id="DimensionDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DimensionDescriptor=ESTAT:ISOC_CI_ID_H(28.0).DimensionDescriptor">
						<s:Dimension id="freq" position="1" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).freq">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="freq" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>
									<Ref agencyID="ESTAT" class="Codelist" id="FREQ" package="codelist" version="1.31"/>
								</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="indic_is" position="2" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).indic_is">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="indic_is" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>
									<Ref agencyID="ESTAT" class="Codelist" id="INDIC_IS" package="codelist" version="9.0"/>
								</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="unit" position="3" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).unit">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="unit" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>
									<Ref agencyID="ESTAT" class="Codelist" id="UNIT" package="codelist" version="16.0"/>
								</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="hhtyp" position="4" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).hhtyp">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="hhtyp" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>
									<Ref agencyID="ESTAT" class="Codelist" id="HHTYP" package="codelist" version="2.0"/>
								</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:Dimension id="geo" position="5" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).geo">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="geo" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>
									<Ref agencyID="ESTAT" class="Codelist" id="GEO" package="codelist" version="14.0"/>
								</s:Enumeration>
							</s:LocalRepresentation>
						</s:Dimension>
						<s:TimeDimension id="TIME_PERIOD" position="6" urn="urn:sdmx:org.sdmx.infomodel.datastructure.TimeDimension=ESTAT:ISOC_CI_ID_H(28.0).TIME_PERIOD">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="TIME_PERIOD" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:TextFormat textType="ObservationalTimePeriod"/>
							</s:LocalRepresentation>
						</s:TimeDimension>
					</s:DimensionList>
					<s:AttributeList id="AttributeDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.AttributeDescriptor=ESTAT:ISOC_CI_ID_H(28.0).AttributeDescriptor">
						<s:Attribute assignmentStatus="Conditional" id="OBS_FLAG" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DataAttribute=ESTAT:ISOC_CI_ID_H(28.0).OBS_FLAG">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="OBS_FLAG" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:Enumeration>
									<Ref agencyID="ESTAT" class="Codelist" id="OBS_FLAG" package="codelist" version="1.31"/>
								</s:Enumeration>
							</s:LocalRepresentation>
							<s:AttributeRelationship>
								<s:PrimaryMeasure>
									<Ref id="OBS_VALUE"/>
								</s:PrimaryMeasure>
							</s:AttributeRelationship>
						</s:Attribute>
					</s:AttributeList>
					<s:MeasureList id="MeasureDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.MeasureDescriptor=ESTAT:ISOC_CI_ID_H(28.0).MeasureDescriptor">
						<s:PrimaryMeasure id="OBS_VALUE" urn="urn:sdmx:org.sdmx.infomodel.datastructure.PrimaryMeasure=ESTAT:ISOC_CI_ID_H(28.0).OBS_VALUE">
							<s:ConceptIdentity>
								<Ref agencyID="ESTAT" class="Concept" id="OBS_VALUE" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/>
							</s:ConceptIdentity>
							<s:LocalRepresentation>
								<s:TextFormat textType="Double"/>
							</s:LocalRepresentation>
						</s:PrimaryMeasure>
					</s:MeasureList>
				</s:DataStructureComponents>
			</s:DataStructure>
		</s:DataStructures>
	</m:Structures>
</m:Structure>

...

in the SDMX Content Constraint

In the API, SDMX Content Constraints artefacts are used to define available positions, it lists for each dimension code the list of position code that are used at least once to refer to a statistical observation.

...

Starting back from the Dataflow it is possible to include the referenced artefacts, at two different level

Special case of Dataset listing

Instead of specifying a dataset code in the dataflow request the ALL keyword can be used to retrieve a list of all Eurostat datasets in one request


Special case of Metadata harvesting

Similarly to the request on ALL dataflows it is possible to get the latest version for all artefacts for a specified type

Data query

Tip

in SDMX 2.1, the data query directly use the dataflow identifier without agencyID

Above link retrieves the complete dataset in default format : SDMX-ML 2.1 Generic Data.

...

  • a blank means no filtering for this dimension 
  • Several values for a dimension must be separated by a '+' character


Scope

Details on the series-keys filterLink

Single time-series

fully specified


DimensionVersion
FREQA
INDIC_ISH_IPC
UNITPC_HH_IACC
HHTYPTOTAL
GEOEA
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/A.H_IPC.PC_HH_IACC.TOTAL.EA 

EU27 and EA data 

As the 

Code Block
....EU27_2020+EA
https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/....EU27_2020+EA 

Filtering on time period

Filtering the observations to be returned based on their TIME_PERIOD value is controller via a FROM-TO filter with the query parameter startPeriod and endPeriod

...

An additional format parameter allows to request a response in a different semantic format

Compression

An additional compressed parameter allows to optimize network transfer by retrieving the content compressed as GZIP


Retrieving Navigation artefacts

...

  • Category Scheme : Hierarchy of categories 
  • Categorisation :  one categorisation is referencing one dataset into a category of a Category Scheme