...
Taking the dataset ISOC_CI_ID_H as example, its main information are available in its Dataflow SDMX artefact
This resource is not versioned, so 1.0 and latest can be used interchangeably
...
Additionally a set of annotations (omitted in previous example, please expand full XML below) would provide additional information
Annotation type | Description | Value(s) (in AnnotationTitle or multi-lingual AnnotationText) |
---|---|---|
OBS_COUNT | Number of statisticals observations in the dataset | 95814 |
OBS_PERIOD_OVERALL_OLDEST | Oldest TIME position reported in an observation | 2002 |
OBS_PERIOD_OVERALL_LATEST | Latest TIME position reported in an observation | 2014 |
UPDATE_STRUCTURE | Timestamp when the dataset structure last changed
| 2021-02-08T23:00:00+0100 |
UPDATE_DATA | Timestamp when the dataset data last changed | 2023-05-10T11:00:00+0200 |
ESMS_HTML | Link to Reference Metadata page | https://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm |
ESMS_SDMX | Link to Reference Metadata archive | https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip |
SOURCE_INSTITUTIONS | Source institution | Eurostat |
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
<m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common"> <m:Header> <m:ID>DF1684040131</m:ID> <m:Test>false</m:Test> <m:Prepared>2023-05-14T06:55:31.654+02:00</m:Prepared> <m:Sender id="ESTAT"/> </m:Header> <m:Structures> <s:Dataflows> <s:Dataflow id="ISOC_CI_ID_H" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dataflow=ESTAT:ISOC_CI_ID_H(1.0)" agencyID="ESTAT" version="1.0" isFinal="false"> <c:Annotations> <c:Annotation> <c:AnnotationTitle>DATASET</c:AnnotationTitle> <c:AnnotationType>DISSEMINATION_OBJECT_TYPE</c:AnnotationType> </c:Annotation> <c:Annotation> <c:AnnotationTitle>95814</c:AnnotationTitle> <c:AnnotationType>OBS_COUNT</c:AnnotationType> </c:Annotation> <c:Annotation> <c:AnnotationTitle>2002</c:AnnotationTitle> <c:AnnotationType>OBS_PERIOD_OVERALL_OLDEST</c:AnnotationType> </c:Annotation> <c:Annotation> <c:AnnotationTitle>2014</c:AnnotationTitle> <c:AnnotationType>OBS_PERIOD_OVERALL_LATEST</c:AnnotationType> </c:Annotation> <c:Annotation> <c:AnnotationTitle>2021-02-08T23:00:00+0100</c:AnnotationTitle> <c:AnnotationType>UPDATE_STRUCTURE</c:AnnotationType> </c:Annotation> <c:Annotation> <c:AnnotationTitle>2023-05-10T11:00:00+0200</c:AnnotationTitle> <c:AnnotationType>UPDATE_DATA</c:AnnotationType> </c:Annotation> <c:Annotation> <c:AnnotationTitle>Explanatory texts (metadata)</c:AnnotationTitle> <c:AnnotationType>ESMS_HTML</c:AnnotationType> <c:AnnotationURL>https://ec.europa.eu/eurostat/cache/metadata/en/isoc_i_esms.htm</c:AnnotationURL> </c:Annotation> <c:Annotation> <c:AnnotationTitle>Explanatory texts (metadata)</c:AnnotationTitle> <c:AnnotationType>ESMS_SDMX</c:AnnotationType> <c:AnnotationURL>https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?file=metadata/isoc_i_esms.sdmx.zip</c:AnnotationURL> </c:Annotation> <c:Annotation> <c:AnnotationType>SOURCE_INSTITUTIONS</c:AnnotationType> <c:AnnotationText xml:lang="fr">Eurostat</c:AnnotationText> <c:AnnotationText xml:lang="en">Eurostat</c:AnnotationText> <c:AnnotationText xml:lang="de">Eurostat</c:AnnotationText> </c:Annotation> <c:Annotation> <c:AnnotationTitle> <adms:identifier xmlns:adms="http://www.w3.org/ns/adms#" xmlns:skos="http://www.w3.org/2004/02/skos/core.html" xmlns:dct="http://purl.org/dc/terms/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <adms:Identifier rdf:about="https://doi.org/10.2908/ISOC_CI_ID_H"> <skos:notation rdf:datatype="http://purl.org/spar/datacite/doi">10.2908/ISOC_CI_ID_H</skos:notation> <dct:creator rdf:resource="http://publications.europa.eu/resource/authority/corporate-body/ESTAT"/> <dct:issued rdf:datatype="http://www.w3.org/2001/XMLSchema#date">2023-01-19T12:50:39</dct:issued> </adms:Identifier> </adms:identifier> </c:AnnotationTitle> <c:AnnotationType>DISSEMINATION_DOI_XML</c:AnnotationType> </c:Annotation> </c:Annotations> <c:Name xml:lang="de">Haushalte - Verfügbarkeit von Internet-Geräten</c:Name> <c:Name xml:lang="en">Households - devices to access the internet</c:Name> <c:Name xml:lang="fr">Ménages - dispositifs pour accéder à l'internet</c:Name> <s:Structure> <Ref id="ISOC_CI_ID_H" version="28.0" agencyID="ESTAT" package="datastructure" class="DataStructure"/> </s:Structure> </s:Dataflow> </s:Dataflows> </m:Structures> </m:Structure> |
...
From the reference present in the dataflow, it is possible to query for the corresponding SDMX Data Structure Definition
DSD Link | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/datastructure/ESTAT/ISOC_CI_ID_H/28.0 |
---|
These resources are versioned, so version present in the reference must be used to ensure consistency.
...
to the to the concept holding the dimension label
Tip icon false In current Eurostat Dissemination Chain, there is one DSD and one ConceptScheme generated for each dataset with identical identifier (but potentially different version).
to the code lists holding the code and labels for the dimension positions
These code lists are reference metadata and may contains more code and labels that the one used by a specific dataset.
To known the list of positions present in the dataset, please refer to the Content Constraint artefact (next section).
...
- the mandatory TIME_PERIOD period dimension where the value are expressed using ISO8601
- the primary measure OBS_VALUE holding the statistical value observation the
the optional value attribute OBS_FLAG hodling the statistical status (also refered as flags)
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
<?xml version='1.0' encoding='UTF-8'?> <m:Structure xmlns:m="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/message" xmlns:s="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/structure" xmlns:c="http://www.sdmx.org/resources/sdmxml/schemas/v2_1/common"> <m:Header> <m:ID>DS-ISOC_CI_ID_H1683705333393</m:ID> <m:Test>false</m:Test> <m:Prepared>2023-05-10T07:55:33.393Z</m:Prepared> <m:Sender id="ESTAT"> <c:Name xml:lang="de">Statistische Amt der Europäischen Union (Eurostat)</c:Name> <c:Name xml:lang="en">Statistical Office of the European Union (Eurostat)</c:Name> <c:Name xml:lang="fr">Office de statistique de l'Union européenne (Eurostat)</c:Name> </m:Sender> <m:Receiver id="unknown"/> </m:Header> <m:Structures> <s:DataStructures> <s:DataStructure agencyID="ESTAT" id="ISOC_CI_ID_H" isFinal="true" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DataStructure=ESTAT:ISOC_CI_ID_H(28.0)" version="28.0"> <c:Annotations> <c:Annotation> <c:AnnotationTitle>OBS_FLAG</c:AnnotationTitle> <c:AnnotationType>DISSEMINATION_FLAG_SETTINGS</c:AnnotationType> <c:AnnotationURL/> </c:Annotation> <c:Annotation> <c:AnnotationTitle>time</c:AnnotationTitle> <c:AnnotationType>DISSEMINATION_TIME_DIMENSION_CODE</c:AnnotationType> <c:AnnotationURL/> </c:Annotation> </c:Annotations> <c:Name xml:lang="en">ISOC_CI_ID_H data structure</c:Name> <s:DataStructureComponents> <s:DimensionList id="DimensionDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DimensionDescriptor=ESTAT:ISOC_CI_ID_H(28.0).DimensionDescriptor"> <s:Dimension id="freq" position="1" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).freq"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="freq" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:Enumeration> <Ref agencyID="ESTAT" class="Codelist" id="FREQ" package="codelist" version="1.31"/> </s:Enumeration> </s:LocalRepresentation> </s:Dimension> <s:Dimension id="indic_is" position="2" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).indic_is"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="indic_is" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:Enumeration> <Ref agencyID="ESTAT" class="Codelist" id="INDIC_IS" package="codelist" version="9.0"/> </s:Enumeration> </s:LocalRepresentation> </s:Dimension> <s:Dimension id="unit" position="3" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).unit"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="unit" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:Enumeration> <Ref agencyID="ESTAT" class="Codelist" id="UNIT" package="codelist" version="16.0"/> </s:Enumeration> </s:LocalRepresentation> </s:Dimension> <s:Dimension id="hhtyp" position="4" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).hhtyp"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="hhtyp" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:Enumeration> <Ref agencyID="ESTAT" class="Codelist" id="HHTYP" package="codelist" version="2.0"/> </s:Enumeration> </s:LocalRepresentation> </s:Dimension> <s:Dimension id="geo" position="5" urn="urn:sdmx:org.sdmx.infomodel.datastructure.Dimension=ESTAT:ISOC_CI_ID_H(28.0).geo"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="geo" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:Enumeration> <Ref agencyID="ESTAT" class="Codelist" id="GEO" package="codelist" version="14.0"/> </s:Enumeration> </s:LocalRepresentation> </s:Dimension> <s:TimeDimension id="TIME_PERIOD" position="6" urn="urn:sdmx:org.sdmx.infomodel.datastructure.TimeDimension=ESTAT:ISOC_CI_ID_H(28.0).TIME_PERIOD"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="TIME_PERIOD" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:TextFormat textType="ObservationalTimePeriod"/> </s:LocalRepresentation> </s:TimeDimension> </s:DimensionList> <s:AttributeList id="AttributeDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.AttributeDescriptor=ESTAT:ISOC_CI_ID_H(28.0).AttributeDescriptor"> <s:Attribute assignmentStatus="Conditional" id="OBS_FLAG" urn="urn:sdmx:org.sdmx.infomodel.datastructure.DataAttribute=ESTAT:ISOC_CI_ID_H(28.0).OBS_FLAG"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="OBS_FLAG" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:Enumeration> <Ref agencyID="ESTAT" class="Codelist" id="OBS_FLAG" package="codelist" version="1.31"/> </s:Enumeration> </s:LocalRepresentation> <s:AttributeRelationship> <s:PrimaryMeasure> <Ref id="OBS_VALUE"/> </s:PrimaryMeasure> </s:AttributeRelationship> </s:Attribute> </s:AttributeList> <s:MeasureList id="MeasureDescriptor" urn="urn:sdmx:org.sdmx.infomodel.datastructure.MeasureDescriptor=ESTAT:ISOC_CI_ID_H(28.0).MeasureDescriptor"> <s:PrimaryMeasure id="OBS_VALUE" urn="urn:sdmx:org.sdmx.infomodel.datastructure.PrimaryMeasure=ESTAT:ISOC_CI_ID_H(28.0).OBS_VALUE"> <s:ConceptIdentity> <Ref agencyID="ESTAT" class="Concept" id="OBS_VALUE" maintainableParentID="ISOC_CI_ID_H" maintainableParentVersion="28.0" package="conceptscheme"/> </s:ConceptIdentity> <s:LocalRepresentation> <s:TextFormat textType="Double"/> </s:LocalRepresentation> </s:PrimaryMeasure> </s:MeasureList> </s:DataStructureComponents> </s:DataStructure> </s:DataStructures> </m:Structures> </m:Structure> |
...
in the SDMX Content Constraint
Content Constraint Link | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/contentconstraint/ESTAT/ISOC_CI_ID_H/1.0 |
---|
In the API, SDMX Content Constraints artefacts are used to define available positions, it lists for each dimension code the list of position code that are used at least once to refer to a statistical observation.
...
Starting back from the Dataflow it is possible to include the referenced artefacts, at two different level
Scope | Link |
---|---|
Dataflow + DSD | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/ISOC_CI_ID_H/1.0?references=children |
Dataflow + DSD + CS and CL | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/ISOC_CI_ID_H/1.0?references=descendants |
Dataflow + DSD + CS and CL filtered on the constraints | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/dataflow/ESTAT/ISOC_CI_ID_H/1.0?references=descendants&detail=referencepartial |
Special case of Dataset listing
Instead of specifying a dataset code in the dataflow request the ALL keyword can be used to retrieve a list of all Eurostat datasets in one request
Special case of Metadata harvesting
Similarly to the request on ALL dataflows it is possible to get the latest version for all artefacts for a specified type
Scope | Link |
---|---|
All code lists | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/codelist/ESTAT/all |
All concept schemes | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/conceptscheme/ESTAT/all |
All data structure definitions | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/datastructure/ESTAT/all |
Data query
Tip |
---|
in SDMX 2.1, the data query directly use the dataflow identifier without agencyID |
Above link retrieves the complete dataset in default format : SDMX-ML 2.1 Generic Data.
...
- a blank means no filtering for this dimension
- Several values for a dimension must be separated by a '+' character
Scope | Details on the series-keys filter | Link | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Single time-series fully specified |
| https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/A.H_IPC.PC_HH_IACC.TOTAL.EA | ||||||||||||
EU27 and EA data | As the
| https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H/....EU27_2020+EA |
Filtering on time period
Filtering the observations to be returned based on their TIME_PERIOD value is controller via a FROM-TO filter with the query parameter startPeriod and endPeriod
...
An additional format parameter allows to request a response in a different semantic format
Format | Description | Link |
---|---|---|
SDMX-ML 2.1 Structured Data | More compact XML format | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H?format=SDMX_2.1_STRUCTURED |
SDMX-CSV 1.0 | SDMX standardized CSV format | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H?format=SDMX-CSV |
TSV | Eurostat specific format | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H?format=TSV |
JSONstat | JSON format usable with JSON-stat toolkit | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H?format=JSON |
Compression
An additional compressed parameter allows to optimize network transfer by retrieving the content compressed as GZIP
Compressed TSV data link | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/data/ISOC_CI_ID_H?format=TSV&compressed=true |
---|
Retrieving Navigation artefacts
...
- Category Scheme : Hierarchy of categories
- Categorisation : one categorisation is referencing one dataset into a category of a Category Scheme
Scope | Link |
---|---|
All category schemes | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/categoryscheme/ESTAT/all |
All categorisations | https://ec.europa.eu/eurostat/api/dissemination/sdmx/2.1/categorisation/ESTAT/all |