Validation is a key task performed in all statistical domains. Efficient data validation is essential for high quality statistics. Guidelines for assigning validation responsibilities within the whole production chain, standard validation levels, a good selection of validation rules, standards for validation reports and error/warning messages and common documentation standards of the validation process are important elements of a good data validation policy. In principle, all data validation processes share a common approach, shown in the diagram below.
Figure 52 – Validation
The data are checked in successive steps:
- the structure of the data set
- the internal relationship between fields (validation rules)
- the raw aggregated results (control tables)
- the cross check with other agricultural statistics
- the cross check with farm structure data from previous years.
Only data having been validated at the last step can be disseminated (note that microdata is never disseminated). Afterwards some errors may be detected during a specific analysis and the data set revised. The consistency of different fields within a single record is checked against the validation rules given in Annex V. Violation of a validation rule does not in all cases imply that data are invalid. Some rules might only highlight cases which could merit further investigation. Data suppliers are asked to apply the rules before transmitting data to Eurostat. Eurostat promotes the discussion of the validation rules in the Working Group. In this occasion, Member States are invited to ask Eurostat to modify the rules in case these rules do not adequately describe the situation in their country.