This space aims to provide information on relevant initiatives on big data in the context of official statistics.

Join this group to stay updated on:

• strategic documents  e.g. the ESS Big Data Action Plan and Roadmap 1.
• relevant resources
  • introductory material
  • training courses
  • conference papers on Big Data
• information on initiatives
  • projects in the ESS
  • European and international meetings on Big Data

Strategic document "What does big data mean for official statistics?"

In 2010 the Bureau of the Conference of European Statisticians set up the High-Level Group for the Modernisation of Statistical Production and Services (HLG) to oversee and coordinate international work relating to the development of enterprise architectures within statistical organisations.

The High-Level Group established a Task Force of national and international experts, coordinated by the UNECE Secretariat, with the requirement to produce "a document explaining the issues surrounding the use of big data in the official statistics community". The Task Force released the document 'What does Big Data mean for Official Statistics?'. It is a strategic paper which discusses definitions and sources, identifies the main challenges with regard to legislation, privacy, financial issues, management, methodology and technology and provides some basic recommendations for national and international statistical bodies. 

Project initiatives

Project proposal for official statistics

The UNECE, in partnership with Eurostat and the OECD, organizes annual meetings on the management of statistical information systems (MSIS). The 2013 MSIS meeting decided that Big data is a key issue for official statistics.

As at 5 May 2013, a temporary task team was set up to work out the key issues with using Big Data for official statistics, identify priority actions and formulate a project proposal. The team formulated a draft project proposal to effectively address some of the challenges identified in the strategic paper mentioned above. The team also formulated a classification scheme for Big Data sources with identified attributes.

Events

Big Data meeting at the DGINS 2013

One of the topic of the 2013 DGINS conference was 'Big Data'. The presentations of the Big Data seminar are publicly available. 

DGINS participants signed a Memorandum which paves the way for the future cooperation between members of the ESS in the collection of large and complex datasets.

Big Data event in 2014

The European Commission (Eurostat) is organizing an event on Big Data on 31 March-1 April 2014. This event aims to increase the awareness of the Big Data issue in the ESS, to identify best practices and synergies where joint efforts might serve the interest of more NSIs and/or the entire ESS. More information is available here.

Relevant document and other resources

Reflections on the use of Big Data for statistical production  (Pilar Rey del Castillo, Eurostat)

Let's Open Big Data! (Pasi Piela, Statistics Finland)

Knowledge Linking for Online Statistics (Marc Spaniol, Natalia Prytkova and Gerhard Weikum, Max-Planck-Institut für Informatik, Germany)

Big Data for official statistics - strategies and some initial European applications (Martin Karlberg and Michail Skaliotis, Eurostat)

You can also find links to papers on big data for official statistics on the  UNECE page on Big Data .

Online training course

 "Introduction to Data Science" organized by University of Washington.