Ever increasing volumes of data are providing a challenge to traditional delivery methods. The Met Office is developing alternative solutions to meet the diverse needs of our users.
From its earliest beginnings over 150 years ago, our weather forecasting was driven by data. Simple observations recorded and used to hand plot synoptic charts have now been exchanged for the 20 million observations we use every day, mainly by satellite but also from weather stations, radar, ocean buoys, planes, shipping and the public.
As this observational data is received it is constantly and rigorously assimilated and tested before it is fed into our suite of models. The expertise of hundreds of scientists, operational meteorologists, mathematicians and collaboration partners throughout the world are essential to ensuring the integrity of the ingressed data.
The 335 million observations of data we store every day require huge computational capability and the Met Office supercomputer provides the storage and processing power needed to manipulate the data in a timely and effective manner. In turn our models create enormous data outputs, which are used for climate and weather prediction and by data users throughout the world to make weather outlooks more accurate than ever before.
But the huge amount of data we collect, model and store continues to increase exponentially, and this is our ‘Big data challenge’. It’s increasingly important to ensure that our data is available in reusable and inter-operable formats and that it can be delivered quickly and efficiently in the most relevant and innovative form to our customers.
We want to ensure our data is discoverable and accessible using industry open standards. This will enable our customers to choose the data they want, in the form they want and at the speed and service level they require to research products and services. This is the challenge we are undertaking.
A new white paper published by the Open Data Institute (ODI) in collaboration with the Met Office explores the evolution in the ways in which weather and climate data are accessed, used and shared.
Weather data infrastructure consists of a combination of data assets, technology, processes and organisations and this report reviews its importance and the steps in its value chain.