Playing the game of uncertainty
13 April 2012
Ken Mylne, Ensemble Forecasting Manager at the Met Office, finds the public is not confused.
Predicting how the weather will behave is a complex undertaking. Each timescale presents its own challenges. In an ideal world, everyone would like us to tell them exactly what the weather will do so they can make definite decisions in their local area.
Nature, however, doesn't work like that. But one thing we can do is tell people about the probabilities of things happening. At the Met Office, we have recently played a game which will help us communicate these probabilities effectively.
In the short range, it is possible to predict with a high degree of accuracy what the weather has in store. However, the science does not exist to allow precise prediction of local details five days ahead. When severe weather may be on the way, it may be very unpredictable even in the short term. In these situations, uncertainty becomes increasingly important in weather forecasting.
The science does not exist to allow precise prediction of local details five days ahead. When severe weather may be on the way, it may be very unpredictable even in the short term. In these situations, uncertainty becomes increasingly important in weather forecasting.
To forecast the weather we first gather observations from around the world to measure what the atmosphere is doing. We use these observations to set up a computer model of the atmosphere that represents its current state. The model then calculates how the atmosphere will evolve over the coming days. Unfortunately, due to chaos, small uncertainties in our observed atmosphere can grow rapidly to give large uncertainties in the forecast.
Over the last 15 years, the Met Office has developed sophisticated techniques to understand these uncertainties, called ensemble forecasts. This means we run the model many times instead of just once, from very slightly different starting conditions. The range of different outcomes gives us a measure of how confident we should be in the overall forecast. On some occasions the uncertainty is quite small and we can be confident - other times much less so. This can help decision makers manage the risks associated with the weather.
Using ensembles is good science but it also presents us with a problem. How do we effectively communicate the forecast, both the bits we can be confident about and the areas where we are less confident? How can we communicate probabilities?
The weather game
To try and answer this question, we recently ran an online weather game in conjunction with the Universities of Bristol and Cambridge. The game was played more than 11,000 times and became the largest research project of its kind ever carried out.
The game saw players help Brad, the ice cream man, run his business by deciding where and when he should sell his ice cream over a four-week period, depending on the weather. Different players were given different presentations of forecasts. These provided simpler or more complex information on the uncertainty in the forecast, using graphics or as written probabilities, or a combination of the two. The differences enabled us to see how effective these presentations were.
Initial results indicate some interesting trends. They show that, when faced with straightforward decisions, providing probabilities doesn't confuse people. For more complex situations, on average people are able to make better decisions using probabilities. And people make the best decisions when they are given more detailed information on forecast uncertainty.
These results only represent the tip of the iceberg. The data is still being analysed and we hope to see a fuller set later this year.
Communication of uncertainty is an issue well beyond tomorrow's weather forecast. The Met Office is also interested in how best to communicate the uncertainty in long-range weather forecasts, and in climate change predictions up to 100 years ahead. This is still work in progress, but the results of the weather game will help us to advance all these areas.
- This article was first published in 'People & Science', the British Science Association's magazine www.britishscienceassociation.org/ps