UK public confidence in AI weather forecasts lags behind existing methods, Met Office study finds
New Met Office research, thought to be the first of its kind, identifies a confidence gap between existing forecasting methods and emerging AI techniques, helping inform how future innovations are introduced.
The study, published in Artificial Intelligence for the Earth Systems, investigates public confidence in using AI forecasts, as well as perceptions of accuracy of AI weather forecasts and highlights the need for public engagement if potential benefits of AI-enhanced weather forecasts are to be fully realised.
AI is likely to play a significant role in future forecasting. The Met Office currently produces forecasts using physics-based modelling and is actively looking at how these can be augmented by AI models. This future blend of approaches could accelerate improvements to speed and accuracy, so the identification of concerns over public confidence using AI forecasts is an important step for enabling future innovation.
The survey of over 6,000 UK adults asked respondents to rate their confidence in using different types of forecasts for everyday decisions as well as high-stakes decisions. They were also asked to assess the perceived accuracy of both existing physics-based methods and (potential) AI-generated forecasts for daily and severe weather.
Greater uncertainty around the use of AI forecasts for weather-related decision making was highlighted. For physics-based models, around 80% said they would be confident in making weather-related decisions, for both daily and important events, while this dropped to around 40% for potential AI forecasts.
This confidence gap was reinforced by the public’s perception of accuracy. Around 78% rated physics-based models as accurate for both daily and severe weather forecasts, compared to around 45% for forecasts that could be generated using AI techniques in the future.
Met Office Science Fellow and lead author of the paper Dr Edward Pope said: “This paper deepens our understanding of current public perceptions as we approach a crucial juncture where AI-based weather models are demonstrating their potential to work alongside physics-based methods. This was a unique opportunity to compare public perceptions of established and emerging approaches to forecasting the weather."
“The gap in confidence and perceived accuracy highlighted in the paper demonstrates the need to clearly and transparently demonstrate the value of new approaches in ways that matter to people. AI techniques are likely to play a significant part in the next generation of weather forecasts, but these improvements will only be fully realised if the public continue to have confidence, and importantly act on, the weather forecast they see.”
Trust is fundamental to the Met Office's work and has been highlighted over the impactful heatwaves of recent months. For the Red Extreme Heat warning in June this year, post-event surveys revealed that 96% of people in the area were aware of it, and 97% of those took action as a result.
Taking action relies on being confident using forecasts for making decisions, which, in turn, is partly underpinned by the perceived accuracy of forecasts for events that affect people. This is why the new research considered those aspects in detail.
Next generation of forecasts
As the UK’s national meteorological service, the Met Office has a vital role to ensure any potential improvements in forecasts are done in a reliable and safe way that improves the accuracy and value of a forecast, including the use of AI and ML techniques. While solely AI models aren't used in current operational output from the Met Office, machine learning techniques have long been used in the forecasting process. The potential positive influence of future innovative AI techniques is increasingly being demonstrated in the scientific community.
Met Office Chief AI Officer Professor Kirstine Dale said: “We are exploring ways of blending physics-based and AI-based modelling to deliver the forecasts that we all rely on. As with all science developments, we will robustly evaluate and validate any changes to our approach to weather forecasting before we introduce them into the model.
“This stringent, and well established, evaluation process builds and maintains public trust and confidence when using the weather model to make decisions. Maintaining public confidence is critical and this study provides us with valuable insight into what’s needed to ensure that we fully realise the benefits of the integrated approach.
“At the Met Office we will continue to push forward the frontiers of weather and climate intelligence so that we can serve the public and will continue to publish peer-reviewed articles, such as this one, to ensure that we move forward as a community.”
The research was supported by Public Weather Service and overseen by the Department of Business, Innovation, Science and Trade of the UK government. It is part of the Met Office’s innovative approach to enhancing the power of weather intelligence, ensuring that the public, customers and government can get more accurate forecasts using the best blend of AI and physics modelling.
Such is the importance of the public’s confidence using forecasts, similar surveys are planned in the coming years to understand changing perceptions and help shape progress, ensuring as many people as possible continue to make the best decisions possible around the weather forecast.
READ MORE: What people think about AI weather forecasts, and what we can do about it.