Benefits of high resolution ensemble forecasts
July 2014 - Recent case study analysis and verification of probabilistic forecasts from the Met Office ensemble system shows benefit from the use of high horizontal resolution when forecasting heavy precipitation.
Probabilistic forecasts of heavy precipitation over the UK have been compared from two configurations of the Met Office Global and Regional Ensemble Prediction System (MOGREPS). These are a high resolution convection permitting UK ensemble (MOGREPS-UK) and a coarser resolution global ensemble (MOGREPS-G).
Case study analysis and objective verification provides an insight into the benefits of higher resolution forecasts. The basic configurations of MOGREPS used in this comparison are outlined below.
Met Office Global and Regional Ensemble Prediction System (MOGREPS)
For the data used in this article, the global version (MOGREPS-G) had 12 members running four times daily at 00, 06, 12 and 18 UTC out to 72 hours (the forecast lead time was extended out to 7 days during July 2014). It has a horizontal resolution of approximately 33km in mid latitudes.
Most MOGREPS-G forecasts (including those verified here) are generated by time-lagging the latest two runs to create a 24 member ensemble out to 66 hours (or 7 days in the July 2016 update). Time lagging helps reduce jumpiness between forecasts and can improve forecast reliability.
The regional version (MOGREPS-UK) runs with a UK domain. It has 12 members with a convection permitting horizontal resolution of 2.2km. The lead time is 36 hours with four runs daily at 03, 09, 15 and 21 UTC. Boundary conditions for each member are taken from MOGREPS-G.
Most MOGREPS-UK forecasts (including those verified here) use neighbourhood processing. Neighbourhood processing is a technique which increases the effective ensemble size by calculating probabilities at a location using not just the local grid-point, but also grid-points within a defined distance from the location.
For the latest (up to date) model configurations see the two links below:
Case study - 4/5 July 2014
This case study directly compares probability forecasts from MOGREPS-G (left map) to MOGREPS-UK (right map). Both maps show the probability of 6 hour precipitation exceeding 10mm (valid between 00 UTC and 06 UTC on 5 July 2014) in colour shading. MOGREPS-G probabilities are derived from the number of members exceeding 10mm at each grid-point and MOGREPS-UK probabilities are derived by neighbourhood processing.
These two days in July 2014 saw a series of closely packed weather fronts, associated with an area of low pressure to the north of Scotland, bring a spell of rain to most areas. The rain moved south-east across the UK, affecting north-western parts during the morning of 4 July 2014 and clearing the south-east during the morning of 5 July 2014.
MOGREPS-UK shows higher probabilities over the higher ground of North Wales, the Pennines and North Yorkshire, showing how the orography is enhancing the rainfall totals in these areas. The higher resolution of the model may also allow the frontal structure to be better resolved and hence enhance the precipitation totals.
MOGREPS-UK is also able to resolve some heavy convective showers off the west coast of Scotland and Ireland, shown by the low probability (green) contouring in this area. These showers are also shown in the verifying radar images below. The areas affected by the showers (radar) are very small compared to the area at risk (forecast), therefore corresponding to the low probabilities of <10% indicated in the forecast chart.
This one case study shows benefits from the use of the higher resolution MOGREPS-UK in the short range (1 to 2 day period). However, objective verification over many cases is needed to make any informative conclusions.
MOGREPS-UK and MOGREPS-G forecasts out to a lead time of 36 hours have been verified over a 12 month period (18 April 2013 to 17 April 2014) with the use of a forecast tool known as EPS-W (referred to as "MOGREPS-W" in Neal et. al. 2013). EPS-W post-processes ensemble data from MOGREPS and ECMWF using criteria from the NSWWS. The tool is designed to provide Met Office meteorologists advanced notice of upcoming severe weather forecast by the ensemble members, for which they may need to consider issuing warnings.
Due to sampling limitations associated with severe (and rare) events, a range of precipitation accumulation thresholds have been verified. These range from the moderate threshold of 10mm in 3 hours, to the more severe thresholds of 15 and 20mm in 3 hours. Results are only shown for 10mm events as higher thresholds have a very small sample size making it difficult to make any definitive conclusions.
Verification is done on the UK county and unitary authority basis, with there being 147 such areas in the UK. EPS-W derives probabilities for an individual area by taking the 95th percentile of grid-point probabilities in that area. This means that a forecast probability applies to ≥ 5% of the area. Forecasts are placed into one of 12 evenly distributed probability bins (with an additional bin for all 0% forecasts) before verification takes place.
Forecasts have been verified against a Met Office 2km resolution analyses. An event is considered to have occurred in a county or unitary authority if ≥ 5% of the 2km resolution analysis grid-points meet or exceed the warning threshold in that area.
Plotting observed frequency against forecast probability for a set of probability forecasts creates a reliability diagram (top plot), where reliability is indicated by proximity of the plotted line to the diagonal. Lines below the diagonal indicate over-forecasting and lines above the diagonal indicate under-forecasting. Sharpness is described by the number of samples in each probability bin.
Results are presented for all lead times combined. The reliability diagram (top plot) shows that MOGREPS-UK has better forecast reliability than MOGREPS-G, with the solid line being closest to the diagonal overall. This was also the case for the higher precipitation thresholds of 15mm and 20mm, although as has already been mentioned, the sample size at these thresholds is too low to be confident about the results.
Both MOGREPS-UK and MOGREPS-G show an over-forecasting signal, which increases for the higher precipitation thresholds. However, overall MOGREPS-UK reliability for 10mm in 3 hours is very good given that it is still a relatively high precipitation accumulation threshold and it shows improved forecast reliability over the coarser resolution MOGREPS-G.
The sample sizes are typical for severe (and rare) events with there being plenty of low probability forecasts and very few medium and high probability forecasts. MOGREPS-UK has a larger number of samples in the lower probability bins compared to MOGREPS-G (see middle and bottom plots), which may be a result of neighbourhood processing.
MOGREPS-G has a slight rise in samples for the very highest probability bin. A breakdown of samples with forecast lead time shows that these higher probability forecasts tend to occur at shorter lead times when the model is more confident and ensemble spread is lower.
A similar analysis of wind forecasts has been carried out and also shows useful skill from the high resolution model.