Met Office: Multi-model ensemble research

	THORPEX: Multi-model ensemble research

To forecast the weather, we first determine the current weather conditions across the entire globe. Then a computer model, based on mathematical equations that describe weather processes, is applied. However, there are errors in both the current conditions and in the computer model. Although these errors might be small, they accumulate during the weather forecasting process so that a weather forecast for 15 days ahead is likely to have large errors.

The aim of ensemble forecasting is to give a range of the possible outcomes, given our knowledge of the uncertainties in both the initial conditions and the model. Typically, the ensemble is produced by using the same forecast model, but with a range of slightly different initial conditions. This method does not explicitly account for the errors in the forecast model, leading to the development of forecast biases (errors in the average behaviour of the model). For example, the temperature forecasts might be always slightly too warm but with the correct variation over time. By subtracting a small value from the temperature forecast values, we can gain more accuracy; this adjustment towards reality is known as calibration.

Figure 1. Diagram of the moving window procedure. The diagonal arrows show forecasts starting at different times. For example, the red arrow shows a forecast that starts at T+0 and finishes at T+15. The blue box indicates the forecasts values that can be used with the truth to calculate the calibration parameters.

TIGGE has allowed access to ensembles from different forecast centres, meaning that ensembles can be combined to create a multi-model ensemble. A key question is whether a multi-model ensemble can give significant improvements in comparison to a calibrated single model ensemble.

To address this question, the Met Office is a developing a calibrated multi-model ensemble. The calibration parameters are calculated using a moving window, as illustrated in Figure 1. This means that the set of most recent forecast-observation pairs is used to calibrate the new forecast. For example, the eight forecasts at a lead time of F+3 from a range of verification times (shown by the blue dots) are used to calibrate the new forecast valid at T+3. However, for a forecast at T+15, the most recent observation is over 15 days ago, making it harder to calibrate. For the Met Office multi-model, we are using a multi-model analysis as the 'truth' and an exponential-moving-average so that the most recent data is given the most weight.

A further question is whether the multi-model ensemble can be improved by giving each model a different weight. To give preliminary results, idealized stuides have been made using two models based on the Lorenz 1963 model but with added model errors. Three methods have been tested: skill based (using the mean-square-error of the ensemble-mean), regression-based (using multiple-regression) and Bayesian model averaging (BMA). The results (Figure 2) show that all three methods pick out a seasonal variation in the weights over the two-year period and that the weights also vary with lead time. Interestingly, although the methods approach the problem in very different ways, they all give very similar results. The Met Office is currently developing a real-time multi-model ensemble with skill-based weights. This will allows us to test whether model-dependent weights can also give improvements in a real ensemble.

Figure 2. Model-dependent weights for one of the two idealized models, shown as a function of lead time and verification time, as calculated by three different methods. The red values indicate a large weight and the blue values indicate a small weight.