> Home >

Daily Land Surface Temperature Benchmarks

These benchmarks have been designed to test homogenisation algorithms on daily mean temperature from observing stations. The aims of this work are threefold:

to test our ability to homogenise surface temperature data at the daily scale
to test our ability to build sufficiently realistic benchmarks at the daily scale
to provide a tool for homogenisation algorithm development at the daily scale

Four regions of the USA are available: Wyoming, the south east, the north east and the south west. Three different error worlds have been designed, exploring a range of inhomogeneity and data characteristics. A fourth world will shortly be available (for Wyoming only) exploring choices within the benchmark data creation model.

The data are designed to be as realistic as possible in terms of temporal and spatial variability and correlation. They cover the period from January 1st 1970 to December 31st 2011. The data were first created free from any inhomogeneity such that all inhomogeneities present have been intentionally added and are therefore known apriori. A perfect homogenisation algorithm is expected to return the inhomogeneous error worlds back to their 'clean' form.

Although the formal benchmark testing is now complete, anyone interested in testing their homogenisation algorithm is encouraged to use the data. The error worlds and clean worlds are available from the download page. Assessment measures and software will be made available at a later date. Assessment will focus on the changepoint detection ability and ability to adjust the error worlds back to their clean form. Formal results should be made available in Spring 2015. We encourage any other users of the data to contact Rachel Warren and welcome addtional results.

This work is funded by EPSRC with CASE sponsorship from the Met Office

Instructions for taking part in the benchmarking:

There are four focus regions for this project:

Wyoming: Latitude: (41°N to 45°N) and Longitude: (-111°W to -104.2°W)
- World 1 and 4: 75 stations
- Worlds 2 and 3: 158 stations
South East: Latitude: (25.1°N to 33°N) and Longitude: (-90°W to -79.7°W)
- World 1 and 4: 153 stations
- Worlds 2 and 3: 210 stations
North East: Latitude: (41.4°N to 47.3°N) and Longitude: (-79.8°W to -67.3°W)
- World 1 and 4: 146 stations
- Worlds 2 and 3: 207 stations
South West: Latitude: (32.6°N to 38.5°N) and Longitude: (-123°W to -113.9°W)
- World 1 and 4: 151 stations
- Worlds 2 and 3: 222 stations

If possible please homogenise all stations in all regions and for all worlds. If this is not possible, please prioritise by homogenising all worlds for a one region rather than all regions for one world. The preferred priroritisation is as follows:

Wyoming: World 1, World 3, World 2, World 4
South East: World 1, World 3, World 2
North East: World 1, World 3, World 2
South West: World 1, World 3, World 2

Please download the data from the download page. After performing homogenisation, please return the homogenisation regions/worlds, preferably in the same format, to Rachel Warren. Please also provide a list of locations of all changepoints found and the size (and shape if possible) of any inhomogeneity adjustments applied.

Electronic Appendices: Code and Algorithm Performance Summaries

Download the R code to create loess smooth surfaces that you can predict from here.

Download the R code to create a compiled series here.

Download a summary of ACMANT here.

Download a summary of Climatol Daily here.

Download a summary of Climatol Monthly here.

Download a summary of MAC-D (only applied to the Wyoming scenarios) here.

Download a summary of MASH here.

Download a summary of DAP, HOM and SPLIDHOM's performance in the North East here.

Download a summary of DAP, HOM and SPLIDHOM's performance in the South East here.

Download a summary of DAP, HOM and SPLIDHOM's performance in the South West here.

Download a summary of DAP, HOM and SPLIDHOM's performance in Wyoming here.

You can download tables that compare the performance of all algorithms on a regional and station by station basis here.

Keep in touch

For more detailed information, follow our Benchmarking Blog blog

This work is part of the International Surface Temperature Initiative (ISTI): http://www.surfacetemperatures.org/benchmarking-and-assessment-working-group

Benchmark Stations for Four Regions in the USA

Data are available from the download page without charge for the purposes of private study and scientific research, but please read the terms and conditions.

Commercial and media enquiries

You can access the Met Office Customer Centre, any time of the day or night by phone, fax or e-mail. Trained staff will help you find the information or products that are right for you.
Contact the Met Office Customer Centre

References

When using the dataset in a paper, the following is the correct citation to use.

R. E. Killick, 2016: Benchmarking the Performance of Homogenisation Algroithms on Daily Temperature Data, University of Exeter, PhD Thesis. PDF available for download
here.

Dataset and Diagnostic Creation Code

The R code used (excluding quality control, homogenisation and regional average uncertainty estimates) was written by Rachel Warren and can be found here. We do not intend to provide in-depth support for this code. We do appreciate constructive feedback on this code.

Dataset produced in collaboration with: