World Weather Research Programme (WWRP)
Forecast Verification Research
- To plan and implement the verification component of WWRP;
- To serve as a focal point for the development and dissemination of new verification methods for WWRP and EPAC, as required;
- To facilitate and encourage training and dissemination of information on verification methodologies;
- To take into account the needs of users so as to ensure the relevance of the practice of forecast verification;
- To facilitate the development and application of improved diagnostic verification methods to assess and enable improvement of the quality of weather forecasts, including forecasts from numerical weather and climate models;
- To encourage the sharing of observational data for verification purposes;
- To encourage greater awareness in the research community of the importance of verification as a vital part of numerical and field experiments rather than an "afterthought";
- To promote collaboration among scientists conducting research on various aspects of forecast verification as well as with model developers and forecast providers; and
- To collaborate on forecast verification with the Working Group on Numerical Experimentation (WGNE) and WCRP and in coordination with CBS.
Forecast Verification Session at the EMS 2017 Annual Conference (European Conference for Applied Meteorology and Climatology, Dublin, Ireland, 4-8 Sep 2017)
Deadline for abstracts (21 April 2017)
more information is available at:
winner of the 2017 Forecast Verification Metric Challenge
The Joint Working Group on Forecast Verification Research (JWGFVR) of WMO's World Weather Research Programme is very pleased to announce that the winner of the Forecast Verification Metric Challenge launched in September 2015
is Helge Goessling from the Alfred Wegener Institute in Germany with his entry "Integrated Ice Edge Error (IIEE) & Spatial Probability Score (SPS)". He has been invited to give a keynote talk at the 7th International Verification Methods Workshop in Berlin, 8-11 May 2017.
Launched in September 2015 by the JWGFVR to contest aimed to encourage the development of verification approaches targeted to forecast users: specifically to develop and demonstrate the best new user-oriented forecast verification metric. A total of 17 entries from 11 countries were received by 13 November 2016.
Second place went to Dominique Brunet from Environment Canada with "A spatio-temporal user-centric distance", applied to lightning probability forecasts at an airport.
In joint third place are Thomas Nipen and Ivar Seierstad from the Norwegian Meteorological Institute on the "Rain-free window accuracy", and William Wang, Andrew Watkins and David Jones from the Australian Bureau of Meteorology with their "Weighted percent consistence" applied to seasonal outlooks.
The challenge supports the WWRP's High Impact Weather (HIWeather), Subseasonal to Seasonal Prediction (S2S), and Polar Prediction (PPP) projects. The WWRP family warmly congratulates all the winners and we look forward to seeing you all in Berlin this May.
The Sixth International Verification Methods Workshop was successfully held in New Delhi, India from 13-19 March 2014. The goal of the workshop is to promote recent advances in the theory and practice of verification of weather and climate forecasts worldwide. The workshop welcomed participants from operational, research, and forecast user communities to discuss methods for more effectively measuring and conveying the accuracy and utility of forecasts and warnings.
The workshop included both a tutorial session (March 13-15) and a scientific program (March 16-19). The scientific program featured keynote addresses as well as contributed talks and posters on new verification techniques and issues related to the practice of forecast verification.
Forecast Verification Method Inter-comparison Project (ICP): Several members of the Joint Working Group on Forecast Verification Research have participated in the ICP, which is an effort to provide useful information on new verification approaches that have been developed in the last few years. The initial focus has been on spatial verification aproaches including object based methods, scale separation approaches, neighborhood methods and field morphing approaches. The methods were all applied to the same datasets, including some artificial cases that allow easy interpretation, as well as some high resolution model forecasts from the central US. The results of these studies are now available for early release as a special collection in the journal Weather and Forecasting. At a recent informal workshop for this project plans were developed to extend the project to new datasets and variables. This project is a significant step toward development of a verification method testbed, which would be supported by this Joint Working Group.
Projects and Activities
- Special Issue of Meteorological Applications on Forecast Verification
- Verification guidance and support for WWRP forecast and research demonstration projects (e.g. MAP D-PHASE, Beijing Olympics)
- Documentation of recommended methods for specific applications (e.g. precipitation)
- Workshops on new verification methods;
- Education and outreach, including web tools and tutorials on verification methods;
- Development of a Real Time Forecast Verification (RTFV) system;
- User-focused verification approaches and
- Methods intercomparisons
co-Chair: Marion MITTERMAIER (Met Office)
Lawrence WILSON (ECCC)
Barbara BROWN (NCAR)
Barbara CASATI (ECCC)
CHEN Jing (CMA)
Caio Augusto dos Santos COELHO (CPTEC)
Manfred DORNINGER (UA)
Thomas HAIDEN (ECMWF)
Stephanie LANDMAN (SAWS)
Chiara MARSIGLI (ARPA)
Yuejian ZHU (NCEP/NOAA)
Annual Meeting of the JWGFVR (Bologna, Italy, 19-21 September 2016)
Annual Meeting of the JWGFVR (Sofia, Bulgaria, 9-10 September 2015) Meeting Notes
Annual Meeting of the Joint (WWRP/WGNE) Working Group on Forecast Verification Research (Montreal, Canada, 22-23 August 2014) Meeting Notes
Annual Meeting of the Joint (WWRP/WGNE) Working Group on Forecast Verification Research (Macao, China 24-25 October 2013)
Joint Meeting of JWGFVR and WG TMR (Macao, China 26 October 2013)
Annual Meeting of the Joint (WWRP/WGNE) Working Group on Forecast Verification Research (Berlin, 8 to 10 September 2011) Minutes & Presentations
Joint Meeting of the JWGFVR and WG SERA (Toulouse, France, 27-29 September 2010) Agenda, Presentations with Action Sheet and Meeting Notes
Roving Seminar on Verification (Jakarta, 17-19 November 2015)
Sixth International Verification Methods Workshop (New Delhi, 13-19 March 2014) Scientific/Tutorial
Fifth International Verification Methods Workshop (Melbourne, 1-7 December 2011) Minutes of JWGFVR post-workshop meeting
Special session on Forecast Verification AM5 " Verification and quality of probabilistic forecast guidance" at the ECAM/EMS Conference (Berlin, 12-16 September 2011)
Hands on weather forecast verification training by WWRP JWGFVR member Pertti Nurmi (FMI) under the bilateral Finnish-Peruvian project for increased capacity building of Servicio Nacional de Meteorologia e Hidrologia del Peru (SENAMHI) Lima, Peru (22 November to 2 December 2010)
Summer Colloquim on Forecast Verification (NCAR Boulder CO USA, 6-28 Jun 2010)
Verification methods for tropical cyclone forecasts (2013-7)
Recommended Methods for Evaluating Clouds and Related Parameters
Proceedings of the Fourth International Verification Methods Workshop (Helsinki, Finland, 8-10 June 2009) (WMO/TD-No.1540 WWRP 2010-1)
Recommendations for the Verification and Intercomparison of QPFs and PQPFs from Operational NWP Models - Revision 2 October 2008 (WMO/TD-No.1485 WWRP 2009-1)
Scientific publications of interest
Special Issue of Meteorological Applications
on Forecast Verification (Volume 15,Issue1)
Report BROKEN LINKS or
Flyer of the JWGFVR
Click here to learn more about forecast verification methods (BOM website)
Alternatively, click here for the pdf version
Forecast Verification - issues, methods and FAQs (Under Construction)