Next-day CSEP forecasts: Establishing a Benchmark

Fabio Silva, Francesco Serafini, José A. Bayona, Maximilian J. Werner, Philip J. Maechling, Kevin R. Milner, & William H. Savran

Published September 8, 2024, SCEC Contribution #14053, 2024 SCEC Annual Meeting Poster #215 (PDF)

Seismicity forecasting models are now routinely used for Operational Earthquake Forecasting (OEF) operations by institutions and state agencies. They provide information on the spatio-temporal evolution of seismicity and increase awareness to disaster responders and decision-makers. Forecasts used for these tasks should be rigorously and fairly tested against observations to quantify the ability of the underlying model to describe seismicity. The evaluation should be carried out in a prospective or pseudo-prospective fashion to ensure that the models are tested in situations resembling reality. The Collaboratory for the Study of Earthquake Predictability (CSEP) developed and operated twenty-seven different models producing daily forecasts between 2007 and 2018 producing an unprecedented database of almost 50,000 prospective oneday seismicity forecasts. The forecasting period includes over 600 earthquake with magnitude above 3.95, the most relevant being the 2010 Mw 6.5 Ferndale, 2010 Mw 7.2 El Mayor-Cucapah, and 2014 Mw 6.0 South Napa seismic sequences. This not only provides the opportunity to assess the ability of the models to forecast future seismicity but also to create a benchmark for future forecasting models for cross-comparison. To this end, we have made the forecasts accessible to the public through the CSEP website and provided code to evaluate the consistency between forecasts and observations and compare the performance of alternative forecasts. This will allow future models to have a benchmark problem to be evaluated with, and benchmark models to be compared against, therefore increasing the possibility of cross comparing different future models using a common alternative. The code we provide works in conjunction with PyCSEP thus leveraging the work that the CSEP community has already done in developing statistical tests to evaluate consistency and perform comparisons, thus providing a homogeneous theoretical framework and software with which evaluate forecasting models for seismicity. Lastly, we compare the CSEP models’ forecasts with daily forecasts obtained using the UCERF3-ETAS model.

Key Words
Open data, CSEP, seismicity forecast evaluation

Citation
Silva, F., Serafini, F., Bayona, J. A., Werner, M. J., Maechling, P. J., Milner, K. R., & Savran, W. H. (2024, 09). Next-day CSEP forecasts: Establishing a Benchmark. Poster Presentation at 2024 SCEC Annual Meeting.

Related Projects & Working Groups
Earthquake Forecasting and Predictability (EFP)