Evaluation of a decade-long prospective earthquake forecasting experiment in Italy
Pablo C. Iturrieta, José A. Bayona, Maximilian J. Werner, Danijel Schorlemmer, Matteo Taroni, Giuseppe Falcone, Fabrice Cotton, Khawaja M. Asim, William H. Savran, & Warner MarzocchiPublished April 12, 2024, SCEC Contribution #12738
Earthquake forecasting models represent our current understanding of the physics and statistics that govern earthquake occurrence processes. Providing such forecasts as falsifiable statements can help us assess a model's hypothesis to be, at the least, a plausible conjecture to explain the observations. Prospective testing (i.e., with future data, once the model and experiment have been fully specified) is fundamental in science because it allows us to confront a model with completely out-of-sample data and zero degrees of freedom. Testing can also help inform decisions regarding the rejection of models, data types or procedures in practical applications, such as Probabilistic Seismic Hazard Analysis and Operational Earthquake Forecasting. In 2010, a 10-year earthquake forecasting experiment began in Italy, where modelers collectively agreed on authoritative data sources, testing rules, and formats to independently evaluate a collection of forecasting models submitted by multiple researchers. In this study, we test the Italy Experiment forecasts with ten years of fully prospective data using a multi-score approach to (i) study the model features that cause a forecast to be consistent or inconsistent with the observations, (ii) evaluate the stability of the experiment's results over time, and (iii) quantify the spatial limitations of the models to provide forecasts consistent with the observations. Our results show that the best-performing models use catalogs that span over 100 years and incorporate fault information, indicating that these data types should not be overlooked in the future. The experiment's results are stable over time in terms of the models ranking, suggesting a 10-year window can work as a first order approximation to discriminate between optimal and sub-optimal forecasts. Finally, no forecast can reproduce earthquakes' spatial clustering adequately, but models including fault information have a lower degree of dissimilarity with the observations. We stress the value of prospective testing to identify relevant assumptions and hypothesis of earthquake processes, thus providing guidelines to develop new models and better inform the decisions that improve our earthquake resilience as society.
Citation
Iturrieta, P. C., Bayona, J. A., Werner, M. J., Schorlemmer, D., Taroni, M., Falcone, G., Cotton, F., Asim, K. M., Savran, W. H., & Marzocchi, W. (2024). Evaluation of a decade-long prospective earthquake forecasting experiment in Italy. Seismological Research Letters,. doi: 10.1785/0220230247.
Related Projects & Working Groups
CSEP, Earthquake Forecasting and Predictability (EFP)
