The Q-score: a magnitude-weighted goodness-of-fit score for earthquake forecasting

Julia Jansson Valter, Alejandra Arjon, Francesco Serafini, & Frederic Schoenberg

In Preparation 2025, SCEC Contribution #14995

Accurate forecasting of large earthquakes is of great importance, yet most current earthquake forecast evaluation metrics, such as the log-likelihood score, do not give additional weight to large-magnitude events. To this end, magnitude-weighted goodness-of-fit scores for earthquake forecasting have recently been introduced, such as potency weighted log-likelihood and the Q-score. In this paper, we define and further investigate the Q-score, which is a quotient emphasizing model fit for the 5% largest earthquakes. We explore theoretical properties of the Q-score, demonstrating that under certain conditions, its ex- pectation is essentially 1 and that it satisfies a law of large numbers. The Q-score is evaluated for 27 next-day gridded earthquake forecasts for California by the Collaborative for the Study of Earthquake Predictability from 2012, 2014, and 2017. We also compare to the log-likelihood scores of the forecasts to assess how well different models perform, both for predicting the largest events but also in terms of overall model fit.

Citation
Jansson Valter, J., Arjon, A., Serafini, F., & Schoenberg, F. (2025). The Q-score: a magnitude-weighted goodness-of-fit score for earthquake forecasting. Annals of Applied Statistics, (in preparation).