Seismological Research Letters

William H. Savran, Pablo C. Iturrieta, José A. Bayona, Khawaja M. Asim, Danijel Schorlemmer, Marcus Herrmann, David A. Rhoades, David D. Jackson, Matthew C. Gerstenberger, Warner Marzocchi, Kenny Graham, Philip J. Maechling, & Maximilian J. Werner

In Preparation May 1, 2023, SCEC Contribution #12726

The Collaboratory for the Study of Earthquake Predictability (CSEP) is an international research community whose mission is to accelerate earthquake predictability research through rigorous testing of probabilistic earthquake forecast models and prediction algorithms.

The CSEP experiments revolves around the prospective evaluation of probabilistic earthquake forecasting models. In these prospective experiments, the parameters of the experiment (including forecast generation, data sets, and evaluation metrics) must be defined with zero degrees of freedom before any evaluations begin.

These experiments are facilitated by cyber-infrastructure colloquially known as the testing center. The first implementation of CSEP testing centers used centralized servers to run autonomous evaluations of earthquake forecasting models.

This approach has been successful for over a decade; however, advancements in open-source software development and open science initiatives have prompted us to improve upon this model.
Over the last couple of years, CSEP has been working to modernize its testing capabilities in large part through a collaboration with the Real-time Earthquake Risk Reduction for a Resilient Europe (RISE) project.

Modernizing CSEP experiments involves two major efforts: (1) Developing an open-source library, pyCSEP, that provides necessary routines for evaluating earthquake forecasts, and (2) developing an open experiment format that decentralizes the testing process and promotes best-practices in open science software development and data management.

We demonstrate the open experiment format using the Global Earthquake Forecasting Experiment (GEFE).
The GEFE addresses the comparability and the stability of test results on quad-tree grids, which provide a significant computational improvement to evaluating global forecasting models.

Additionally, the open experiment format can be the basis for future experiments developed by CSEP and independent researchers.

Citation
Savran, W. H., Iturrieta, P. C., Bayona, J. A., Asim, K. M., Schorlemmer, D., Herrmann, M., Rhoades, D. A., Jackson, D. D., Gerstenberger, M. C., Marzocchi, W., Graham, K., Maechling, P. J., & Werner, M. J. (2023). Seismological Research Letters. Modernizing CSEP Earthquake Forecasting Experiments: The Floating Testing Center, (in preparation).