Group B, Poster #214, Earthquake Forecasting and Predictability (EFP)

Modernizing CSEP experiments: The Floating Testing Center

William H. Savran, José A. Bayona, Pablo C. Iturrieta, Khawaja M. Asim, Han Bao, Kirsty Bayliss, Marcus Herrmann, Danijel Schorlemmer, Philip J. Maechling, & Maximilian J. Werner
Poster Image: 

Poster Presentation

2022 SCEC Annual Meeting, Poster #214, SCEC Contribution #12583 VIEW PDF
The Collaboratory for the Study of Earthquake Predictability (CSEP) is an open and international community whose mission is to accelerate earthquake predictability research through rigorous testing of probabilistic earthquake forecast models and prediction algorithms. CSEP testing revolves around the prospective evaluation of probabilistic earthquake forecasting models. In prospective experiments, the parameters of the experiment (including forecast generation, data sets, and evaluation metrics) must be defined with zero degrees of freedom before any evaluations begin. These experiments are facilitated by cyberinfrastructure colloquially known as the testing center. CSEP testing centers are ...centralized servers that can run autonomous evaluations of earthquake forecasting models. This approach has been successful for over a decade; however, advancements in open-source software development and open science initiatives have prompted us to improve upon this model. Over the last year, CSEP has been working to modernize its testing capabilities in large part through a collaboration with the Real-time Earthquake Risk Reduction for a Resilient Europe (RISE) project. Modernizing CSEP experiments involves a two major efforts: (1) Developing an open-source library, pyCSEP, that provides necessary routines for evaluating earthquake forecasts, and (2) developing an open experiment format that decentralizes the testing process and promotes best-practices in open science software development and data management. Recent pyCSEP efforts include expanding the forecasting class to forecasts defined on Quadtree grids, adding evaluation methods such as binary likelihood scores and receiver operating characteristics, and new visualization routines. Many of these contributions were made by members the community. We demonstrate the open experiment format using the Global Earthquake Forecasting Experiment (GEFE) developed in collaboration with CSEP and RISE. This format ensures all experiment artifacts are made available through public software and data repositories, and the forecasts and evaluation results can be reproduced by other researchers on their own computers or servers. The GEFE addresses the comparability and the stability of test results on Quadtree grids, which provide a significant computational improvement to evaluating global forecasting models. Additionally, this experiment format will be the basis for future benchmark problems developed by CSEP.
SHOW MORE

Citation: Savran, W. H., Bayona, J. A., Iturrieta, P. C., Asim, K. M., Bao, H., Bayliss, K., Herrmann, M., Schorlemmer, D., Maechling, P. J., & Werner, M. J. (2022, 09). Modernizing CSEP experiments: The Floating Testing Center. Poster Presentation at 2022 SCEC Annual Meeting.
Relevant SCEC Themes:
Probabilistic Seismic Hazard Analysis
Operational Earthquake Forecasting