SCEC Award Number 25166 View PDF
Proposal Category Individual Research Project (Single Investigator / Institution)
Proposal Title Potency-weighted scoring metrics for assessing CSEP Forecasts
Investigator(s)
Name Organization
Frederic Schoenberg University of California, Los Angeles
SCEC Milestones C1,2,3-1, D2-1, D3-1 SCEC Groups EFP, RC, ASI
Report Due Date 03/15/2026 Date Report Submitted 10/11/2025
Project Abstract
 The proposed research will extend current standard scoring metrics such as the Brier score and log-likelihood score so that they properly weight earthquakes according to their potency. We will evaluate all short-term (daily) models in the Collaboratory for the Study of Earthquake Predictability (CSEP) for each year since CSEP's inception, to determine which models have the highest efficacy at forecasting according to these metrics. Weighting earthquakes by their potency in the model evaluation will allow us to identify models that best succeed at forecasting the largest and most destructive events. In order to highlight the best-fitting models and to disseminate the results to an audience of potentially interested scholars, we will identify the best-fitting model for each year for each score considered. The main idea for this endeavor came about during a conversation between the Principal Investigator and Ilya Zaliapin, who died on 5/2/2023 at the age of just 50, and this project is proposed and will be completed in his honor.
Intellectual Merit We proposed potency-weighted measures for evaluating the goodness-of-fit of CSEP models and studied their statistical properties. These measures should be used in the future to help identify models that most accurately forecast the largest and most destructive earthquakes.
Broader Impacts My PhD student, Alejandra Arjon, who is a Latina female, participated in this project and has advanced to candidacy at UCLA as a PhD candidate by working with me on the research in this project.
Project Participants Frederic Schoenberg and Alejandra Arjon worked on the project, and we collaborated with Francesco Serafini, Maximilian Werner, and Julia Jansson.
Exemplary Figure Figure 1.
Caption: (a) 44 events above magnitude 3.95 in California in 2014, with three in the top 5% of magnitudes (M5.08). (b) Q-scores of various CSEP models for the year 2014. STEP-JAVA has the 2nd highest Q score among all CSEP models for 2014.
Linked Publications

Add missing publication or edit citation shown. Enter the SCEC project ID to link publication.