Hamlet: An Application for Seismic Hazard Model Evaluation and Testing

Richard H. Styron, Yufang Rong, Kendra Johnson, & Marco Pagani

Published September 11, 2022, SCEC Contribution #12188, 2022 SCEC Annual Meeting Poster #200

Although the desirability of evaluating the statistical performance of seismic hazard models is widely felt, the practice of such evaluations has remained limited. This is due to a variety of factors, including the limited data suitable for testing (particularly that are not used in model construction), the fact that the tools used to perform such testing have only recently become readily available, and that particular tests or evaluations implemented may not be perfectly suited to the needs of all users.

We separate the use cases of hazard model evaluation into two categories: adjudication among competing models, and iterative refinement of a single model during development. The former use case has received the most attention, e.g., through the Center for the Study of Earthquake Predictability (CSEP). The latter is less discussed, though it has great potential to improve hazard model performance because it is real-time and directly influences the quality of the forthcoming model. Though not identical, both approaches share much of their conceptual framework and statistical machinery, and improvements and insight from one approach can benefit the other.

The Global Earthquake Model Foundation (GEM) is developing the application Hamlet (Hazard Model Evaluation and Testing), which performs the M, N, S, and L tests developed by CSEP as well as additional evaluations using a variety of metrics as well as simple sanity checks to better quantify model performance. Hamlet is also capable of evaluating subsets of a hazard model based on region, source type, and logic tree branch, to increase the resolution and insight of the analysis.

We have used Hamlet to analyze over 30 models in the GEM global hazard mosaic to gauge the applicability of different tests and evaluations to a variety of cases (e.g., seismic hazard models with different characteristics), and to test some widely used assumptions in models to identify which work and which do not. Furthermore, we are working to incorporate analysis with Hamlet more deeply into our model development workflow. Here, we discuss strategies, preliminary results, challenges, and future directions for hazard model evaluation and development.

Key Words
PHSA, seismic hazard model testing

Citation
Styron, R. H., Rong, Y., Johnson, K., & Pagani, M. (2022, 09). Hamlet: An Application for Seismic Hazard Model Evaluation and Testing. Poster Presentation at 2022 SCEC Annual Meeting.

Related Projects & Working Groups
Earthquake Forecasting and Predictability (EFP)