SCEC Award Number 16067 View PDF
Proposal Category Individual Proposal (Integration and Theory)
Proposal Title An Examination of the Correlations Between Different Goodness-of-Fit Metrics Based on a Large Dataset of Ground Motion Simulation Validations
Investigator(s)
Name Organization
Ricardo Taborda University of Memphis
Other Participants Shima Azizzadeh-Roodpish (Ph.D. student, supported)
Naeem Khoshnevis (Ph.D. student, not supported)
SCEC Priorities 6e, 6c, 6a SCEC Groups GMSV, EEII, GMP
Report Due Date 03/15/2017 Date Report Submitted 03/17/2017
Project Abstract
 In this project we identified the relationships that exist between different goodness-of-fit (GOF) metrics currently used in the validation of ground motion simulations by conducting a statistical analysis on a large dataset of comparisons between synthetics and records from past earthquakes. We identified metrics with strong correlations with a previously established GOF scoring scale and in doing so we singled out those sets of metrics that show significant independence from other sets of similarly independent parameters. This identification is done both through simple correlation analysis and also through a principal components analysis. The main result of this project is that the described analysis allowed us to narrow the number of metrics from eleven to three, without a significant impact on the final scoring scale if compared against a reference GOF method. The new approach, which consists mainly of a new combination GOF expression with weights used to compute the final score, uses metrics based on measures of a signal’s amplitude, time evolution, and frequency content. This composition is consistent, in principle, with other proposed methods by different authors which also measure these characteristics, but it advances such previously proposed methods in that our new GOF combination formula also offers consistency with a widely used GOF method, thus allowing future results to be compatible with previous ones.
Intellectual Merit Prior efforts in evaluation of ground motion simulations through comparisons with data have evidenced that validation methods were highly subjective to the user’s preferences and—arguably—to the application intended for the simulation. We argued that this limited our capacity to bring about a broader recognition of physics-based simulation, and therefore, the intellectual merit of the project rested on the idea of reducing such subjectivity by identifying which goodness-of-fit metrics were the most relevant and constituted a sufficient set of metrics necessary to determine the validity of synthetics if measured with respect to prior results from a larger set of metrics and simulations. In this regard, we conducted a data-driven analysis that allowed us to investigate the correlations that exist within a wide range of validation metrics, and arrived to an informed decision about the weight carried by a narrower set of metrics. With this research we have contributed to identify the level of independence or correlation that exist between different metrics used for validation; the sensitivity of GOF metrics to specific earthquakes and simulation parameters such as maximum frequency ranges and velocity models; and proposed a new GOF method that uses only three metrics as opposed to ten or eleven, as used before. By narrowing the spectrum of parameters used for validation we expect to be able to transfer knowledge to engineers interested in validation of ground motion simulations and use this knowledge to advance the broader recognition of physics based ground motion simulation methods.

In reference to SCEC, this project contributed to Proposal Category B: Theory and integration; it ad-dressed Fundamental Problem 6: Seismic wave generation and scattering – Prediction of strong ground motions; and contributed to Research Priority 6e. In this regard, the outcome of the project contribute more directly to collaborations with the engineering community in the area of validation and utilization of ground motion simulations, and the activities of the interdisciplinary focus areas in Ground Motion Pre-diction and Earthquake Engineering Implementation Interface, namely the Technical Activity Group on Ground Motion Simulation Validation, the activities of the Broadband Platform, High-F, and CyberShake special projects; and in passing, to the objectives of the Committee for Utilization of Ground Motion Simulations.
Broader Impacts This project contributed to advancing efforts in validation that will ultimately help broaden the impact of physics-based ground motion simulation and use of ground motion synthetics in engineering and other related fields. This has been accomplished by simplifying a validation method and by showing that only a few validation metrics are necessary to capture the most relevant characteristics in comparisons be-tween synthetics and records.

This project provided funding to support the work of PhD student and an Assistant Professor from underrepresented groups (women and Latino, respectively).
Exemplary Figure Included in the front matter of the report.
Linked Publications

Add missing publication or edit citation shown. Enter the SCEC project ID to link publication.