Fused Seismic Simulations with the Discontinuous Galerkin Method at Extreme-Scale

Alexander N. Breuer, Alexander Heinecke, & Yifeng Cui

Published August 14, 2017, SCEC Contribution #7644, 2017 SCEC Annual Meeting Poster #280

We present EDGE, a solver for hyperbolic partial differential equations with emphasis on seismic setups.
EDGE is designed from scratch to exploit inter-simulation parallelism by fusing simulations.
Fused simulations are a novel technique addressing one of the grand challenges in computational seismology:
large ensemble runs of geometrically similar forward simulations.
While a traditional solver s uses fixed input i to obtain observations o=s(i), our approach operates on multiple inputs.
Assuming a set of m different inputs I = (i_1, i_2, ..., i_m), a single execution of EDGE's solver S operates on the entire set of inputs in parallel to obtain the set of observations O = (o_1, o_2, ..., o_m) = S(I).
High-dimensional application fields include, but are not limited to: derivation of accurate velocity models through tomographic inversion and uncertainty quantification in the context of seismic hazard analysis.

EDGE tackles complicated model geometries (topography, material contrasts and internal fault boundaries) by using the Discontinuous Galerkin Finite Element Method (DG-FEM) method for spatial and Arbitrary high order DERivatives (ADER) for time discretization, implemented for unstructured tetrahedral meshes.
In addition, our software allows for rapid prototyping through the support of line elements, quadrilaterals, triangles, and hexahedrons.

Using convergence rates 2-4, EDGE’s fused approach outperforms the simulation throughput of SeisSol (version 201511) by a factor of 1.8 - 4.6 on the Knights Landing architecture.
Further, EDGE's extreme-scale performance in scaling studies is unprecedented.
By utilizing 612,000 cores of Cori Phase II, we were able to sustain 10.4 DP-PFLOPS, the highest ever sustained performance for a seismic simulation.

We conclude our poster presentation by giving an outlook on under development extensions of the solver, targeting advanced spontaneous rupture earthquake simulations.

Citation
Breuer, A. N., Heinecke, A., & Cui, Y. (2017, 08). Fused Seismic Simulations with the Discontinuous Galerkin Method at Extreme-Scale. Poster Presentation at 2017 SCEC Annual Meeting.

Related Projects & Working Groups
Computational Science (CS)