Development and optimizations of a SCEC community anelastic wave propagation platform for multicore systems and GPU-based accelerators

Yifeng Cui, Kim B. Olsen, Jun Zhou, Patrick Small, Amit Chourasia, Steven M. Day, Philip J. Maechling, & Thomas H. Jordan

Published 2012, SCEC Contribution #1851

AWP-ODC is a scalable finite-difference application package, involving col- laborative development coordinated by the SCEC Community Modeling Environment. This platform has undergone many optimizations in recent years, transformed from Olsen’s personal research code into a community code for large-scale dynamic rupture and wave propagation modeling. The code recently achieved “M8”, a full dynamical simulation of a magnitude-8 earthquake on the southern San Andreas fault up to 2-Hz using 223, 074 cores with a sustained per- formance of 220 TFlops. Current SCEC efforts involving AWP-ODC include computation of strain Green’s tensors, as part of SCEC CyberShake 3.0 effort to compute deterministic and probabilistic seismic hazard in California.

This presentation will describe the software capabilities and components from mesh generation to post-processing. Optimization problems tend to emerge at large-scale that are not significant in smaller scale simulations. Multicore NUMA architectures and many-core co-processors have further increased com- plexity that has pushed the burden of obtaining good performance to the applica- tion level. We will summarize the optimization techniques that have allowed our application to run efficiently on petascale supercomputers: efficient algorithms for load balancing, effective intra-node and inter-node communications, optimal cache utilization, scalable IO, fault tolerance, and understanding the underly- ing characteristics of the parallel file system involved. Recently, AWP-ODC has been ported to CUDA-MPI preparing for GPU-based acceleration, where we will introduce the benchmarks of performance on NVIDIA Tesla M2090 and C2050 graphics cards. The presentation will conclude with a discussion on how the seismology community can prepare for the challenges of Exascale computing.

Citation
Cui, Y., Olsen, K. B., Zhou, J., Small, P., Chourasia, A., Day, S. M., Maechling, P. J., & Jordan, T. H. (2012). Development and optimizations of a SCEC community anelastic wave propagation platform for multicore systems and GPU-based accelerators. Seismological Research Letters, 83(2), 396.