Application-specific I/O optimizations on petascale supercomputers

Efecan Poyraz, Heming Xu, & Yifeng Cui

Published June 6, 2014, SCEC Contribution #2036

Data-intensive science frontiers and challenges are emerging as computer technology has evolved substantially. Large-scale simulations demand significant I/O workload, and as a result the I/O performance often becomes a bottleneck preventing high performance in scientific applications. In this paper we introduce a variety of I/O optimization techniques developed and implemented when scaling a seismic application to petascale. These techniques include file system striping, data aggregation, reader/writer limiting and less interleaving of data, collective MPI-IO, and data staging. The optimizations result in nearly perfect scalability of the target applicat ion on some of the most advanced petascale systems. The techniques introduced in this paper are applicable to other scientific applications facing similar petascale I/O challenges.

Key Words
Earthquake simulation, io optimizations, lustre striping, petascale io

Citation
Poyraz, E., Xu, H., & Cui, Y. (2014). Application-specific I/O optimizations on petascale supercomputers. Procedia Computer Science, 29, 910-923. doi: 10.1016/j.procs.2014.05.082.

Related Projects & Working Groups
Computational Science, Community Modeling Environment