Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance Tracking: The CyberShake Example

Ewa Deelman, Scott Callaghan, Hunter Francoeur, Robert W. Graves, Nitin Gupta, Vipin Gupta, Thomas H. Jordan, Carl Kesselman, Philip J. Maechling, John Mehringer, Gaurang Mehta, David A. Okaya, Karan Vahi, & Li Zhao

Published December 2006, SCEC Contribution #1060

This paper discusses the process of building an environment where large-scale, complex, scientific analysis can be scheduled onto a heterogeneous collection of computational and storage resources. The example application is the Southern California Earthquake Center (SCEC) CyberShake project, an analysis designed to compute probabilistic seismic hazard curves for sites in the Los Angeles area. We explain which software tools were used to build to the system, describe their functionality and interactions. We show the results of running the CyberShake analysis that included over 250,000 jobs using resources available through SCEC and the TeraGrid.

Citation
Deelman, E., Callaghan, S., Francoeur, H., Graves, R. W., Gupta, N., Gupta, V., Jordan, T. H., Kesselman, C., Maechling, P. J., Mehringer, J., Mehta, G., Okaya, D. A., Vahi, K., & Zhao, L. (2006, 12). Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance Tracking: The CyberShake Example. Presentation at e-Science'06. doi: 10.1109/E-SCIENCE.2006.261098.