Unified and Continuous Software Development for AWP-ODC-OS

Yifeng Cui, Alexander N. Breuer, Rajdeep Konwar, & David Lenz

Published August 15, 2018, SCEC Contribution #8787, 2018 SCEC Annual Meeting Poster #294

State-of-the-art software development and engineering techniques are in high demand for increased code quality and usability. Important best practices include sound licensing, unified software development, version control for software and data, as well as Continuous Integration (CI) and Continuous Delivery (CD) solutions. Our poster showcases the application of best practices in software development and software engineering to AWP-ODC-OS, a highly scalable finite difference solver for seismic wave propagation. The final goal of our fully integrated software and data infrastructure is 1) a clear and transparent license structure for software and data, 2) well-documented and continuously tested examples and benchmarks, and 3) a significant reduction of development time through fast and automated status reports on the software's health.

The first part of the poster illustrates our newly introduced structure for related software and data. We showcase the layout of the respective repositories and provide insights on the technical aspects of the implementation. Further, we motivate and explain our newly introduced licenses, as well as required institutional steps for this open source release. We use the established BSD3 license for all software products and waivers (CC0) for all data products. This practice provides solid structure for inter-institutional collaboration, also extending to the commercial sector.

The second part of our poster showcases our CI and CD pipelines, using the open-source continuous delivery server GoCD (Apache License 2.0). Our GoCD server schedules work to agents, running on remote resources. For example, these resources could be hosted in terms of a local department-clusters, or in compute clouds, e.g., Amazon's AWS EC2. This setup overcomes computational limitations of commercial solutions for CI/CD, typically running a single CPU core. In addition, we interpret selected agents as interfaces to supercomputing infrastructure, e.g., TACC's Stampede or SDSC's Comet. By using the low-level workflow tool SAGA-Python, our agents are able to automatically communicate with diverse job submission system. In summary, our final CI/CD pipelines will automatically analyze AWP-ODC-OS from simple sanity checks, e.g., memory debugging, towards continuous verification on supercomputing infrastructure.

Key Words
AWP-ODC, wave propagation modeling, continuous software development

Cui, Y., Breuer, A. N., Konwar, R., & Lenz, D. (2018, 08). Unified and Continuous Software Development for AWP-ODC-OS. Poster Presentation at 2018 SCEC Annual Meeting.

Related Projects & Working Groups
Computational Science (CS)