Projects
PERTURBO
Electron Interactions and Dynamics in Materials
PI: Marco Bernardi (Division of Engineering and Applied Sciences)
SASE: Donnie Pinkston, Instructor
PERTURBO, an open-source project, uses first-principles methods to accurately predict the microscopic motion and interactions of electrons in materials. The code provides unique features and workflows for these calculations, and has enabled important scientific advances. PERTURBO has been implemented to scale favorably up to hundreds of thousands of CPU cores in high-performance computing environments using both MPI and OpenMP parallelization, but until recently was unable to take advantage of modern GPU hardware in its calculations.
Collaborating closely with members of Marco Bernardi’s team, the Schmidt Academy software engineer was able to introduce multiple significant enhancements into the PERTURBO codebase. A new data structure and computational approach slashed PERTURBO’s memory requirements down to less than half of the original approach, while also speeding up CPU-based computations by approximately 3x. Even before moving PERTURBO onto GPU hardware, it is already able to run larger simulations on the same compute infrastructure in less time, as compared to previous versions.
The new data structures also allowed PERTURBO data and computations to be migrated onto the GPU using OpenACC alongside MPI and OpenMP, improving computational performance by more than an order of magnitude over the previous version. Careful profiling and code-tuning were able to achieve a highly performant GPU code that continues PERTURBO’s pattern of highly favorable strong-scaling characteristics on dozens or hundreds of GPUs.
Several other improvements were also made to the codebase, including the reporting of memory and GPU resources used by PERTURBO, several tools to simplify the setup of large experiments in an HPC environment, the identification and elimination of buggy language-usage patterns, and other contributions that will improve both the usability and robustness of the PERTURBO project.
More details on this work, along with the new version of PERTURBO, will be shared in upcoming publications from Bernardi’s team.
https://perturbo-code.github.io/
