Future ExaScale computing systems are expected to exceed a hundred-thousand nodes and contain a rich mix of heterogeneous computing resources created by coupling multi-core processors with accelerator processors. However, current applications are not yet able to exploit more than a few tens of thousands of these nodes, or effectively manage this heterogeneous mix of resources to balance workloads. Moreover, this heterogeneous mixture of resources in hybrid or multi-paradigm systems presents programmers with new, but challenging opportunities for exploiting the various types of parallelism within algorithms. This CAREER research will formulate an inclusive hierarchical framework for performance modeling and analysis of hybrid computing systems that include multi-core processors and accelerators. The framework includes a coarse-grained model for rapid assessment of the fitness match between the application and the hybrid system and a fine-grained model for predicting execution time and workload balance analysis. These models will be useful for researchers and scientists developing, optimizing, and maintaining scientific codes.
The research project includes forming a taxonomy of application and architecture characteristics through a rigorous study of application optimization and execution on multi-core and accelerator processors. The study focuses on the inter-node parallelism of the applications across a mix of heterogeneous nodes and the intra-node parallelism on the multi-paradigm nodes. These hybrid system studies elucidate the application and architecture characteristics included in the coarse and fine-grained models of the hierarchical framework. The optimization methods (resource ratio optimization, load balancing and memory management) and the model framework are qualitatively assessed for other application domains not included in this study. The project also transitions this knowledge into educational modules suitable for classroom use in the undergraduate and graduate curriculum.
