TY - JOUR

T1 - Janus

T2 - a framework to boost HPC applications in the cloud based on SDN path provisioning

AU - Pretto, Guilherme R.

AU - Dalmazo, Bruno L.

AU - Marques, Jonatas A.

AU - Wu, Zhongke

AU - Wang, Xingce

AU - Korkhov, Vladimir

AU - Navaux, Philippe O.A.

AU - Gaspary, Luciano Paschoal

N1 - Publisher Copyright: © 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

PY - 2022/4

Y1 - 2022/4

N2 - Data centers, clusters, and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, we have witnessed consistent efforts to run HPC applications in the cloud in the recent past. The potential advantages of this shift include higher scalability and lower costs. If, on the one hand, app instantiation—through customized Virtual Machines (VMs)—is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To bridge this gap, we present Janus, a framework for dynamic, just-in-time path provisioning in cloud infrastructures. By leveraging emerging software-defined networking principles, the framework allows for an HPC application, once deployed, to have interprocess communication paths configured upon usage based on least-used network links (instead of resorting to shortest, pre-computed paths). Janus is fully configurable to cope with different operating parameters and communication strategies, providing a rich ecosystem for application execution speed up. Through an extensive experimental evaluation, we provide evidence that the proposed framework can lead to significant gains regarding runtime. Moreover, we show what one can expect in terms of system overheads, providing essential insights on how better benefiting from Janus.

AB - Data centers, clusters, and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, we have witnessed consistent efforts to run HPC applications in the cloud in the recent past. The potential advantages of this shift include higher scalability and lower costs. If, on the one hand, app instantiation—through customized Virtual Machines (VMs)—is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To bridge this gap, we present Janus, a framework for dynamic, just-in-time path provisioning in cloud infrastructures. By leveraging emerging software-defined networking principles, the framework allows for an HPC application, once deployed, to have interprocess communication paths configured upon usage based on least-used network links (instead of resorting to shortest, pre-computed paths). Janus is fully configurable to cope with different operating parameters and communication strategies, providing a rich ecosystem for application execution speed up. Through an extensive experimental evaluation, we provide evidence that the proposed framework can lead to significant gains regarding runtime. Moreover, we show what one can expect in terms of system overheads, providing essential insights on how better benefiting from Janus.

KW - Cloud infrastructures

KW - Framework

KW - HPC applications

KW - Link usage-aware path provisioning

KW - NETWORK

UR - http://www.scopus.com/inward/record.url?scp=85119881928&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/ed0b142c-685c-33d7-87d3-9fe9b0f10720/

U2 - 10.1007/s10586-021-03470-6

DO - 10.1007/s10586-021-03470-6

M3 - Article

AN - SCOPUS:85119881928

VL - 25

SP - 947

EP - 964

JO - Cluster Computing

JF - Cluster Computing

SN - 1386-7857

IS - 2

ER -