Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning

Guilherme R. Pretto, Bruno L. Dalmazo, Jonatas A. Marques, Zhongke Wu, Xingce Wang, Vladimir Korkhov, Philippe O.A. Navaux, Luciano P. Gaspary

Research output

Abstract

Data centers, clusters and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, in recent past, we have witnessed consistent efforts to run HPC applications in the cloud. The potential advantages of this shift include higher scalability and lower costs. If on the one hand, app instantiation – through customized Virtual Machines (VMs) – is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To alleviate this problem, and taking advantage of new advances in programmable networks, we propose a mechanism for dynamic, just-in-time path provisioning in cloud infrastructures. It continuously monitors the network conditions and, given the current communication patterns of the application, systematically (re)programs paths to avoid uncongested links and reduce end-to-end delays. The proposed mechanism achieves a speedup of up to 44.24% regarding application runtime when compared to the traditional shortest-path, static approach.

Original languageEnglish
Pages (from-to)702-716
Number of pages15
JournalLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11622
DOIs
Publication statusPublished - 1 Jul 2019
Event19th International Conference on Computational Science and Its Applications, ICCSA 2019 - Saint Petersburg
Duration: 1 Jul 20194 Jul 2019

Fingerprint

Boosting
High Performance
Traffic
Path
Computing
Virtual Machine
Infrastructure
End-to-end Delay
Data Center
Application programs
Shortest path
Scalability
Monitor
Speedup
Grid
Communication
Costs
Virtual machine

Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Pretto, Guilherme R. ; Dalmazo, Bruno L. ; Marques, Jonatas A. ; Wu, Zhongke ; Wang, Xingce ; Korkhov, Vladimir ; Navaux, Philippe O.A. ; Gaspary, Luciano P. / Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2019 ; Vol. 11622. pp. 702-716.
@article{ed7091807984499eaf0794071974d638,
title = "Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning",
abstract = "Data centers, clusters and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, in recent past, we have witnessed consistent efforts to run HPC applications in the cloud. The potential advantages of this shift include higher scalability and lower costs. If on the one hand, app instantiation – through customized Virtual Machines (VMs) – is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To alleviate this problem, and taking advantage of new advances in programmable networks, we propose a mechanism for dynamic, just-in-time path provisioning in cloud infrastructures. It continuously monitors the network conditions and, given the current communication patterns of the application, systematically (re)programs paths to avoid uncongested links and reduce end-to-end delays. The proposed mechanism achieves a speedup of up{\^A} to 44.24{\%} regarding application runtime when compared to the traditional shortest-path, static approach.",
keywords = "Cloud infrastructures, HPC applications, Link usage-aware path provisioning",
author = "Pretto, {Guilherme R.} and Dalmazo, {Bruno L.} and Marques, {Jonatas A.} and Zhongke Wu and Xingce Wang and Vladimir Korkhov and Navaux, {Philippe O.A.} and Gaspary, {Luciano P.}",
year = "2019",
month = "7",
day = "1",
doi = "10.1007/978-3-030-24305-0_52",
language = "English",
volume = "11622",
pages = "702--716",
journal = "Lecture Notes in Computer Science",
issn = "0302-9743",
publisher = "Springer",

}

Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning. / Pretto, Guilherme R.; Dalmazo, Bruno L.; Marques, Jonatas A.; Wu, Zhongke; Wang, Xingce; Korkhov, Vladimir; Navaux, Philippe O.A.; Gaspary, Luciano P.

In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 11622, 01.07.2019, p. 702-716.

Research output

TY - JOUR

T1 - Boosting HPC Applications in the Cloud Through JIT Traffic-Aware Path Provisioning

AU - Pretto, Guilherme R.

AU - Dalmazo, Bruno L.

AU - Marques, Jonatas A.

AU - Wu, Zhongke

AU - Wang, Xingce

AU - Korkhov, Vladimir

AU - Navaux, Philippe O.A.

AU - Gaspary, Luciano P.

PY - 2019/7/1

Y1 - 2019/7/1

N2 - Data centers, clusters and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, in recent past, we have witnessed consistent efforts to run HPC applications in the cloud. The potential advantages of this shift include higher scalability and lower costs. If on the one hand, app instantiation – through customized Virtual Machines (VMs) – is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To alleviate this problem, and taking advantage of new advances in programmable networks, we propose a mechanism for dynamic, just-in-time path provisioning in cloud infrastructures. It continuously monitors the network conditions and, given the current communication patterns of the application, systematically (re)programs paths to avoid uncongested links and reduce end-to-end delays. The proposed mechanism achieves a speedup of up to 44.24% regarding application runtime when compared to the traditional shortest-path, static approach.

AB - Data centers, clusters and grids have historically supported High-Performance Computing (HPC) applications. Due to the high capital and operational expenditures associated with such infrastructures, in recent past, we have witnessed consistent efforts to run HPC applications in the cloud. The potential advantages of this shift include higher scalability and lower costs. If on the one hand, app instantiation – through customized Virtual Machines (VMs) – is a well-solved issue, on the other, the network still represents a significant bottleneck. When switching HPC applications to be executed on the cloud, we lose control of where VMs will be positioned and of the paths that will be traversed for processes to communicate with one another. To alleviate this problem, and taking advantage of new advances in programmable networks, we propose a mechanism for dynamic, just-in-time path provisioning in cloud infrastructures. It continuously monitors the network conditions and, given the current communication patterns of the application, systematically (re)programs paths to avoid uncongested links and reduce end-to-end delays. The proposed mechanism achieves a speedup of up to 44.24% regarding application runtime when compared to the traditional shortest-path, static approach.

KW - Cloud infrastructures

KW - HPC applications

KW - Link usage-aware path provisioning

UR - http://www.scopus.com/inward/record.url?scp=85068610059&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-24305-0_52

DO - 10.1007/978-3-030-24305-0_52

M3 - Article

AN - SCOPUS:85068610059

VL - 11622

SP - 702

EP - 716

JO - Lecture Notes in Computer Science

JF - Lecture Notes in Computer Science

SN - 0302-9743

ER -