Standard
Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark. / Korkhov, Vladimir; Gankevich, Ivan; Iakushkin, Oleg; Gushchanskiy, Dmitry; Khmel, Dmitry; Ivashchenko, Andrey; Pyayt, Alexander; Zobnin, Sergey; Loginov, Alexander.
Computational Science and Its Applications – ICCSA 2017: 17th International Conference, Trieste, Italy, July 3-6, 2017, Proceedings, Part V. Springer Nature, 2017. p. 387-398 (Lecture Notes in Computer Science; Vol. 10408).
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
Harvard
Korkhov, V, Gankevich, I, Iakushkin, O, Gushchanskiy, D, Khmel, D
, Ivashchenko, A, Pyayt, A, Zobnin, S & Loginov, A 2017,
Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark. in
Computational Science and Its Applications – ICCSA 2017: 17th International Conference, Trieste, Italy, July 3-6, 2017, Proceedings, Part V. Lecture Notes in Computer Science, vol. 10408, Springer Nature, pp. 387-398, 17th International Conference on Computational Science and Its Applications, ICCSA 2017, Trieste, Italy,
2/07/17.
https://doi.org/10.1007/978-3-319-62404-4_28
APA
Korkhov, V., Gankevich, I., Iakushkin, O., Gushchanskiy, D., Khmel, D.
, Ivashchenko, A., Pyayt, A., Zobnin, S., & Loginov, A. (2017).
Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark. In
Computational Science and Its Applications – ICCSA 2017: 17th International Conference, Trieste, Italy, July 3-6, 2017, Proceedings, Part V (pp. 387-398). (Lecture Notes in Computer Science; Vol. 10408). Springer Nature.
https://doi.org/10.1007/978-3-319-62404-4_28
Vancouver
Korkhov V, Gankevich I, Iakushkin O, Gushchanskiy D, Khmel D
, Ivashchenko A et al.
Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark. In Computational Science and Its Applications – ICCSA 2017: 17th International Conference, Trieste, Italy, July 3-6, 2017, Proceedings, Part V. Springer Nature. 2017. p. 387-398. (Lecture Notes in Computer Science).
https://doi.org/10.1007/978-3-319-62404-4_28
Author
Korkhov, Vladimir ; Gankevich, Ivan ; Iakushkin, Oleg ; Gushchanskiy, Dmitry ; Khmel, Dmitry
; Ivashchenko, Andrey ; Pyayt, Alexander ; Zobnin, Sergey ; Loginov, Alexander. /
Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark. Computational Science and Its Applications – ICCSA 2017: 17th International Conference, Trieste, Italy, July 3-6, 2017, Proceedings, Part V. Springer Nature, 2017. pp. 387-398 (Lecture Notes in Computer Science).
BibTeX
@inproceedings{32be9b70c2d4486aa80100f0d55218f9,
title = "Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark",
abstract = "Modern architectures of data acquisition and processing often consider low-cost and low-power devices that can be bound together to form a distributed infrastructure. In this paper we overview possibilities to organize a distributed computing testbed based on microcomputers similar to Raspberry Pi and Intel Edison. The goal of the research is to investigate and develop a scheduler for orchestrating distributed data processing and general purpose computations on such unreliable and resource-constrained hardware. Also we consider integration of the scheduler with well-known distributed data processing framework Apache Spark. We outline the project carried out in collaboration with Siemens LLC to compare different configurations of the hardware and software deployment and evaluate performance and applicability of the tools to the testbed.",
keywords = "Microcomputers, scheduling, Apache Spark, Raspberry Pi, Fault tolerance, High availability",
author = "Vladimir Korkhov and Ivan Gankevich and Oleg Iakushkin and Dmitry Gushchanskiy and Dmitry Khmel and Andrey Ivashchenko and Alexander Pyayt and Sergey Zobnin and Alexander Loginov",
note = "Korkhov V. et al. (2017) Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark. In: Gervasi O. et al. (eds) Computational Science and Its Applications – ICCSA 2017. ICCSA 2017. Lecture Notes in Computer Science, vol 10408. Springer, Cham. https://doi.org/10.1007/978-3-319-62404-4_28; 17th International Conference on Computational Science and Its Applications, ICCSA 2017 ; Conference date: 02-07-2017 Through 05-07-2017",
year = "2017",
doi = "10.1007/978-3-319-62404-4_28",
language = "English",
isbn = "978-3-319-62403-7",
series = "Lecture Notes in Computer Science",
publisher = "Springer Nature",
pages = "387--398",
booktitle = "Computational Science and Its Applications – ICCSA 2017",
address = "Germany",
}
RIS
TY - GEN
T1 - Distributed Data Processing on Microcomputers with Ascheduler and Apache Spark
AU - Korkhov, Vladimir
AU - Gankevich, Ivan
AU - Iakushkin, Oleg
AU - Gushchanskiy, Dmitry
AU - Khmel, Dmitry
AU - Ivashchenko, Andrey
AU - Pyayt, Alexander
AU - Zobnin, Sergey
AU - Loginov, Alexander
N1 - Conference code: 17
PY - 2017
Y1 - 2017
N2 - Modern architectures of data acquisition and processing often consider low-cost and low-power devices that can be bound together to form a distributed infrastructure. In this paper we overview possibilities to organize a distributed computing testbed based on microcomputers similar to Raspberry Pi and Intel Edison. The goal of the research is to investigate and develop a scheduler for orchestrating distributed data processing and general purpose computations on such unreliable and resource-constrained hardware. Also we consider integration of the scheduler with well-known distributed data processing framework Apache Spark. We outline the project carried out in collaboration with Siemens LLC to compare different configurations of the hardware and software deployment and evaluate performance and applicability of the tools to the testbed.
AB - Modern architectures of data acquisition and processing often consider low-cost and low-power devices that can be bound together to form a distributed infrastructure. In this paper we overview possibilities to organize a distributed computing testbed based on microcomputers similar to Raspberry Pi and Intel Edison. The goal of the research is to investigate and develop a scheduler for orchestrating distributed data processing and general purpose computations on such unreliable and resource-constrained hardware. Also we consider integration of the scheduler with well-known distributed data processing framework Apache Spark. We outline the project carried out in collaboration with Siemens LLC to compare different configurations of the hardware and software deployment and evaluate performance and applicability of the tools to the testbed.
KW - Microcomputers
KW - scheduling
KW - Apache Spark
KW - Raspberry Pi
KW - Fault tolerance
KW - High availability
UR - https://www.scopus.com/inward/record.uri?eid=2-s2.0-85026766337&doi=10.1007%2f978-3-319-62404-4_28&partnerID=40&md5=00b89ba048825a6c725d24ee622b3625
U2 - 10.1007/978-3-319-62404-4_28
DO - 10.1007/978-3-319-62404-4_28
M3 - Conference contribution
SN - 978-3-319-62403-7
T3 - Lecture Notes in Computer Science
SP - 387
EP - 398
BT - Computational Science and Its Applications – ICCSA 2017
PB - Springer Nature
T2 - 17th International Conference on Computational Science and Its Applications, ICCSA 2017
Y2 - 2 July 2017 through 5 July 2017
ER -
