Standard
Factory: Master node high-availability for Big Data applications and beyond. / Gankevich, Ivan; Tipikin, Yuri; Korkhov, Vladimir; Gaiduchok, Vladimir; Degtyarev, Alexander; Bogdanov, Alexander.
Computational Science and Its Applications – ICCSA 2016: 16th International Conference, Beijing, China, July 4-7, 2016, Proceedings, Part II. Springer Nature, 2016. p. 379-389 (Lecture Notes in Computer Science; Vol. 9787).
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Research › peer-review
Harvard
Gankevich, I, Tipikin, Y, Korkhov, V, Gaiduchok, V
, Degtyarev, A & Bogdanov, A 2016,
Factory: Master node high-availability for Big Data applications and beyond. in
Computational Science and Its Applications – ICCSA 2016: 16th International Conference, Beijing, China, July 4-7, 2016, Proceedings, Part II. Lecture Notes in Computer Science, vol. 9787, Springer Nature, pp. 379-389, 16th International Conference on Computational Science and Its Applications, Beijing, China,
4/07/16.
https://doi.org/10.1007/978-3-319-42108-7_29
APA
Gankevich, I., Tipikin, Y., Korkhov, V., Gaiduchok, V.
, Degtyarev, A., & Bogdanov, A. (2016).
Factory: Master node high-availability for Big Data applications and beyond. In
Computational Science and Its Applications – ICCSA 2016: 16th International Conference, Beijing, China, July 4-7, 2016, Proceedings, Part II (pp. 379-389). (Lecture Notes in Computer Science; Vol. 9787). Springer Nature.
https://doi.org/10.1007/978-3-319-42108-7_29
Vancouver
Gankevich I, Tipikin Y, Korkhov V, Gaiduchok V
, Degtyarev A, Bogdanov A.
Factory: Master node high-availability for Big Data applications and beyond. In Computational Science and Its Applications – ICCSA 2016: 16th International Conference, Beijing, China, July 4-7, 2016, Proceedings, Part II. Springer Nature. 2016. p. 379-389. (Lecture Notes in Computer Science).
https://doi.org/10.1007/978-3-319-42108-7_29
Author
Gankevich, Ivan ; Tipikin, Yuri ; Korkhov, Vladimir ; Gaiduchok, Vladimir
; Degtyarev, Alexander ; Bogdanov, Alexander. /
Factory: Master node high-availability for Big Data applications and beyond. Computational Science and Its Applications – ICCSA 2016: 16th International Conference, Beijing, China, July 4-7, 2016, Proceedings, Part II. Springer Nature, 2016. pp. 379-389 (Lecture Notes in Computer Science).
BibTeX
@inproceedings{d17d3f4aa7634b9697898178185f2f58,
title = "Factory: Master node high-availability for Big Data applications and beyond",
abstract = "Master node fault-tolerance is the topic that is often dimmed in the discussion of big data processing technologies. Although failure of a master node can take down the whole data processing pipeline, this is considered either improbable or too difficult to encounter. The aim of the studies reported here is to propose rather simple technique to deal with master-node failures. This technique is based on temporary delegation of master role to one of the slave nodes and transferring updated state back to the master when one step of computation is complete. That way the state is duplicated and computation can proceed to the next step regardless of a failure of a delegate or the master (but not both). We run benchmarks to show that a failure of a master is almost “invisible” to other nodes, and failure of a delegate results in recomputation of only one step of data processing pipeline. We believe that the technique can be used not only in Big Data processing but in other types of applications.",
keywords = "Parallel computing, Big data processing, Distributed computing, Backup node, State transfer, Delegation, Cluster computing, Fault-tolerance",
author = "Ivan Gankevich and Yuri Tipikin and Vladimir Korkhov and Vladimir Gaiduchok and Alexander Degtyarev and Alexander Bogdanov",
note = "Gankevich I., Tipikin Y., Korkhov V., Gaiduchok V., Degtyarev A., Bogdanov A. (2016) Factory: Master Node High-Availability for Big Data Applications and Beyond. In: Gervasi O. et al. (eds) Computational Science and Its Applications – ICCSA 2016. ICCSA 2016. Lecture Notes in Computer Science, vol 9787. Springer, Cham. https://doi.org/10.1007/978-3-319-42108-7_29; 16th International Conference on Computational Science and Its Applications ; Conference date: 04-07-2016 Through 06-07-2016",
year = "2016",
doi = "10.1007/978-3-319-42108-7_29",
language = "English",
isbn = "978-3-319-42107-0",
series = "Lecture Notes in Computer Science",
publisher = "Springer Nature",
pages = "379--389",
booktitle = "Computational Science and Its Applications – ICCSA 2016",
address = "Germany",
}
RIS
TY - GEN
T1 - Factory: Master node high-availability for Big Data applications and beyond
AU - Gankevich, Ivan
AU - Tipikin, Yuri
AU - Korkhov, Vladimir
AU - Gaiduchok, Vladimir
AU - Degtyarev, Alexander
AU - Bogdanov, Alexander
N1 - Conference code: 16
PY - 2016
Y1 - 2016
N2 - Master node fault-tolerance is the topic that is often dimmed in the discussion of big data processing technologies. Although failure of a master node can take down the whole data processing pipeline, this is considered either improbable or too difficult to encounter. The aim of the studies reported here is to propose rather simple technique to deal with master-node failures. This technique is based on temporary delegation of master role to one of the slave nodes and transferring updated state back to the master when one step of computation is complete. That way the state is duplicated and computation can proceed to the next step regardless of a failure of a delegate or the master (but not both). We run benchmarks to show that a failure of a master is almost “invisible” to other nodes, and failure of a delegate results in recomputation of only one step of data processing pipeline. We believe that the technique can be used not only in Big Data processing but in other types of applications.
AB - Master node fault-tolerance is the topic that is often dimmed in the discussion of big data processing technologies. Although failure of a master node can take down the whole data processing pipeline, this is considered either improbable or too difficult to encounter. The aim of the studies reported here is to propose rather simple technique to deal with master-node failures. This technique is based on temporary delegation of master role to one of the slave nodes and transferring updated state back to the master when one step of computation is complete. That way the state is duplicated and computation can proceed to the next step regardless of a failure of a delegate or the master (but not both). We run benchmarks to show that a failure of a master is almost “invisible” to other nodes, and failure of a delegate results in recomputation of only one step of data processing pipeline. We believe that the technique can be used not only in Big Data processing but in other types of applications.
KW - Parallel computing
KW - Big data processing
KW - Distributed computing
KW - Backup node
KW - State transfer
KW - Delegation
KW - Cluster computing
KW - Fault-tolerance
UR - https://www.scopus.com/inward/record.uri?eid=2-s2.0-84978842943&doi=10.1007%2f978-3-319-42108-7_29&partnerID=40&md5=d68b10e23ed672f877d643e18e4b5d9e
U2 - 10.1007/978-3-319-42108-7_29
DO - 10.1007/978-3-319-42108-7_29
M3 - Conference contribution
SN - 978-3-319-42107-0
T3 - Lecture Notes in Computer Science
SP - 379
EP - 389
BT - Computational Science and Its Applications – ICCSA 2016
PB - Springer Nature
T2 - 16th International Conference on Computational Science and Its Applications
Y2 - 4 July 2016 through 6 July 2016
ER -
