Factory: Master node high-availability for Big Data applications and beyond

Результат исследований: Публикации в книгах, отчётах, сборниках, трудах конференцийстатья в сборнике материалов конференции

4 Цитирования (Scopus)


Master node fault-tolerance is the topic that is often dimmed in the discussion of big data processing technologies. Although failure of a master node can take down the whole data processing pipeline, this is considered either improbable or too difficult to encounter. The aim of the studies reported here is to propose rather simple technique to deal with master-node failures. This technique is based on temporary delegation of master role to one of the slave nodes and transferring updated state back to the master when one step of computation is complete. That way the state is duplicated and computation can proceed to the next step regardless of a failure of a delegate or the master (but not both). We run benchmarks to show that a failure of a master is almost “invisible” to other nodes, and failure of a delegate results in recomputation of only one step of data processing pipeline. We believe that the technique can be used not only in Big Data processing but in other types of applications.
Язык оригиналаанглийский
Название основной публикацииComputational Science and Its Applications – ICCSA 2016
Подзаголовок основной публикации16th International Conference, Beijing, China, July 4-7, 2016, Proceedings, Part II
ИздательSpringer Nature
ISBN (электронное издание)978-3-319-42108-7
ISBN (печатное издание)978-3-319-42107-0
СостояниеОпубликовано - 2016
СобытиеInternational Conference on Computational Science and Its Applications - Beijing, Китай
Продолжительность: 4 июл 20166 июл 2016
Номер конференции: 16

Серия публикаций

НазваниеLecture Notes in Computer Science
ИздательSpringer Nature
ISSN (печатное издание)0302-9743


конференцияInternational Conference on Computational Science and Its Applications
Сокращенный заголовокICCSA 2016

Fingerprint Подробные сведения о темах исследования «Factory: Master node high-availability for Big Data applications and beyond». Вместе они формируют уникальный семантический отпечаток (fingerprint).