Distributed computing today is in demand in batch data processing tasks, but current solutions that allow to use them in Python either are too specific, or do not guarantee full fault tolerance. As a part of the final qualification work, a high-level Python interface (hereinafter SBN-Python) was developed for the new C++ framework called Subordination, in which the last problem was solved. The interface was implemented as an extension of the Python interpreter to achieve low-level compatibility and adaptation of all functionality. The purpose of this work was to test the possibility of using the new interface on a real case of LLC «Gazpromneft – Digital Solution» and to demonstrate the its core principles. To achieve this goal the current solution was analysed, a new architecture using SBN-Python was thought out and implemented, and eventually, the resulting solution was deployed at the company's facilities. As a result of the work, it turned out that using SBN-Python on a real case also scales performance with the number of nodes in the cluster, gives the ability to process various scenarios of node failure in a limited time, as well as some architectural advantages in organizing calculations. In the future, it is planned to expand the boundaries of the new interface, implementing the possibility of building distributed web services on its basis.
Translated title of the contributionDISTRIBUTED FAULT-TOLERANT COMPUTING WITH SBN-PYTHON ON A REAL COMPANY CASE
Original languageRussian
Title of host publicationProceedings of the 9th International Conference "Distributed Computing and Grid Technologies in Science and Education" (GRID'2021), Dubna, Russia, July 5-9, 2021
PublisherRWTH Aahen University
Pages519-524
StatePublished - 2021
Event9th International Conference "Distributed Computing and Grid Technologies in Science and Education", GRID 2021 - Dubna, Russian Federation
Duration: 5 Jul 20219 Jul 2021
Conference number: 9
https://indico.jinr.ru/event/1086/overview

Publication series

NameCEUR workshop proceedings
Volume3041
ISSN (Electronic)1613-0073

Conference

Conference9th International Conference "Distributed Computing and Grid Technologies in Science and Education", GRID 2021
Abbreviated titleGRID'2021
Country/TerritoryRussian Federation
CityDubna
Period5/07/219/07/21
Internet address

ID: 89536525