Modern architectures of data acquisition and processing often consider low-cost and low-power devices that can be bound together to form a distributed infrastructure. In this paper we overview possibilities to organize a distributed computing testbed based on microcomputers similar to Raspberry Pi and Intel Edison. The goal of the research is to investigate and develop a scheduler for orchestrating distributed data processing and general purpose computations on such unreliable and resource-constrained hardware. Also we consider integration of the scheduler with well-known distributed data processing framework Apache Spark. We outline the project carried out in collaboration with Siemens LLC to compare different configurations of the hardware and software deployment and evaluate performance and applicability of the tools to the testbed.
Original languageEnglish
Title of host publicationComputational Science and Its Applications – ICCSA 2017
Subtitle of host publication17th International Conference, Trieste, Italy, July 3-6, 2017, Proceedings, Part V
PublisherSpringer Nature
Pages387-398
ISBN (Electronic)978-3-319-62404-4
ISBN (Print)978-3-319-62403-7
DOIs
StatePublished - 2017
Event17th International Conference on Computational Science and Its Applications, ICCSA 2017 - Trieste, Italy
Duration: 2 Jul 20175 Jul 2017
Conference number: 17

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Nature
Volume10408
ISSN (Print)0302-9743

Conference

Conference17th International Conference on Computational Science and Its Applications, ICCSA 2017
Abbreviated titleICCSA 2017
Country/TerritoryItaly
CityTrieste
Period2/07/175/07/17

    Research areas

  • Microcomputers, scheduling, Apache Spark, Raspberry Pi, Fault tolerance, High availability

ID: 71305063