An optimistic approach to handle out-of-order events within analytical stream processing

Igor E. Kuralenok, Nikita Marshalkin, Artem Trofimov, Boris Novikov

Research output

Abstract

—In recent years, there has been a growth in research and industrial solutions in the field of distributed stream processing. However, even state-of-the-art stream processing systems experience difficulties with out-of-order data arrival. The most common solution to this issue is buffering. Its main problem is the extra cost for blocking before each order-sensitive operation. The goal of this paper is to propose and evaluate an optimistic approach to handle out-of-order events. We introduce a method that is suitable for any stateful operation and needs a single buffer for the complete computational pipeline. Such technique requires extra network transfers and re-computations, but the experiments demonstrate that a prototype of our approach is able to create low overhead while ensuring the correct ordering.

Original languageEnglish
Pages (from-to)22-29
Number of pages8
JournalCEUR Workshop Proceedings
Volume2135
Publication statusPublished - 1 Jan 2018
Event3rd Conference on Software Engineering and Information Management, SEIM 2018 - Saint Petersburg
Duration: 14 Apr 2018 → …

Fingerprint

Processing
Pipelines
Costs
Experiments

Scopus subject areas

  • Computer Science(all)

Cite this

Kuralenok, I. E., Marshalkin, N., Trofimov, A., & Novikov, B. (2018). An optimistic approach to handle out-of-order events within analytical stream processing. CEUR Workshop Proceedings, 2135, 22-29.
Kuralenok, Igor E. ; Marshalkin, Nikita ; Trofimov, Artem ; Novikov, Boris. / An optimistic approach to handle out-of-order events within analytical stream processing. In: CEUR Workshop Proceedings. 2018 ; Vol. 2135. pp. 22-29.
@article{93248a687c55429292dc84435184929b,
title = "An optimistic approach to handle out-of-order events within analytical stream processing",
abstract = "—In recent years, there has been a growth in research and industrial solutions in the field of distributed stream processing. However, even state-of-the-art stream processing systems experience difficulties with out-of-order data arrival. The most common solution to this issue is buffering. Its main problem is the extra cost for blocking before each order-sensitive operation. The goal of this paper is to propose and evaluate an optimistic approach to handle out-of-order events. We introduce a method that is suitable for any stateful operation and needs a single buffer for the complete computational pipeline. Such technique requires extra network transfers and re-computations, but the experiments demonstrate that a prototype of our approach is able to create low overhead while ensuring the correct ordering.",
author = "Kuralenok, {Igor E.} and Nikita Marshalkin and Artem Trofimov and Boris Novikov",
year = "2018",
month = "1",
day = "1",
language = "English",
volume = "2135",
pages = "22--29",
journal = "CEUR Workshop Proceedings",
issn = "1613-0073",
publisher = "RWTH Aahen University",

}

Kuralenok, IE, Marshalkin, N, Trofimov, A & Novikov, B 2018, 'An optimistic approach to handle out-of-order events within analytical stream processing', CEUR Workshop Proceedings, vol. 2135, pp. 22-29.

An optimistic approach to handle out-of-order events within analytical stream processing. / Kuralenok, Igor E.; Marshalkin, Nikita; Trofimov, Artem; Novikov, Boris.

In: CEUR Workshop Proceedings, Vol. 2135, 01.01.2018, p. 22-29.

Research output

TY - JOUR

T1 - An optimistic approach to handle out-of-order events within analytical stream processing

AU - Kuralenok, Igor E.

AU - Marshalkin, Nikita

AU - Trofimov, Artem

AU - Novikov, Boris

PY - 2018/1/1

Y1 - 2018/1/1

N2 - —In recent years, there has been a growth in research and industrial solutions in the field of distributed stream processing. However, even state-of-the-art stream processing systems experience difficulties with out-of-order data arrival. The most common solution to this issue is buffering. Its main problem is the extra cost for blocking before each order-sensitive operation. The goal of this paper is to propose and evaluate an optimistic approach to handle out-of-order events. We introduce a method that is suitable for any stateful operation and needs a single buffer for the complete computational pipeline. Such technique requires extra network transfers and re-computations, but the experiments demonstrate that a prototype of our approach is able to create low overhead while ensuring the correct ordering.

AB - —In recent years, there has been a growth in research and industrial solutions in the field of distributed stream processing. However, even state-of-the-art stream processing systems experience difficulties with out-of-order data arrival. The most common solution to this issue is buffering. Its main problem is the extra cost for blocking before each order-sensitive operation. The goal of this paper is to propose and evaluate an optimistic approach to handle out-of-order events. We introduce a method that is suitable for any stateful operation and needs a single buffer for the complete computational pipeline. Such technique requires extra network transfers and re-computations, but the experiments demonstrate that a prototype of our approach is able to create low overhead while ensuring the correct ordering.

UR - http://www.scopus.com/inward/record.url?scp=85050503211&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85050503211

VL - 2135

SP - 22

EP - 29

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

SN - 1613-0073

ER -

Kuralenok IE, Marshalkin N, Trofimov A, Novikov B. An optimistic approach to handle out-of-order events within analytical stream processing. CEUR Workshop Proceedings. 2018 Jan 1;2135:22-29.