loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Aleksey Burdakov 1 ; Uriy Grigorev 1 ; Andrey Ploutenko 2 and Oleg Ermakov 1

Affiliations: 1 Bauman Moscow State Technical University, Moscow, Russia ; 2 Amur State University, Blagoveschensk, Russia

Keyword(s): Streaming Processing, Analysis Layer, Sketch, Count-Min Sketch Algorithm.

Abstract: Analysis is an important part of the widely used streaming data processing. The frequency of flow element occurrence and their values sum are calculated during analysis. The algorithms like Count-Min Sketch and others give a big error in restoring the aggregate with a large number of elements. The article proposes application of a vector matrix. Each vector has a length of 'n'. If the number of different elements approaches 'n', then the window size is automatically reduced. This allows accurate storage of the aggregate without element loss. The SELECT operator for searching in a vector array is also proposed. It allows getting various slices of the aggregated data accumulated over the window. The comparison of the developed method with the Count-Min Sketch data processing method in the Analysis Layer was performed. The experiment showed that the method based on the vector matrix more than twice reduces memory consumption. It also ensures the exact SELECT statement execution. An intr oduction of a floating window allows maintaining the calculation accuracy and avoiding losing records from the stream. The same query sketch-based execution error reaches 200%. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.140.186.189

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Burdakov, A.; Grigorev, U.; Ploutenko, A. and Ermakov, O. (2021). Analysis Layer Implementation Method for a Streaming Data Processing System. In Proceedings of the 6th International Conference on Internet of Things, Big Data and Security - IoTBDS; ISBN 978-989-758-504-3; ISSN 2184-4976, SciTePress, pages 262-269. DOI: 10.5220/0010465902620269

@conference{iotbds21,
author={Aleksey Burdakov. and Uriy Grigorev. and Andrey Ploutenko. and Oleg Ermakov.},
title={Analysis Layer Implementation Method for a Streaming Data Processing System},
booktitle={Proceedings of the 6th International Conference on Internet of Things, Big Data and Security - IoTBDS},
year={2021},
pages={262-269},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010465902620269},
isbn={978-989-758-504-3},
issn={2184-4976},
}

TY - CONF

JO - Proceedings of the 6th International Conference on Internet of Things, Big Data and Security - IoTBDS
TI - Analysis Layer Implementation Method for a Streaming Data Processing System
SN - 978-989-758-504-3
IS - 2184-4976
AU - Burdakov, A.
AU - Grigorev, U.
AU - Ploutenko, A.
AU - Ermakov, O.
PY - 2021
SP - 262
EP - 269
DO - 10.5220/0010465902620269
PB - SciTePress

- Science and Technology Publications, Lda.
RESOURCES

Proceedings

Papers

Authors

Ontology

CONTACTS

Science and Technology Publications, Lda
Avenida de S. Francisco Xavier, Lote 7 Cv. C,
2900-616 Setúbal, Portugal.

Phone: +351 265 520 185 (National fixed network call)
Fax: +351 265 520 186
Email: info@scitepress.org

EXTERNAL LINKS

PRIMORIS

INSTICC

SCITEVENTS

CROSSREF

PROCEEDINGS SUBMITTED FOR INDEXATION BY:

dblp

Ei Compendex

SCOPUS

Semantic Scholar

Google Scholar

Microsoft Academic