Allan Variance-based Granulation Technique for Large Temporal Databases

Lorina Sinanaj, Hossein Haeri, Liming Gao, Satya Prasad Maddipatla, Cindy Chen, Kshitij Jerath, Craig Beal, Sean Brennan

2021

Abstract

In the era of Big Data, conducting complex data analysis tasks efficiently, becomes increasingly important and challenging due to large amounts of data available. In order to decrease query response time with limited main memory and storage space, data reduction techniques that preserve data quality are needed. Existing data reduction techniques, however, are often computationally expensive and rely on heuristics for deciding how to split or reduce the original dataset. In this paper, we propose an effective granular data reduction technique for temporal databases, based on Allan Variance (AVAR). AVAR is used to systematically determine the temporal window length over which data remains relevant. The entire dataset to be reduced is then separated into granules with size equal to the AVAR-determined window length. Data reduction is achieved by generating aggregated information for each such granule. The proposed method is tested using a large database that contains temporal information for vehicular data. Then comparison experiments are conducted and the outstanding runtime performance is illustrated by comparing with three clustering-based data reduction methods. The performance results demonstrate that the proposed Allan Variance-based technique can efficiently generate reduced representation of the original data without losing data quality, while significantly reducing computation time.

Paper Citation

in Harvard Style

Sinanaj L., Haeri H., Gao L., Maddipatla S., Chen C., Jerath K., Beal C. and Brennan S. (2021). Allan Variance-based Granulation Technique for Large Temporal Databases. In Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021) - Volume 3: KMIS; ISBN 978-989-758-533-3, SciTePress, pages 17-28. DOI: 10.5220/0010651500003064

in Bibtex Style

@conference{kmis21,
author={Lorina Sinanaj and Hossein Haeri and Liming Gao and Satya Prasad Maddipatla and Cindy Chen and Kshitij Jerath and Craig Beal and Sean Brennan},
title={Allan Variance-based Granulation Technique for Large Temporal Databases},
booktitle={Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021) - Volume 3: KMIS},
year={2021},
pages={17-28},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010651500003064},
isbn={978-989-758-533-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2021) - Volume 3: KMIS
TI - Allan Variance-based Granulation Technique for Large Temporal Databases
SN - 978-989-758-533-3
AU - Sinanaj L.
AU - Haeri H.
AU - Gao L.
AU - Maddipatla S.
AU - Chen C.
AU - Jerath K.
AU - Beal C.
AU - Brennan S.
PY - 2021
SP - 17
EP - 28
DO - 10.5220/0010651500003064
PB - SciTePress