A Hybrid Framework for Resource-Efficient Query Processing by Effective Utilization of Existing Resources

Mayank Patel, Minal Bhise

2024

Abstract

Scientific experiments and contemporary applications generate substantial volumes of data daily, posing a challenge for traditional database management systems (DBMS) that expend considerable time and resources on data loading. In-situ engines offer a distinct advantage by enabling immediate querying on raw data. Re-searchers have observed that resources are often underutilized during data loading. In contrast, in-situ engines spend ample time and resources in reparsing required data multiple times. Allocating query specific resources is another challenging task that must be addressed to reduce overall workload execution time and resource utilization. This research paper introduces a novel approach called the Resource Availability & Workload-aware Hybrid Framework (RAW-HF), designed to enhance the efficiency of data querying by judiciously utilizing optimal resources in systems comprising an in-situ engine and DBMS. RAW-HF incorporates modules that facilitate the optimization of resources necessary for executing a given workload, striving to maximize the utilization of available resources. The effectiveness of RAW-HF is demonstrated using the scientific dataset Sloan Digital Sky Survey (SDSS) and Linked Observation data (LOD). Comparative analysis with the state-of-the-art workload-aware partial loading technique (WA) reveals that RAW-HF excels in allocating query-specific resources and implementing resource-aware task scheduling. Results from the study indicate that RAW-HF outperforms WA, reducing workload execution time by 26%. It also reduces CPU and IO resource utilization by 26% and 25% compared to WA at a cost of 33% additional RAM.

Paper Citation

in Harvard Style

Patel M. and Bhise M. (2024). A Hybrid Framework for Resource-Efficient Query Processing by Effective Utilization of Existing Resources. In Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 1: ICEIS; ISBN 978-989-758-692-7, SciTePress, pages 337-344. DOI: 10.5220/0012691600003690

in Bibtex Style

@conference{iceis24,
author={Mayank Patel and Minal Bhise},
title={A Hybrid Framework for Resource-Efficient Query Processing by Effective Utilization of Existing Resources},
booktitle={Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 1: ICEIS},
year={2024},
pages={337-344},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012691600003690},
isbn={978-989-758-692-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 1: ICEIS
TI - A Hybrid Framework for Resource-Efficient Query Processing by Effective Utilization of Existing Resources
SN - 978-989-758-692-7
AU - Patel M.
AU - Bhise M.
PY - 2024
SP - 337
EP - 344
DO - 10.5220/0012691600003690
PB - SciTePress