Optimizing Storage Efficiency in Hadoop Distribution File System Through Data Deduplication

Amrita Raj, G. Uma Devi

2024

Abstract

The exponential growth of data in modern computing environments poses significant challenges in managing storage resources efficiently. This study addresses the optimization of storage efficiency in the HDFS through strategic utilization of data deduplication process. The primary objective is to improve the overall performance and cost- effectiveness of Hadoop clusters by minimizing storage requirements and maximizing data accessibility. The proposed approach involves the implementation of data deduplication strategies to eliminate redundant copies of information, thereby reducing the overall storage footprint. This research addresses the challenges of managing large datasets in distributed environments, aiming to streamline data storage, minimize redundancy, and mitigate network traffic. Through data deduplication the study seeks to provide a comprehensive approach to optimizing storage resources within the Hadoop ecosystem.

Paper Citation

in Harvard Style

Raj A. and Devi G. (2024). Optimizing Storage Efficiency in Hadoop Distribution File System Through Data Deduplication. In Proceedings of the 1st International Conference on Cognitive & Cloud Computing - Volume 1: IC3Com; ISBN 978-989-758-739-9, SciTePress, pages 13-19. DOI: 10.5220/0013398600004646

in Bibtex Style

@conference{ic3com24,
author={Amrita Raj and G. Uma Devi},
title={Optimizing Storage Efficiency in Hadoop Distribution File System Through Data Deduplication},
booktitle={Proceedings of the 1st International Conference on Cognitive & Cloud Computing - Volume 1: IC3Com},
year={2024},
pages={13-19},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013398600004646},
isbn={978-989-758-739-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Cognitive & Cloud Computing - Volume 1: IC3Com
TI - Optimizing Storage Efficiency in Hadoop Distribution File System Through Data Deduplication
SN - 978-989-758-739-9
AU - Raj A.
AU - Devi G.
PY - 2024
SP - 13
EP - 19
DO - 10.5220/0013398600004646
PB - SciTePress