Authors:
Dimitri Pertin
1
;
Sylvain David
2
;
Pierre Evenou
2
;
Benoît Parrein
3
and
Nicolas Normand
3
Affiliations:
1
LUNAM Université, Université de Nantes and Fizians SAS, France
;
2
Fizians SAS, France
;
3
LUNAM Université and Université de Nantes, France
Keyword(s):
Distributed File System, RozoFS, Erasure Coding, Mojette Transform, IOzone, Video Editing.
Related
Ontology
Subjects/Areas/Topics:
Cloud Application Scalability and Availability
;
Cloud Applications Performance and Monitoring
;
Cloud Computing
;
Cloud Computing Enabling Technology
;
Cloud Ilities (Scalability, Availability, Reliability)
;
Monitoring of Services, Quality of Service, Service Level Agreements
;
Platforms and Applications
;
Programming Models, Systems, and Fault-Tolerant Computing
;
Services Science
;
Services Security and Reliability
Abstract:
Distributed storage systems take advantage of the network, storage and computational resources to provide a scalable infrastructure. But in such large system, failures are frequent and expected. Data replication is the common technique to provide fault-tolerance but suffers from its important storage consumption. Erasure coding is an alternative that offers the same data protection but reduces significantly the storage consumption. As it entails additional workload, current storage providers limit its use for longterm storage. We present the Mojette Transform (MT), an erasure code whose computations rely on fast XOR operations. The MT is part of RozoFS, a distributed file system that provides a global namespace relying on a cluster of storage nodes. This work is part of our ongoing effort to prove that erasure coding is not necessarily a bottleneck for intense I/O applications. In order to validate our approach, we consider a case study involving a storage cluster of RozoFS that supp
orts video editing as an I/O intensive application.
(More)