A Peer-to-Peer Distributed Filesystem with Erasure Encoding

Abstract

Idea of divorcing data from the location is emerged considering the facts of resource management, accessibility, fault tolerance security and many other benefits. Various types of distributed file systems were proposed and developed to achieve these targets. In a distributed file system, a file is stored in many peers. In case of an attack, network failure or any other reason, machines which hold the file may not be accessible and the file becomes unavailable. With mirroring or using threshold schemes, solution of a replication of data can be introduced to overcome this issue. Considering the availability and durability factor it is evident that use of threshold schemes provides better performance. Here, the distributed file system is developed with Reed-Solomon encoding and using the idea of multicasting. The encoded file is fragmented and distributed among the members in the multicast group and when a machine in the group need to retrieve the file it will collect fragments from the members in the multicast group. After collecting sufficient number of fragments, decoding the fragments of the file generates the original file. The evaluation of the prototype shows that the time for encoding the file is independent from the file type. Further it shows, the time to encode varies linearly with the increasing the file size and time per one Kilo Byte.