Experimental Study on Block and Ratio Storage Strategy and Parallel Transmission Algorithms

Authors

Chun-mao Jiang, School of Computer Science and Technology, Harbin Engineering University, Harbin, Heilongjiang 150001, P.R.China
Guo-yin Zhang
Ming-cheng Qu

Author Country (or Countries)

P.R.China

Abstract

Parallel transmission algorithms CLBA and DAS greatly improved the speed of transferring data files, but they cause a large overhead of storage space and network traffic for deployment of duplications of large files, in addition to their poor feasibility of deploying large complete replicas on nodes with limited storage space. In view of Google and Hadoop distributed file systems all adopted block storage mechanism, a parallel data transmission algorithm based on block storage strategy is proposed, but the interrelation between block storage and parallel transmission was not properly taken into consideration and so the nodes of slower bandwidth greatly affected the overall transmission performance. Therefore, we proposed in this paper block and ratio storage strategies for files of massive data in distributed systems and parallel transmission algorithms PTBM and PTRM for the two storage strategies. Experimental results indicate the proposed algorithms are better than CLBA, and close to DAS, and they save more than 50% storage space, and can adapt to the deployment of large data files.

Suggested Reviewers

N/A

Recommended Citation

Jiang, Chun-mao; Zhang, Guo-yin; and Qu, Ming-cheng (2013) "Experimental Study on Block and Ratio Storage Strategy and Parallel Transmission Algorithms," Applied Mathematics & Information Sciences: Vol. 07: Iss. 3, Article 44.
Available at: https://digitalcommons.aaru.edu.jo/amis/vol07/iss3/44