An Efficient Way to Avoid Redundancy Record in Big Data

International Journal of Computer Science (IJCS) Published by SK Research Group of Companies (SKRGC)

Abstract

Distributed computing offers another method for benefit arrangement by re-organizing different assets over the Internet. The most essential and prevalent cloud benefit is information stockpiling. Keeping in mind the end goal to save the security of information holders, information are frequently put away in cloud in an encoded frame. Nonetheless, encoded information present new difficulties for cloud information deduplication, which winds up essential for huge information stockpiling and preparing in cloud. In this thesis, our propose build a hybrid deduplication mechanism named Stream Locality Aware Deduplication to avoid de-duplicate encrypted data stored in cloud based on ownership challenge and proxy re-encryption. This is, unsurprising, due to the digitization of our society. In this research, achieved high security, low error rate and low computational complexity of the scheme for potential practical deployment.

References

[1] J. R. Douceur, A. Adya, W. J. Bolosky, D. Simon, and M. Theimer, ?Reclaiming space from duplicate files in a serverless distributed file system,? in Proc. IEEE Int. Conf. Distrib. Comput. Syst., 2002, pp. 617–624.

[2] G. Wallace, et al., ?Characteristics of backup workloads in production systems,? in Proc. USENIX Conf. File Storage Technol., 2012, pp. 1–16.

[3] G. Ateniese, K. Fu, M. Green, and S. Hohenberger, ?Improved proxy re-encryption schemes with applications to secure distributed storage,? ACM Trans. Inform. Syst. Secur., vol. 9, no. 1, pp. 1–30.

[4] D. T. Meyer and W. J Bolosky, ?A study of practical deduplication,? ACM Trans. Storage, vol. 7, no. 4, pp. 1–20.

[5] J. Pettitt, ?Hash of plaintext as key?? (2016). [Online]. Available: https://cypherpunks.venona.com/date/1996/02/msg02013.html.

[6] M. Bellare, S. Keelveedhi, and T. Ristenpart, ?Message-locked encryption and secure deduplication,? in Proc. Cryptology—EUROCRYPT, 2013, pp. 296–312.

[7] D. Perttula, B. Warner, and Z. Wilcox-O’Hearn, ?Attacks on convergent encryption.? (2016).

[8] C. Y. Liu, X. J. Liu, and L. Wan, ?Policy-based deduplication in secure cloud storage,? in Proc. Trustworthy Comput. Serv., 2013, pp. 250–262.

[9] P. Puzio, R. Molva, M. Onen, and S. Loureiro, ?ClouDedup: Secure deduplication with encrypted data for cloud storage,? in Proc. IEEE Int. Cof. Cloud Comput. Technol. Sci., 2013, pp. 363–370.

[10] Z. Sun, J. Shen, and J. M. Yong, ?DeDu: Building a deduplication storage system over cloud computing,? in Proc. IEEE Int. Conf. Comput. Supported Cooperative Work Des., 2011, pp. 348–355.

[11] Z. C. Wen, J. M. Luo, H. J. Chen, J. X. Meng, X. Li, and J. Li, ?A verifiable data deduplication scheme in cloud computing,? in Proc. Int. Conf. Intell. Netw. Collaborative Syst., 2014, pp. 85–90.

[12] J. Li, Y. K. Li, X. F. Chen, P. P. C. Lee, and W. J. Lou, ?A hybrid cloud approach for secure authorized deduplication,? IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 5, pp. 1206–1216.

[13] P. Meye, P. Raipin, F. Tronel, and E. Anceaume, ?A secure two phase data deduplication scheme,? in Proc. HPCC/CSS/ICESS, 2014, pp. 802–809.

[14] J. Paulo and J. Pereira, ?A survey and classification of storage deduplication systems,? ACM Comput. Surveys, vol. 47, no. 1, pp. 1–30.

[15] Y.-K. Li, M. Xu, C.-H. Ng, and P. P. C. Lee, ?Efficient hybrid inline and out-of-line deduplication for backup storage,? ACM Trans. Storage, vol. 11, no. 1, pp. 2:1-2:21.

[16] M. Fu, et al., ?Accelerating restore and garbage collection in deduplication-based backup systems via exploiting historical information,? in Proc. USENIX Annu. Tech. Conf., 2014, pp. 181–192.

[17] M. Kaczmarczyk, M. Barczynski, W. Kilian, and C. Dubnicki, ?Reducing impact of data fragmentation caused by in-line deduplication,? in Proc. 5th Annu. Int. Syst. Storage Conf., 2012, pp. 15:1–15:12.

[18] M. Lillibridge, K. Eshghi, and D. Bhagwat, ?Improving restore speed for backup systems that use inline chunk-based deduplication,? in Proc.USENIX Conf. File Storage Technol., 2013, pp. 183–198.

[19] L. J. Gao, ?Game theoretic analysis on acceptance of a cloud data access control scheme based on reputation,? M.S. thesis, Xidian University, State Key Lab of ISN, School of Telecommunications Engineering, Xi’an, China, 2015.

Keywords

Deduplication, hybrid mechanism, Stream Locality Aware Deduplication