COST OPTIMIZATION FOR DISTRIBUTED BIG DATA ANALYTICS

Sri Vasavi College, Erode Self-Finance Wing, 3rd February 2017. National Conference on Computer and Communication, NCCC’17. International Journal of Computer Science (IJCS) Published by SK Research Group of Companies (SKRGC)

Abstract

The explosive growth of demands on big data processing imposes a heavy burden on computation, storage, and communication in data centers, which hence incurs considerable operational expenditure to data center providers. Therefore, cost minimization has become an emergent issue for the upcoming big data era. As a result, three factors, i.e., task assignment, data placement, and data movement, deeply in the operational expenditure of data centers. In this paper, the cost minimization problem via a joint optimization of these three factors for big data services in geo-distributed data centers considered. To describe the task completion time with the consideration of both data transmission and computation, to propose a 2-D Markov chain and derive the average task completion time in closed-form.

References

[1] Data Center Locations [Online]. Available:https://www.google.com/about/datacenters/inside/locations/index.html. (2013).

[2] R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, and X. Zhu, ``No` power' struggles: Coordinated multi-level power management for the data center,'' in Proc. 13th Int. Conf. Archit. Support Program. Lang. Oper. Syst.,2008, pp. 48_59.

[3] L. Rao, X. Liu, L. Xie, and W. Liu, ``Minimizing electricity cost: Optimization of distributed internet data centers in a multi-electricity-market environment,'' in Proc. 29th Int. Conf. Comput. Commun., 2010, pp. 1_9.

[4] Z. Liu, M. Lin, A. Wierman, S. H. Low, and L. L. Andrew, ``Greening geographical load balancing,'' in Proc. Int. Conf. Meas. Model. Comput.Syst., 2011, pp. 233_244.

[5] R. Urgaonkar, B. Urgaonkar, M. J. Neely, and A. Sivasubramaniam, ``Optimal power cost management using stored energy in data centers,'' in Proc.Int. Conf. Meas. Model. Comput. Syst., 2011, pp. 221_232.

[6] H. Xu, C. Feng, and B. Li, ``Temperature aware workload management in geo-distributed datacenters,'' in Proc. Int. Conf. Meas. Model. Comput. Syst., 2013, pp. 33_36.

 [7] J. Dean and S. Ghemawat, ``Mapreduce: Simpli_ed data processing on large clusters,'' Commun. ACM, vol. 51, no. 1, pp. 107_113, 2008. 322 VOLUME 2, NO. 3, SEPTEMBER 2014

[8] S. A. Yazd, S. Venkatesan, and N. Mittal, ``Boosting energy efficiency with mirrored data block replication policy and energy scheduler,'' SIGOPS Oper. Syst. Rev., vol. 47, no. 2, pp. 33_40, 2013.

[9] I. Marshall and C. Roadknight, ``Linking cache performance to user behaviour,'' Comput. Netw. ISDN Syst., vol. 30, no. 223, pp. 2123_2130, 1998.

[10] H. Jin et al., ``Joint host-network optimization for energy-efficient data center networking,'' in Proc. 27th Int. Symp. Parallel Distrib. Process., 2013, pp. 623_634.

[11] A. Qureshi, R.Weber, H. Balakrishnan, J. Guttag, and B. Maggs, ``Cutting the electric bill for internet-scale systems,'' in Proc. ACM Special Interest Group Data Commun., 2009, pp. 123_134.

[12] X. Fan, W.-D. Weber, and L. A. Barroso, ``Power provisioning for a warehouse-sized computer,'' in Proc. 34th Annu. ISCA, 2007, pp. 13_23.

[13] S. Govindan, A. Sivasubramaniam, and B. Urgaonkar, ``Benefits and limitations of tapping into stored energy for datacenters,'' in Proc. 38th Annu. ISCA, 2011, pp. 341_352.

[14] P. X. Gao, A. R. Curtis, B. Wong, and S. Keshav, ``It's not easy being Green,'' in Proc. ACM SIGCOMM, 2012, pp. 211_222.

[15] Z. Liu et al., ``Renewable and cooling aware workload management for sustainable data centers,'' in Proc. Int. Conf. Meas. Model. Comput. Syst., 2012, pp. 175_186.

Keywords

Big data, data flow, data placement, distributed data centers, cost optimization, job assignment.