Multi-dimensional Resource Allocation for Data-intensive Large-scale Cloud Applications

Foued Jrad, Jie Tao, Ivona Brandic, Achim Streit

2014

Abstract

Large scale applications are emerged as one of the important applications in distributed computing. Today, the economic and technical benefits offered by the Cloud computing technology encouraged many users to migrate their applications to Cloud. On the other hand, the variety of the existing Clouds requires them to make decisions about which providers to choose in order to achieve the expected performance and service quality while keeping the payment low. In this paper, we present a multi-dimensional resource allocation scheme to automate the deployment of data-intensive large scale applications in Multi-Cloud environments. The scheme applies a two level approach in which the target Clouds are matched with respect to the Service Level Agreement (SLA) requirements and user payment at first and then the application workloads are distributed to the selected Clouds using a data locality driven scheduling policy. Using an implemented Multi-Cloud simulation environment, we evaluated our approach with a real data-intensive workflow application in different scenarios. The experimental results demonstrate the effectiveness of the implemented matching and scheduling policies in improving the workflow execution performance and reducing the amount and costs of Intercloud data transfers.

References

  1. Asker, J. and Cantillon, E. (2008). Properties of scoring auctions. The RAND Journal of Economics, 39(1):69- 85.
  2. Calheiros, R. N., Ranjan, R., Beloglazov, A., Rose, C. A. F. D., and Buyya, R. (2011). Cloudsim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience, 41(1):23- 50.
  3. Dastjerdi, A. V., Garg, S. K., and Buyya, R. (2011). QoSaware Deployment of Network of Virtual Appliances Across Multiple Clouds. 2011 IEEE Third International Conference on Cloud Computing Technology and Science, pages 415-423.
  4. Dean, J. and Ghemawat, S. (2008). Mapreduce: simplified data processing on large clusters. Commun. ACM, 51(1):107-113.
  5. Deelman, E., Singh, G., Livny, M., Berriman, B., and Good, J. (2008). The cost of doing science on the cloud: The montage example. In Proceedings of the 2008 ACM/IEEE Conference on Supercomputing, SC 7808, pages 50:1-50:12, Piscataway, NJ, USA. IEEE Press.
  6. Fard, H. M., Prodan, R., Barrionuevo, J. J. D., and Fahringer, T. (2012). A Multi-objective Approach for Workflow Scheduling in Heterogeneous Environments. 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012), pages 300-309.
  7. Fard, H. M., Prodan, R., and Fahringer, T. (2013). A Truthful Dynamic Workflow Scheduling Mechanism for Commercial Multicloud Environments. IEEE Transactions on Parallel and Distributed Systems, 24(6):1203-1212.
  8. Freund, R., Gherrity, M., Ambrosius, S., Campbell, M., Halderman, M., Hensgen, D., Keith, E., Kidd, T., Kussow, M., Lima, J., Mirabile, F., Moore, L., Rust, B., and Siegel, H. (1998). Scheduling resources in multi-user, heterogeneous, computing environments with smartnet. In Heterogeneous Computing Workshop, 1998. (HCW 98) Proceedings. 1998 Seventh, pages 184-199.
  9. Jin, J., Luo, J., Song, A., Dong, F., and Xiong, R. (2011). Bar: An efficient data locality driven task scheduling algorithm for cloud computing. In Cluster, Cloud and Grid Computing (CCGrid), 2011 11th IEEE/ACM International Symposium on, pages 295-304.
  10. Jrad, F., Tao, J., Knapper, R., Flath, C. M., and Streit, A. (2013a). A utility-based approach for customised cloud service selection. Int. J. Computational Science and Engineering, in press.
  11. Jrad, F., Tao, J., and Streit, A. (2013b). A broker-based framework for multi-cloud workflows. In Proceedings of the 2013 international workshop on Multi-cloud applications and federated clouds, MultiCloud 7813, pages 61-68, New York, NY, USA. ACM.
  12. Lamparter, S., Ankolekar, S., Grimm, S., and R.Studer (2007). Preference-based Selection of Highly Configurable Web Services. In Proc. of the 16th Int. World Wide Web Conference (WWW'07), pages 1013-1022, Banff, Canada.
  13. Lukasiewycz, M., Glaß, M., Reimann, F., and Teich, J. (2011). Opt4J - A Modular Framework for Metaheuristic Optimization. In Proceedings of the Genetic and Evolutionary Computing Conference (GECCO 2011), pages 1723-1730, Dublin, Ireland.
  14. Mian, R., Martin, P., Brown, A., and Zhang, M. (2011). Managing data-intensive workloads in a cloud. In Fiore, S. and Aloisio, G., editors, Grid and Cloud Database Management, pages 235-258. Springer Berlin Heidelberg.
  15. Oliveira, D., Ocan˜a, K. a. C. S., Baia˜o, F., and Mattoso, M. (2012). A Provenance-based Adaptive Scheduling Heuristic for Parallel Scientific Workflows in Clouds. Journal of Grid Computing, 10(3):521-552.
  16. Pandey, S., Wu, L., Guru, S., and Buyya, R. (2010). A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In Advanced Information Networking and Applications (AINA), 2010 24th IEEE International Conference on, pages 400-407.
  17. Szabo, C., Sheng, Q. Z., Kroeger, T., Zhang, Y., and Yu, J. (2013). Science in the Cloud: Allocation and Execution of Data-Intensive Scientific Workflows. Journal of Grid Computing.
  18. Weiwei, C. and Ewa, D. (2012). Workflowsim: A toolkit for simulating scientific workflows in distributed environments. In The 8th IEEE International Conference on eScience, Chicago. IEEE, IEEE.
  19. Yuan, D., Yang, Y., Liu, X., and Chen, J. (2010). A data placement strategy in scientific cloud workflows. Future Gener. Comput. Syst., 26(8):1200-1214.
  20. Zeng, L., Benatallah, B., Dumas, M., Kalagnanam, J., and Sheng, Q. Z. (2003). Quality driven web services composition. In Proceedings of the 12th International Conference on World Wide Web, WWW 7803, pages 411-421, New York, NY, USA. ACM.
Download


Paper Citation


in Harvard Style

Jrad F., Tao J., Brandic I. and Streit A. (2014). Multi-dimensional Resource Allocation for Data-intensive Large-scale Cloud Applications . In Proceedings of the 4th International Conference on Cloud Computing and Services Science - Volume 1: MultiCloud, (CLOSER 2014) ISBN 978-989-758-019-2, pages 691-702. DOI: 10.5220/0004971906910702


in Bibtex Style

@conference{multicloud14,
author={Foued Jrad and Jie Tao and Ivona Brandic and Achim Streit},
title={Multi-dimensional Resource Allocation for Data-intensive Large-scale Cloud Applications},
booktitle={Proceedings of the 4th International Conference on Cloud Computing and Services Science - Volume 1: MultiCloud, (CLOSER 2014)},
year={2014},
pages={691-702},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004971906910702},
isbn={978-989-758-019-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 4th International Conference on Cloud Computing and Services Science - Volume 1: MultiCloud, (CLOSER 2014)
TI - Multi-dimensional Resource Allocation for Data-intensive Large-scale Cloud Applications
SN - 978-989-758-019-2
AU - Jrad F.
AU - Tao J.
AU - Brandic I.
AU - Streit A.
PY - 2014
SP - 691
EP - 702
DO - 10.5220/0004971906910702