Distribution Data Across Multiple Cloud Storage using Reinforcement Learning Method

Abdullah Algarni, Daniel Kudenko


Storing data on a single cloud storage service may cause several potential problems for the data owner such as service continuity, availability, performance, security, and the risk of vendor lock-in. A promising solution to tackle some of these issues is to distribute the data across multiple cloud storage services (MCSS). However, the distinguishing characteristics of different cloud providers, in terms of pricing schemes and service performance, make it difficult to optimise the cost and the performance concurrently on MCSS. This paper proposes a framework for automatically tuning the data distribution policies across MCSS from the client side based on file access patterns. The aim of this work is to optimise the average cost and the average service performance (mainly latency time) on MCSS. To achieve this goal, two different machine learning algorithms are used in this work: (1) supervised learning to predict file access patterns, and (2) reinforcement learning to control data distribution parameters based on the prediction of file access pattern. The framework was tested on a cloud storage emulator, where its was set to act like several common cloud storage services. The result of testing this framework shows a significant improvement in the cost and performance of storing data in multiple clouds, as compared to the commonly used uniform file distribution.


  1. AlZain, M., Soh, B., and Pardede, E. (2011). Mcdb: Using multi-clouds to ensure security in cloud computing. In Dependable, Autonomic and Secure Computing (DASC), 2011 IEEE Ninth International Conference on, pages 784-791. IEEE.
  2. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., and Zaharia, M. (2010). A view of cloud computing. Commun. ACM, 53:50-58.
  3. Bessani, A., Correia, M., Quaresma, B., Andr, F., and Sousa, P. (2011). Depsky: Dependable and secure storage in a cloud-of-clouds. In Proceedings of the Sixth Conference on Computer Systems, EuroSys 7811, pages 31-46, New York, NY, USA. ACM.
  4. Bort, J. (2016). Google apologizes for cloud outage that one person describes as a comedy of errors.
  5. Bowers, K. D., Juels, A., and Oprea, A. (2009). Hail: A high-availability and integrity layer for cloud storage. In Proceedings of the 16th ACM Conference on Computer and Communications Security, pages 187-198, New York, NY, USA. ACM.
  6. Brinkmann, M. (2016). Copy cloud storage services life ends on may 1, 2016. http://www.ghacks.net/2016/02/02/copy-cloudstorage-services-life-ends-on-may-1-201. Online; accessed 19-10-2016.
  7. Buyya, R., Cortes, T., and Jin, H. (2002). A Case for Redundant Arrays of Inexpensive Disks (RAID), pages 2-14. Wiley-IEEE Press.
  8. Furht, B. (2010). Cloud Computing Fundamentals, chapter 1. Springer.
  9. Gatti, C. (2015). Design of Experiments for Reinforcement Learning. Springer, Heidelberg New York Dordrecht London.
  10. Hasselt, H. (2012). Reinforcement Learning in Continuous State And action Spaces, pages 207-251. Springer, Heidelberg New York Dordrecht London.
  11. Jclouds, A. (2016). The java multi-cloud toolkit. http://jclouds.apache.org. Online; accessed 08-07- 2014.
  12. Konda, V. R. and Tsitsiklis, J. N. (2003). On actor-critic algorithms. SIAM J. Control Optim., 42(4):1143-1166.
  13. Libclouds, A. (2016). One interface to rule them all. https://libcloud.apache.org. Online; accessed 08-07- 2014.
  14. LYNN, S. (2014). Raid levels explained.
  15. Marshall, D. (2013). Cloud storage provider nirvanix is closing its doors. http://www.infoworld.com/article/2612299/cloudstorage/cloud- storage-provider-nirvanix-is-closingits-doors.html. Online; accessed 19-10-2016.
  16. Mu, S., Chen, K., Gao, P., Ye, F., Wu, Y., and Zheng, W. (2012). libcloud: Providing high available and uniform accessing to multiple cloud storages. In Grid Computing (GRID), 2012 ACM/IEEE 13th International Conference on, pages 201-208, Beijing. IEEE.
  17. Papaioannou, T., Bonvin, N., and Aberer, K. (2012). Scalia: An adaptive scheme for efficient multi-cloud storage. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference, pages 1-10. ACM.
  18. Rebello, J. (2012). Subscriptions to cloud storage services to reach half-billion level this year.
  19. Shamir, A. (1979). How to share a secret. Commun. ACM, 22:612-613.
  20. Solomon, M. G., Kim, D., and Carrell, J. L. (2014). Fundamentals Of Communications And Networking. Jones and Bartlett Publishers, Inc., USA, 2nd edition.
  21. Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: an Introduction. MIT Press.
  22. Thakur, N. and Lead, Q. (2010). Performance testing in cloud. Citeseer A pragmatic approach.
  23. Tsidulko, J. (2015). Overnight aws outage reminds world how important aws stability really is. CRN. Online; accessed 01-06-2016.
  24. Whiteson, S. (2012). Evolutionary Computation for Reinforcement Learning, pages 325-355. Springer, Heidelberg New York Dordrecht London.
  25. Zhou, A. C., He, B., Cheng, X., and Lau, C. T. (2015). A declarative optimization engine for resource provisioning of scientific workflows in iaas clouds. In Proceedings of the 24th International Symposium on High-Performance Parallel and Distributed Computing, HPDC 7815, pages 223-234, New York, NY, USA. ACM.

Paper Citation

in Harvard Style

Algarni A. and Kudenko D. (2017). Distribution Data Across Multiple Cloud Storage using Reinforcement Learning Method . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-220-2, pages 431-438. DOI: 10.5220/0006124804310438

in Bibtex Style

author={Abdullah Algarni and Daniel Kudenko},
title={Distribution Data Across Multiple Cloud Storage using Reinforcement Learning Method},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},

in EndNote Style

JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Distribution Data Across Multiple Cloud Storage using Reinforcement Learning Method
SN - 978-989-758-220-2
AU - Algarni A.
AU - Kudenko D.
PY - 2017
SP - 431
EP - 438
DO - 10.5220/0006124804310438