DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS

Adam Barker, Rajkumar Buyya

2011

Abstract

Service-oriented workflows in the scientific domain are commonly composed as Directed Acyclic Graphs (DAGs), formed from a collection of vertices and directed edges. When orchestrating service-oriented DAGs, intermediate data are typically routed through a single centralised engine, which results in unnecessary data transfer, increasing the execution time of a workflow and causing the engine to become a performance bottleneck. This paper introduces an architecture for deploying and executing a service-oriented DAG-based workflows across a peer-to-peer proxy network. A workflow is divided into a set of vertices, disseminated to a group of proxies and executed without centralised control over a peer-to-peer proxy network. Through a Web services implementation, we demonstrate across PlanetLab that by reducing intermediate data transfer and by sharing the workload between distributed proxies, end-to-end workflows are sped up. Furthermore, our architecture is non-intrusive: Web services owned and maintained by different institutions do not have to be altered prior to execution.

References

  1. Allan, P., Bentley, B., and et al. (2001 [26/02/2010]). AstroGrid. Technical report, Available at: www.astrogrid.org.
  2. Allan, P., Bentley, B., and et al. (2001 [26/02/2010]). AstroGrid. Technical report, Available at: www.astrogrid.org.
  3. Apache Axis. http://ws.apache.org/axis [22/02/2011].
  4. Apache Axis. http://ws.apache.org/axis [22/02/2011].
  5. Barker, A. and van Hemert, J. (2008). Scientific Workflow: A Survey and Research Directions. In Wyrzykowski, R. and et al., editors, Seventh International Conference on Parallel Processing and Applied Mathematics, Revised Selected Papers, volume 4967 of LNCS, pages 746-753. Springer.
  6. Barker, A. and van Hemert, J. (2008). Scientific Workflow: A Survey and Research Directions. In Wyrzykowski, R. and et al., editors, Seventh International Conference on Parallel Processing and Applied Mathematics, Revised Selected Papers, volume 4967 of LNCS, pages 746-753. Springer.
  7. Barker, A., Weissman, J. B., and van Hemert, J. (2008a). Eliminating the Middle Man: Peer-to-Peer Dataflow. In HPDC 7808: Proceedings of the 17th International Symposium on High Performance Distributed Computing, pages 55-64. ACM.
  8. Barker, A., Weissman, J. B., and van Hemert, J. (2008a). Eliminating the Middle Man: Peer-to-Peer Dataflow. In HPDC 7808: Proceedings of the 17th International Symposium on High Performance Distributed Computing, pages 55-64. ACM.
  9. Barker, A., Weissman, J. B., and van Hemert, J. (2008b). Orchestrating Data-Centric Workflows. In The 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), pages 210-217. IEEE Computer Society.
  10. Barker, A., Weissman, J. B., and van Hemert, J. (2008b). Orchestrating Data-Centric Workflows. In The 8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid), pages 210-217. IEEE Computer Society.
  11. Bertin, E. and Arnouts, S. Sextractor: Software for source extraction, Astronomy and Astrophysics, Suppl. Ser., 117:393-404, 1996.
  12. Bertin, E. and Arnouts, S. Sextractor: Software for source extraction, Astronomy and Astrophysics, Suppl. Ser., 117:393-404, 1996.
  13. Binder, W., Constantinescu, I., and Faltings, B. (2006). Decentralized Ochestration of Composite Web Services. In Proceedings of ICWS'06, pages 869-876. IEEE Computer Society.
  14. Binder, W., Constantinescu, I., and Faltings, B. (2006). Decentralized Ochestration of Composite Web Services. In Proceedings of ICWS'06, pages 869-876. IEEE Computer Society.
  15. Chafle, G. B., Chandra, S., Mann, V., and Nanda, M. G. (2004). Decentralized orchestration of composite web services. In Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pages 134-143. ACM.
  16. Chafle, G. B., Chandra, S., Mann, V., and Nanda, M. G. (2004). Decentralized orchestration of composite web services. In Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pages 134-143. ACM.
  17. Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., and Bowman, M. (2003). Planetlab: an overlay testbed for broad-coverage services. SIGCOMM Comput. Commun. Rev., 33(3):3-12.
  18. Chun, B., Culler, D., Roscoe, T., Bavier, A., Peterson, L., Wawrzoniak, M., and Bowman, M. (2003). Planetlab: an overlay testbed for broad-coverage services. SIGCOMM Comput. Commun. Rev., 33(3):3-12.
  19. www.cs.wisc.edu/condor/dagman Condor Team. [22/02/2011].
  20. www.cs.wisc.edu/condor/dagman Condor Team. [22/02/2011].
  21. Deelman, E. and et al. (2006). Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance tracking: The CyberShake Example. In Proceedings of the Second IEEE International Conference on e-Science and Grid Computing.
  22. Deelman, E. and et al. (2006). Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance tracking: The CyberShake Example. In Proceedings of the Second IEEE International Conference on e-Science and Grid Computing.
  23. Hollingsworth, D. (1995). The Workflow Reference Model. Workflow Management Coalition.
  24. Hollingsworth, D. (1995). The Workflow Reference Model. Workflow Management Coalition.
  25. Liu, D., Law, K. H., and Wiederhold, G. (2002). Dataflow Distribution in FICAS Service Composition Infrastructure. In Proceedings of the 15th International Conference on Parallel and Distributed Computing Systems.
  26. Liu, D., Law, K. H., and Wiederhold, G. (2002). Dataflow Distribution in FICAS Service Composition Infrastructure. In Proceedings of the 15th International Conference on Parallel and Distributed Computing Systems.
  27. Martin, D., Wutke, D., and Leymann, F. (2008). A Novel Approach to Decentralized Workflow Enactment. EDOC 7808. 12th International IEEE Conference on Enterprise Distributed Object Computing, pages 127-136.
  28. Martin, D., Wutke, D., and Leymann, F. (2008). A Novel Approach to Decentralized Workflow Enactment. EDOC 7808. 12th International IEEE Conference on Enterprise Distributed Object Computing, pages 127-136.
  29. Oinn, T. and et al (2004). Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics, 20(17):3045-3054.
  30. Oinn, T. and et al (2004). Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics, 20(17):3045-3054.
  31. Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. (2001). Chord: A scalable peer-topeer lookup service for internet applications. In SIGCOMM 7801: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, pages 149-160, New York, NY, USA. ACM.
  32. Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., and Balakrishnan, H. (2001). Chord: A scalable peer-topeer lookup service for internet applications. In SIGCOMM 7801: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, pages 149-160, New York, NY, USA. ACM.
  33. Sulakhe, D., Rodriguez, A., Wilde, M., Foster, I. T., and Maltsev, N. (2008). Interoperability of GADU in Using Heterogeneous Grid Resources for Bioinformatics Applications. IEEE Transactions on Information Technology in Biomedicine, 12(2):241-246.
  34. Sulakhe, D., Rodriguez, A., Wilde, M., Foster, I. T., and Maltsev, N. (2008). Interoperability of GADU in Using Heterogeneous Grid Resources for Bioinformatics Applications. IEEE Transactions on Information Technology in Biomedicine, 12(2):241-246.
  35. Taylor, I., Shields, M., Wang, I., and Philp, R. (2003). Distributed P2P Computing within Triana: A Galaxy Visualization Test Case. In 17th International Parallel and Distributed Processing Symposium (IPDPS 2003), pages 16-27. IEEE Computer Society.
  36. Taylor, I., Shields, M., Wang, I., and Philp, R. (2003). Distributed P2P Computing within Triana: A Galaxy Visualization Test Case. In 17th International Parallel and Distributed Processing Symposium (IPDPS 2003), pages 16-27. IEEE Computer Society.
  37. Taylor, I. J., Deelman, E., Gannon, D. B., and Shields, M., editors (2006). Workflows for e- Science: Scientific Workflows for Grids. Springer-Verlag.
  38. Taylor, I. J., Deelman, E., Gannon, D. B., and Shields, M., editors (2006). Workflows for e- Science: Scientific Workflows for Grids. Springer-Verlag.
  39. Wieland, M., Gorlach, K., Schumm, D., and Leymann, F. (2009). Towards reference passing in web service and workflow-based applications. In Enterprise Distributed Object Computing Conference, 2009. EDOC 7809. IEEE International, pages 109 -118.
  40. Wieland, M., Gorlach, K., Schumm, D., and Leymann, F. (2009). Towards reference passing in web service and workflow-based applications. In Enterprise Distributed Object Computing Conference, 2009. EDOC 7809. IEEE International, pages 109 -118.
Download


Paper Citation


in Harvard Style

Barker A. and Buyya R. (2011). DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS . In Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-8425-52-2, pages 222-231. DOI: 10.5220/0003384302220231


in Harvard Style

Barker A. and Buyya R. (2011). DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS . In Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-8425-52-2, pages 222-231. DOI: 10.5220/0003384302220231


in Bibtex Style

@conference{closer11,
author={Adam Barker and Rajkumar Buyya},
title={DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS},
booktitle={Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2011},
pages={222-231},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003384302220231},
isbn={978-989-8425-52-2},
}


in Bibtex Style

@conference{closer11,
author={Adam Barker and Rajkumar Buyya},
title={DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS},
booktitle={Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2011},
pages={222-231},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003384302220231},
isbn={978-989-8425-52-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS
SN - 978-989-8425-52-2
AU - Barker A.
AU - Buyya R.
PY - 2011
SP - 222
EP - 231
DO - 10.5220/0003384302220231


in EndNote Style

TY - CONF
JO - Proceedings of the 1st International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - DECENTRALISED ORCHESTRATION OF SERVICE-ORIENTED SCIENTIFIC WORKFLOWS
SN - 978-989-8425-52-2
AU - Barker A.
AU - Buyya R.
PY - 2011
SP - 222
EP - 231
DO - 10.5220/0003384302220231