BENCHMARKING OF WEB SERVICES PLATTFORMS - An Evaluation with the TPC-App Benchmark

Daniel F. García, Javier García, Manuel García, Ivan Peteira, Rodrigo García, Pablo Valledor

2006

Abstract

Web services are becoming an essential technology for the development of current distributed applications. Therefore, the organizations must be aware of the possibilities and limitations of the web services and their enabling technologies related to interoperability, performance, security, etc. Benchmarking techniques can provide very useful insights about which technologies are viable and what are the current limitations of the available implementations of those technologies. Furthermore, well established benchmarks provide a way to carry out useful comparisons between two or more implementations. In this paper we present several issues of how web service implementations could be benchmarked. We describe the TPC-App benchmark and explain the most interesting issues of our implementations. Finally we present benchmarking results for the two predominant development platforms, .NET and J2EE.

References

  1. Doculabs, 2003. @Bench web services performance benchmark study. Technical Report of Doculabs.
  2. The Middleware Company, 2002. J2EE and .NET application server and web services benchmark. Technical Report of The Middleware Company.
  3. García, D. F., García, J., 2003. TPC-W E-Commerce benchmark evaluation. IEEE Computer, February 2003, pag 42-48.
  4. Menascé, D., 2003. QoS issues in web services. In IEEE Internet Computing, Nov-Dec 2003, pag 72-75.
  5. Microsoft, 2001. Implementing Sun's Java PetStore using Microsoft .NET. Report of Microsoft Corporation. http://www.gotdotnet.com/team/compare/veritest.aspx
  6. Microsoft, 2004. Web services performance: Comparing Java2EE and .NET framework. A response to Sun Microsystems benchmark. http://www.theserverside.net/articles/showarticle.tss?id=Sun BenchmarkResponse
  7. Subramanyam, S., et al. Designing a web services benchmark. In ICWS'03. International Conference on Web Services, Las Vegas, USA.
  8. Sun, 2001. Java Pet Store. White Paper of Sun. http://java.sun.com/developer/releases/petstore/.
  9. Sun, 2004. Web services performance: Comparing Java2EE and .NET framework. White Paper of Sun. http://java.sun.com/performance/reference/whitepapers/
  10. TPC, 2004. TPC Benchmark App (Application Server) specification, version 1.0. Transaction Processing Performance Council. http://www.tpc.org/tpc_app/.
Download


Paper Citation


in Harvard Style

F. García D., García J., García M., Peteira I., García R. and Valledor P. (2006). BENCHMARKING OF WEB SERVICES PLATTFORMS - An Evaluation with the TPC-App Benchmark . In Proceedings of WEBIST 2006 - Second International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-972-8865-46-7, pages 75-80. DOI: 10.5220/0001239600750080


in Bibtex Style

@conference{webist06,
author={Daniel F. García and Javier García and Manuel García and Ivan Peteira and Rodrigo García and Pablo Valledor},
title={BENCHMARKING OF WEB SERVICES PLATTFORMS - An Evaluation with the TPC-App Benchmark},
booktitle={Proceedings of WEBIST 2006 - Second International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2006},
pages={75-80},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001239600750080},
isbn={978-972-8865-46-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of WEBIST 2006 - Second International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - BENCHMARKING OF WEB SERVICES PLATTFORMS - An Evaluation with the TPC-App Benchmark
SN - 978-972-8865-46-7
AU - F. García D.
AU - García J.
AU - García M.
AU - Peteira I.
AU - García R.
AU - Valledor P.
PY - 2006
SP - 75
EP - 80
DO - 10.5220/0001239600750080