Mohammed Al-Badawi, Siobhán North, Barry Eaglestone


In the context of benchmarking XML implementations, several XML benchmarks have been produced to either test the application’s overall performance or evaluate individual XML functionalities of a specific XML implementation. Among six popular XML benchmarks investigated in this article, all techniques rely on code-generated datasets which disregard many of XML’s irregular aspects such as varying the depth and breadth of the XML documents’ structure. This paper introduces a new test-model called the “3D XML benchmark” which aims to address these limitations by extending the dataset and query-set of existing XML benchmarks. Our experimental results have shown that XML techniques can perform inconsistently over different XML databases for some query classes, thus justifying the use of an improved benchmark.


  1. Berglund, A., Boag, S., Chamberlin, D., Fernández, M., Kay, M., Robie, J., and Siméon, J. (2007) XML Path Language (XPath) 2.0, [Online] Avail: [30/08/2009].
  2. Boag, S., Chamberlin, D., Fernández, M., Florescu, D., Robie, J., and Siméon, J. (2007) XQuery 1.0: An XML Query Language [Online] Avail: [30/08/2009].
  3. Böhme, T. and Rahm, E., (2003) 'Multi-User Evalaution of XML Data Management Systems with XMach178, EEXTT'02, pp 148-159.
  4. Carey, M., DeWitt, D. and Naughton, J. (1993), 'The OO7 Benchmark', ACM/SIGMOD Record, 22(2), pp 12-21.
  5. Cohen, S. (2008) 'Generating XML Structure Using Examples and Constraints', PVLDB'08, pp 490-501.
  6. Florescu, D., and Kossmann, D. (1999) 'A Performance Evaluation of alternative Mapping Schemas for Storing XML Data in a Relational Database', TR:3680, May 1999, INRIA, Rocquencourt, France.
  7. Harder, T., Haustein, M., Mathis, C., and Wagner, M. (2007) 'Node Labelling Schemes for Dynamic XML Documents Reconsidered'. Int'l Journal of DKE, 60(1), pp 126-149.
  8. Jiang, H., Lu, H., Wang, W., and Yu, J. (2002) 'XParent: An Efficient RDBMS-Based XML Database System', ICDE'02, CA, USA, pp 1-2.
  9. Li, Y., Bressan, S., Dobbie, G., Lacroix, Z., Lee, M., Nambiar, U. and Wadhwa, B. (2001) 'XOO7: Applying OO7 Benchmark to XML Query Processing Tool', ACM/CIKM., Atlanta, USA, pp 167-173.
  10. Liefke, H., and Suciu, D. (2000) 'XMill: an efficient compressor for XML data', ACM/SIGMOD'00, pp 153-164.
  11. Lu, H., Yu, J., Wang, G., Zheng, S., Jiang, H., Yu, G. and Zhou, A. (2005) 'What Makes the Differences: Benchmarking XML Database Implementations', ACM/IT., 5(1), NY, USA, pp 154-194.
  12. Miklau, G. (2009) UW XMLData Repository [Online] Avail: xmldatasets/ [15/06/2009].
  13. Mlynkova, I. (2008) 'XML Benchmarking', IADIS'08, pp 59-66.
  14. Nicola, M., Kogan, I.., and Schiefer, B. (2007) 'An XML Transaction Processing Benchmark', ACM/SIGMOD'07, pp 937-948.
  15. PennProj. (1999) The Penn Treebank Project [Online] Avail: [21/06/2009].
  16. Runapongsa, K., Patel, M., Jagadish, H., Chen, Y., and AlKhalifa. S. (2006) 'The Michigan Benchmark: Towards XML Query Performance Diagnostics', Int'l Journal of IS, 31(2), pp 73-97.
  17. Schmidt, A., Waas, F., Kersten, M., Carey, D., Manolescu, I., and Busse. R. (2002) 'XMark: A Benchmark for XML Data Management', VLDB'02, Hong Kong, China, pp 1-12.
  18. Schmidt, A., Waas, F., Kersten, M., Florescu, D., Carey, M., Manolescu, J. and Busse, R. (2001), 'Why and How to Benchmark XML Databases', ACM/SIGMOD'01, CA, USA, pp 27-32.
  19. Yao, B. Özeu, M., and Khandelwal, N., (2004) 'XBench Benchmark and Performance Testing of XML DBMSs', ICDE'04, pp 621-632.
  20. Wu, Y., Lele, N., Aroskar, R., Chinnusamy, S., and Brenes, S. (2009) 'XQGen-An Algebra-based XPath Query Generator for Micro-Benchmarking', ACM/CIKM'09, pp 2109-2110.

Paper Citation

in Harvard Style

Al-Badawi M., North S. and Eaglestone B. (2010). THE 3D XML BENCHMARK . In Proceedings of the 6th International Conference on Web Information Systems and Technology - Volume 1: WEBIST, ISBN 978-989-674-025-2, pages 13-20. DOI: 10.5220/0002774300130020

in Bibtex Style

author={Mohammed Al-Badawi and Siobhán North and Barry Eaglestone},
booktitle={Proceedings of the 6th International Conference on Web Information Systems and Technology - Volume 1: WEBIST,},

in EndNote Style

JO - Proceedings of the 6th International Conference on Web Information Systems and Technology - Volume 1: WEBIST,
SN - 978-989-674-025-2
AU - Al-Badawi M.
AU - North S.
AU - Eaglestone B.
PY - 2010
SP - 13
EP - 20
DO - 10.5220/0002774300130020