A Machine Learning Approach for Layout Inference in Spreadsheets

Elvis Koci, Maik Thiele, Oscar Romero, Wolfgang Lehner

2016

Abstract

Spreadsheet applications are one of the most used tools for content generation and presentation in industry and the Web. In spite of this success, there does not exist a comprehensive approach to automatically extract and reuse the richness of data maintained in this format. The biggest obstacle is the lack of awareness about the structure of the data in spreadsheets, which otherwise could provide the means to automatically understand and extract knowledge from these files. In this paper, we propose a classification approach to discover the layout of tables in spreadsheets. Therefore, we focus on the cell level, considering a wide range of features not covered before by related work. We evaluated the performance of our classifiers on a large dataset covering three different corpora from various domains. Finally, our work includes a novel technique for detecting and repairing incorrectly classified cells in a post-processing step. The experimental results show that our approach delivers very high accuracy bringing us a crucial step closer towards automatic table extraction.

References

  1. Abraham, R. and Erwig, M. (2004). Header and unit inference for spreadsheets through spatial analyses. In VL/HCC'04, pages 165-172. IEEE.
  2. Adelfio, M. D. and Samet, H. (2013). Schema extraction for tabular data on the web. VLDB'13, 6(6):421-432.
  3. Barik, T., Lubick, K., Smith, J., Slankas, J., and MurphyHill, E. (2015). FUSE: A Reproducible, Extendable, Internet-scale Corpus of Spreadsheets. In MSR'15.
  4. Breiman, L. (2001). Random forests. Machine Learning, 45(1):5-32.
  5. Breiman, L., Friedman, J., Olshen, R., and Stone, C. (1984). Classification and Regression Trees . Wadsworth.
  6. Chen, Z. and Cafarella, M. (2013). Automatic web spreadsheet data extraction. In SSW'13, page 1. ACM.
  7. Chen, Z. and Cafarella, M. (2014). Integrating spreadsheet data via accurate and low-effort extraction. In SIGKDD'14, pages 1126-1135. ACM.
  8. Crestan, E. and Pantel, P. (2011). Web-scale table census and classification. In WSDM'11, pages 545-554. ACM.
  9. Eberius, J., Braunschweig, K., Hentsch, M., Thiele, M., Ahmadov, A., and Lehner, W. (2015). Building the dresden web table corpus: A classification approach. In BDC'15. IEEE/ACM.
  10. Eberius, J., Werner, C., Thiele, M., Braunschweig, K., Dannecker, L., and Lehner, W. (2013). Deexcelerator: A framework for extracting relational data from partially structured documents. In CIKM'13, pages 2477-2480. ACM.
  11. Fisher, M. and Rothermel, G. (2005). The euses spreadsheet corpus: a shared resource for supporting experimentation with spreadsheet dependability mechanisms. In SIGSOFT'05, volume 30, pages 1-5. ACM.
  12. Hermans, F. and Murphy-Hill, E. (2015). Enron's spreadsheets and related emails: A dataset and analysis. In Proceedings of ICSE 7815. IEEE.
  13. Liu, H. and Yu, L. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4):491-502.
  14. Platt, J. C. (1998). Fast training of support vector machines using sequential minimal optimization. In Advances in Kernel Methods - Support Vector Learning. MIT Press.
  15. Quinlan, J. R. (1993). C4.5: Programs for machine learning. Morgan Kaufmann Publishers Inc.
  16. Vapnik, V. (1982). Estimation of Dependences Based on Empirical Data. Springer Series in Statistics. Springer-Verlag New York, Inc.
  17. Wang, Y. and Hu, J. (2002). A machine learning based approach for table detection on the web. In WWW'02, pages 242-250. ACM.
Download


Paper Citation


in Harvard Style

Koci E., Thiele M., Romero O. and Lehner W. (2016). A Machine Learning Approach for Layout Inference in Spreadsheets . In Proceedings of the 8th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, (IC3K 2016) ISBN 978-989-758-203-5, pages 77-88. DOI: 10.5220/0006052200770088


in Bibtex Style

@conference{kdir16,
author={Elvis Koci and Maik Thiele and Oscar Romero and Wolfgang Lehner},
title={A Machine Learning Approach for Layout Inference in Spreadsheets},
booktitle={Proceedings of the 8th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, (IC3K 2016)},
year={2016},
pages={77-88},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006052200770088},
isbn={978-989-758-203-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR, (IC3K 2016)
TI - A Machine Learning Approach for Layout Inference in Spreadsheets
SN - 978-989-758-203-5
AU - Koci E.
AU - Thiele M.
AU - Romero O.
AU - Lehner W.
PY - 2016
SP - 77
EP - 88
DO - 10.5220/0006052200770088