Supervised Machine Learning and Feature Selection for a Document Analysis Application

James Pope, Daniel Powers, J. A. (Jim) Connell, Milad Jasemi, David Taylor, Xenofon Fafoutis

2020

Abstract

Over the past three decades large amounts of information have been converted to image formats from paper documents. Though in digital form, extracting the information, usually textual, from these documents requires complex image processing and optical character recognition techniques. The processing pipeline from the image to information typically includes an orientation correction task, document identification task, and text analysis task. When there are many document variants the tasks become difficult requiring complex subanalysis for each variant and quickly exceeds human capability. In this work, we demonstrate a document analysis application with the orientation correction and document identification task carried out by supervised machine learning techniques for a large, international airline. The documents have been amassed over forty years with numerous variants and are mostly black and white, typically consist of text and lines, and some have extensive noise. Low level symbols are extracted from the raw images and separated into partitions. The partitions are used to generate statistical features which are then used to train the classifiers. We compare the classifiers for each task (e.g. decision tree, support vector machine, and random forest) to choose the most appropriate. We also perform feature selection to reduce the complexity of the document type classifiers. These parsimonious models result in comparable accuracy with 80% or fewer features.

Paper Citation

in Harvard Style

Pope J., Powers D., Connell J., Jasemi M., Taylor D. and Fafoutis X. (2020). Supervised Machine Learning and Feature Selection for a Document Analysis Application. In Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-397-1, pages 415-424. DOI: 10.5220/0008925104150424

in Bibtex Style

@conference{icpram20,
author={James Pope and Daniel Powers and J. A. (Jim) Connell and Milad Jasemi and David Taylor and Xenofon Fafoutis},
title={Supervised Machine Learning and Feature Selection for a Document Analysis Application},
booktitle={Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2020},
pages={415-424},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008925104150424},
isbn={978-989-758-397-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Supervised Machine Learning and Feature Selection for a Document Analysis Application
SN - 978-989-758-397-1
AU - Pope J.
AU - Powers D.
AU - Connell J.
AU - Jasemi M.
AU - Taylor D.
AU - Fafoutis X.
PY - 2020
SP - 415
EP - 424
DO - 10.5220/0008925104150424