Fast Document Similarity Computations using GPGPU

Parijat Shukla, Arun K. Somani

2018

Abstract

Several Big Data problems involve computing similarities between entities, such as records, documents, etc., in timely manner. Recent studies point that similarity-based deduplication techniques are efficient for document databases. Delta encoding-like techniques are commonly leveraged to compute similarities between documents. Operational requirements dictate low latency constraints. The previous researches do not consider parallel computing to deliver low latency delta encoding solutions. This paper makes two-fold contribution in context of delta encoding problem occurring in document databases: (1) develop a parallel processing-based technique to compute similarities between documents, and (2) design a GPU-based document cache framework to accelerate the performance of delta encoding pipeline. We experiment with real datasets. We achieve throughput of more than 500 similarity computations per millisecond. And the similarity compuatation framework achieves a throughput in the range of 237-312 MB per second which is up to 10X higher throughput when compared to the hashing-based approaches.

Paper Citation

in Harvard Style

Shukla P. and Somani A. (2018). Fast Document Similarity Computations using GPGPU. In Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2018) - Volume 1: KDIR; ISBN 978-989-758-330-8, SciTePress, pages 323-331. DOI: 10.5220/0006960303230331

in Bibtex Style

@conference{kdir18,
author={Parijat Shukla and Arun K. Somani},
title={Fast Document Similarity Computations using GPGPU},
booktitle={Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2018) - Volume 1: KDIR},
year={2018},
pages={323-331},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006960303230331},
isbn={978-989-758-330-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2018) - Volume 1: KDIR
TI - Fast Document Similarity Computations using GPGPU
SN - 978-989-758-330-8
AU - Shukla P.
AU - Somani A.
PY - 2018
SP - 323
EP - 331
DO - 10.5220/0006960303230331
PB - SciTePress