Fast Filtering for Similarity Search Using Conjunctive Enumeration of Sketches in Order of Hamming Distance

Naoya Higuchi, Yasunobu Imamura, Vladimir Mic, Takeshi Shinohara, Kouichi Hirata, Tetsuji Kuboyama

2024

Abstract

Sketches are compact bit-string representations of points, often employed for speeding up searches through the effects of dimensionality reduction and data compression. In this paper, we propose a novel sketch enumeration method and demonstrate its ability to realize fast filtering for approximate nearest neighbor search in metric spaces. Whereas the Hamming distance between the query’s sketch and sketches of points to be searched has been used for sketch prioritization traditionally, recent research has introduced asymmetric distances, enabling higher recall rates with fewer candidates. Additionally, sketch enumeration methods that speed up the filtering such that high-priority solution candidates are selected based on the priority of the sketch to the given query without the need for direct sketch comparisons have been proposed. Our primary goal in this paper is to further accelerate sketch enumeration through parallel processing. While Hamming distance-based enumeration can be parallelized relatively easily, achieving high recall rates requires a large number of candidates, and speeding up the filtering alone is insufficient for overall similarity search acceleration. Therefore, we introduce the conjunctive enumeration method, which concatenates two Hamming distance-based enumerations to approximate asymmetric distance-based enumeration. Then, we validate the effectiveness of the proposed method through experiments using large-scale public datasets. Our approach offers a significant acceleration effect, thereby enhancing the efficiency of similarity search operations.

Paper Citation

in Harvard Style

Higuchi N., Imamura Y., Mic V., Shinohara T., Hirata K. and Kuboyama T. (2024). Fast Filtering for Similarity Search Using Conjunctive Enumeration of Sketches in Order of Hamming Distance. In Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-684-2, SciTePress, pages 499-510. DOI: 10.5220/0012322700003654

in Bibtex Style

@conference{icpram24,
author={Naoya Higuchi and Yasunobu Imamura and Vladimir Mic and Takeshi Shinohara and Kouichi Hirata and Tetsuji Kuboyama},
title={Fast Filtering for Similarity Search Using Conjunctive Enumeration of Sketches in Order of Hamming Distance},
booktitle={Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2024},
pages={499-510},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012322700003654},
isbn={978-989-758-684-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - Fast Filtering for Similarity Search Using Conjunctive Enumeration of Sketches in Order of Hamming Distance
SN - 978-989-758-684-2
AU - Higuchi N.
AU - Imamura Y.
AU - Mic V.
AU - Shinohara T.
AU - Hirata K.
AU - Kuboyama T.
PY - 2024
SP - 499
EP - 510
DO - 10.5220/0012322700003654
PB - SciTePress