Classification and Embedding of Semantic Scene Graphs for Active Cross-Domain Self-Localization

Yoshida Mitsuki, Yamamoto Ryogo, Wakayama Kazuki, Hiroki Tomoe, Tanaka Kanji

2023

Abstract

In visual robot self-localization, semantic scene graph (S2G) has attracted recent research attention as a valuable scene model that is robust against both viewpoint and appearance changes. However, the use of S2G in the context of active self-localization has not been sufficiently explored yet. In general, an active self-localization system consists of two essential modules. One is the visual place recognition (VPR) model, which aims to classify an input scene to a specific place class. The other is the next-best-view (NBV) planner, which aims to map the current state to the NBV action. We propose an efficient trainable framework of active self-localization in which a graph neural network (GNN) is effectively shared by these two modules. Specifically, first, the GNN is trained as a S2G classifier for VPR in a self-supervised learning manner. Second, the trained GNN is reused as a means of the dissimilarity-based embedding to map an S2G to the fixed-length state vector. To summarize, our approach uses the GNN in two ways: (1) passive single-view self-localization, (2) knowledge transfer from passive to active self-localization. Experiments using the public NCLT dataset have shown that the proposed framework outperforms other baseline self-localization methods.

Paper Citation

in Harvard Style

Mitsuki Y., Ryogo Y., Kazuki W., Tomoe H. and Kanji T. (2023). Classification and Embedding of Semantic Scene Graphs for Active Cross-Domain Self-Localization. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 562-569. DOI: 10.5220/0011621200003417

in Bibtex Style

@conference{visapp23,
author={Yoshida Mitsuki and Yamamoto Ryogo and Wakayama Kazuki and Hiroki Tomoe and Tanaka Kanji},
title={Classification and Embedding of Semantic Scene Graphs for Active Cross-Domain Self-Localization},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP},
year={2023},
pages={562-569},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011621200003417},
isbn={978-989-758-634-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 4: VISAPP
TI - Classification and Embedding of Semantic Scene Graphs for Active Cross-Domain Self-Localization
SN - 978-989-758-634-7
AU - Mitsuki Y.
AU - Ryogo Y.
AU - Kazuki W.
AU - Tomoe H.
AU - Kanji T.
PY - 2023
SP - 562
EP - 569
DO - 10.5220/0011621200003417
PB - SciTePress