Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans

Alexandr Notchenko, Vladislav Ishimtsev, Alexey Artemov, Vadim Selyutin, Emil Bogomolov, Evgeny Burnaev

2022

Abstract

We propose Scan2Part, a method to segment individual parts of objects in real-world, noisy indoor RGB-D scans. To this end, we vary the part hierarchies of objects in indoor scenes and explore their effect on scene understanding models. Specifically, we use a sparse U-Net-based architecture that captures the fine-scale detail of the underlying 3D scan geometry by leveraging a multi-scale feature hierarchy. In order to train our method, we introduce the Scan2Part dataset, which is the first large-scale collection providing detailed semantic labels at the part level in the real-world setting. In total, we provide 242,081 correspondences between 53,618 PartNet parts of 2,477 ShapeNet objects and 1,506 ScanNet scenes, at two spatial resolutions of 2 cm3 and 5 cm3. As output, we are able to predict fine-grained per-object part labels, even when the geometry is coarse or partially missing. Overall, we believe that both our method as well as newly introduced dataset is a stepping stone forward towards structural understanding of real-world 3D environments.

Paper Citation

in Harvard Style

Notchenko A., Ishimtsev V., Artemov A., Selyutin V., Bogomolov E. and Burnaev E. (2022). Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 711-722. DOI: 10.5220/0010848200003124

in Bibtex Style

@conference{visapp22,
author={Alexandr Notchenko and Vladislav Ishimtsev and Alexey Artemov and Vadim Selyutin and Emil Bogomolov and Evgeny Burnaev},
title={Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={711-722},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010848200003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans
SN - 978-989-758-555-5
AU - Notchenko A.
AU - Ishimtsev V.
AU - Artemov A.
AU - Selyutin V.
AU - Bogomolov E.
AU - Burnaev E.
PY - 2022
SP - 711
EP - 722
DO - 10.5220/0010848200003124
PB - SciTePress