SPNeRF: Open Vocabulary 3D Neural Scene Segmentation with Superpoints

Weiwen Hu, Niccolò Parodi, Niccolò Parodi, Marcus Zepp, Ingo Feldmann, Oliver Schreer, Peter Eisert, Peter Eisert

2025

Abstract

Open-vocabulary segmentation, powered by large visual-language models like CLIP, has expanded 2D segmentation capabilities beyond fixed classes predefined by the dataset, enabling zero-shot understanding across diverse scenes. Extending these capabilities to 3D segmentation introduces challenges, as CLIP’s image-based embeddings often lack the geometric detail necessary for 3D scene segmentation. Recent methods tend to address this by introducing additional segmentation models or replacing CLIP with variations trained on segmentation data, which lead to redundancy or loss on CLIP’s general language capabilities. To overcome this limitation, we introduce SPNeRF, a NeRF based zero-shot 3D segmentation approach that leverages geometric priors. We integrate geometric primitives derived from the 3D scene into NeRF training to produce primitive-wise CLIP features, avoiding the ambiguity of point-wise features. Additionally, we propose a primitive-based merging mechanism enhanced with affinity scores. Without relying on additional segmentation models, our method further explores CLIP’s capability for 3D segmentation and achieves notable improvements over orig-inal LERF.

Paper Citation

in Harvard Style

Hu W., Parodi N., Zepp M., Feldmann I., Schreer O. and Eisert P. (2025). SPNeRF: Open Vocabulary 3D Neural Scene Segmentation with Superpoints. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 669-676. DOI: 10.5220/0013255100003912

in Bibtex Style

@conference{visapp25,
author={Weiwen Hu and Niccolò Parodi and Marcus Zepp and Ingo Feldmann and Oliver Schreer and Peter Eisert},
title={SPNeRF: Open Vocabulary 3D Neural Scene Segmentation with Superpoints},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP},
year={2025},
pages={669-676},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013255100003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP
TI - SPNeRF: Open Vocabulary 3D Neural Scene Segmentation with Superpoints
SN - 978-989-758-728-3
AU - Hu W.
AU - Parodi N.
AU - Zepp M.
AU - Feldmann I.
AU - Schreer O.
AU - Eisert P.
PY - 2025
SP - 669
EP - 676
DO - 10.5220/0013255100003912
PB - SciTePress