Segmentation-Guided Neural Radiance Fields for Novel Street View Synthesis

Yizhou Li, Yusuke Monno, Masatoshi Okutomi, Yuuichi Tanaka, Seiichi Kataoka, Teruaki Kosiba

2025

Abstract

Recent advances in Neural Radiance Fields (NeRF) have shown great potential in 3D reconstruction and novel view synthesis, particularly for indoor and small-scale scenes. However, extending NeRF to large-scale outdoor environments presents challenges such as transient objects, sparse cameras and textures, and varying lighting conditions. In this paper, we propose a segmentation-guided enhancement to NeRF for outdoor street scenes, focusing on complex urban environments. Our approach extends ZipNeRF and utilizes Grounded SAM for segmentation mask generation, enabling effective handling of transient objects, modeling of the sky, and regularization of the ground. We also introduce appearance embeddings to adapt to inconsistent lighting across view sequences. Experimental results demonstrate that our method outperforms the baseline ZipNeRF, improving novel view synthesis quality with fewer artifacts and sharper details.

Paper Citation

in Harvard Style

Li Y., Monno Y., Okutomi M., Tanaka Y., Kataoka S. and Kosiba T. (2025). Segmentation-Guided Neural Radiance Fields for Novel Street View Synthesis. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 591-597. DOI: 10.5220/0013244200003912

in Bibtex Style

@conference{visapp25,
author={Yizhou Li and Yusuke Monno and Masatoshi Okutomi and Yuuichi Tanaka and Seiichi Kataoka and Teruaki Kosiba},
title={Segmentation-Guided Neural Radiance Fields for Novel Street View Synthesis},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2025},
pages={591-597},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013244200003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Segmentation-Guided Neural Radiance Fields for Novel Street View Synthesis
SN - 978-989-758-728-3
AU - Li Y.
AU - Monno Y.
AU - Okutomi M.
AU - Tanaka Y.
AU - Kataoka S.
AU - Kosiba T.
PY - 2025
SP - 591
EP - 597
DO - 10.5220/0013244200003912
PB - SciTePress