SMVLift: Lifting Semantic Segmentation to 3D on XR Devices

Marcus Valtonen Örnhag, Puren Güler, Anastasia Grebenyuk, Hiba Alqaysi, Tobias Widmark

2025

Abstract

Creating an immersive mixed-reality experience, where virtual objects are seamlessly blending into physical environments, requires a careful integration of 3D environmental understanding with the underlying contextual semantics. State-of-the-art methods in this field often rely on large and dense 3D point clouds, which are not feasible for real-time performance in standalone XR headsets. We introduce Sparse Multi-View Lifting (SMVLift), a lightweight 3D instance segmentation method capable of running on constrained hardware, which demonstrates on par or superior performance compared to a state-of-the-art method while being significantly less computationally demanding. Lastly, we use the framework in downstream XR applications with satisfactory performance on real hardware.

Paper Citation

in Harvard Style

Örnhag M., Güler P., Grebenyuk A., Alqaysi H. and Widmark T. (2025). SMVLift: Lifting Semantic Segmentation to 3D on XR Devices. In Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-730-6, SciTePress, pages 555-562. DOI: 10.5220/0012982800003905

in Bibtex Style

@conference{icpram25,
author={Marcus Örnhag and Puren Güler and Anastasia Grebenyuk and Hiba Alqaysi and Tobias Widmark},
title={SMVLift: Lifting Semantic Segmentation to 3D on XR Devices},
booktitle={Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2025},
pages={555-562},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012982800003905},
isbn={978-989-758-730-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - SMVLift: Lifting Semantic Segmentation to 3D on XR Devices
SN - 978-989-758-730-6
AU - Örnhag M.
AU - Güler P.
AU - Grebenyuk A.
AU - Alqaysi H.
AU - Widmark T.
PY - 2025
SP - 555
EP - 562
DO - 10.5220/0012982800003905
PB - SciTePress