3D View Reconstruction from Endoscopic Videos for Gastrointestinal Tract Surgery Planning

Xiaohong W. Gao, Annisa Rahmanti, Barbara Braden

2025

Abstract

This paper investigates the application of neural radiance field (NeRF) to reconstruct a 3D model from 2D endoscopic videos for surgical planning and removal of gastrointestinal lesions. It comprises three stages. The first one is video preprocess to remove frames with artefact of colour misalignment based on a deep learning network. Then the remaining frames are converted into NeRF compatible format. This stage includes extraction of camera information regarding intrinsic, extrinsic and ray pathway parameters as well as conversion to NeRF format based on COLMAP library, a pipeline built upon structure-from-motion (SfM) with multi-view stereo (MVS). Finally the training takes place for establishment of NeRF model implemented upon Nerfstudio library. Initial results illustrate that this end-to-end, i.e. from 2D video input to 3D model output deep learning architecture presents great potentials for reconstruction of gastrointestinal tract. Base on the two sets of data containing 2600 images, the similarity measures of SSIM, PSNR and LPIPS between original (ground truth) and rendered images are 19.46 ± 2.56, 0.70 ± 0.054, and 0.49 ± 0.05 respectively. Future work includes enlarging dataset and removal of ghostly artefact from rendered images.

Paper Citation

in Harvard Style

Gao X., Rahmanti A. and Braden B. (2025). 3D View Reconstruction from Endoscopic Videos for Gastrointestinal Tract Surgery Planning. In Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 1: BIOIMAGING; ISBN 978-989-758-731-3, SciTePress, pages 221-228. DOI: 10.5220/0013125000003911

in Bibtex Style

@conference{bioimaging25,
author={Xiaohong Gao and Annisa Rahmanti and Barbara Braden},
title={3D View Reconstruction from Endoscopic Videos for Gastrointestinal Tract Surgery Planning},
booktitle={Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 1: BIOIMAGING},
year={2025},
pages={221-228},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013125000003911},
isbn={978-989-758-731-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 1: BIOIMAGING
TI - 3D View Reconstruction from Endoscopic Videos for Gastrointestinal Tract Surgery Planning
SN - 978-989-758-731-3
AU - Gao X.
AU - Rahmanti A.
AU - Braden B.
PY - 2025
SP - 221
EP - 228
DO - 10.5220/0013125000003911
PB - SciTePress