An Occlusion Aware Five-view Stereo System and Its Application in Video Post-production

Changan Zhu, Taryn Laurendeau, Chris Joslin

2022

Abstract

Extracting the live-action elements from videos has been a time-consuming process in the post-production pipeline. The disparity map, however, shows the order of elements in a scene by indicating the distance between the elements and the camera, which could potentially become an effective tool for separating videos into ordered layers and preserving the 3D structure of the elements. In this research, we explored the possibility of simplifying the live-action video element extraction technique with disparity sequences. We developed a five-view disparity estimation and enhancement system with a two-axis setup that helps reduce the occlusions in stereo vision. The system is independent from temporal reconstruction hence is compatible with both dynamic and stationary camera paths. Our results show that the disparities from our system have visually and quantitatively better performance than the traditional binocular stereo method, and its element extraction result is comparable with the existing mature matting techniques in most cases. Ideally, the system design could be applied in cinematography by replacing the center camera with a cinematographic camera, and the output can be used for video object extraction, visual effects composition, video’s 2D to 3D conversion, or producing the training data for neural-network-based depth estimation research.

Paper Citation

in Harvard Style

Zhu C., Laurendeau T. and Joslin C. (2022). An Occlusion Aware Five-view Stereo System and Its Application in Video Post-production. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 844-854. DOI: 10.5220/0010884400003124

in Bibtex Style

@conference{visapp22,
author={Changan Zhu and Taryn Laurendeau and Chris Joslin},
title={An Occlusion Aware Five-view Stereo System and Its Application in Video Post-production},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={844-854},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010884400003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - An Occlusion Aware Five-view Stereo System and Its Application in Video Post-production
SN - 978-989-758-555-5
AU - Zhu C.
AU - Laurendeau T.
AU - Joslin C.
PY - 2022
SP - 844
EP - 854
DO - 10.5220/0010884400003124
PB - SciTePress