Using Video Motion Vectors for Structure from Motion 3D Reconstruction

Richard Turner, Natasha Banerjee, Sean Banerjee

2022

Abstract

H.264 video compression has become the prevalent choice for devices which require live video streaming and include mobile phones, laptops and Micro Aerial Vehicles (MAV). H.264 utilizes motion estimation to predict the distance of pixels, grouped together as macroblocks, between two or more video frames. Live video compression using H.264 is ideal as each frame contains much of the information found in previous and future frames. By estimating the motion vector of each macroblock for every frame, significant compression can be obtained. Combined with Socket on Chip (SoC) encoders, high quality video with low power and bandwidth is now achievable. 3D scene reconstruction utilizing structure from motion (SfM) is a highly computational intensive process, typically performed offline with high computing devices. A significant portion of the computation required for SfM is in the feature detection, matching and correspondence tracking necessary for the 3D scene reconstruction. We present a SfM pipeline which uses H.264 motion vectors to replace much of the processing required to detect, match and track correspondences across video frames. Our pipeline results have shown a significant decrease in computation, while accurately reconstructing a 3D scene.

Paper Citation

in Harvard Style

Turner R., Banerjee N. and Banerjee S. (2022). Using Video Motion Vectors for Structure from Motion 3D Reconstruction. In Proceedings of the 19th International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP, ISBN 978-989-758-591-3, pages 13-22. DOI: 10.5220/0011263600003289

in Bibtex Style

@conference{sigmap22,
author={Richard Turner and Natasha Banerjee and Sean Banerjee},
title={Using Video Motion Vectors for Structure from Motion 3D Reconstruction},
booktitle={Proceedings of the 19th International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP,},
year={2022},
pages={13-22},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011263600003289},
isbn={978-989-758-591-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Conference on Signal Processing and Multimedia Applications - Volume 1: SIGMAP,
TI - Using Video Motion Vectors for Structure from Motion 3D Reconstruction
SN - 978-989-758-591-3
AU - Turner R.
AU - Banerjee N.
AU - Banerjee S.
PY - 2022
SP - 13
EP - 22
DO - 10.5220/0011263600003289