Plane Fitting and Depth Variance Based Upsampling for Noisy Depth Map from 3D-ToF Cameras in Real-time

Kazuki Matsumoto, Francois de Sorbier, Hideo Saito

2015

Abstract

Recent advances of ToF depth sensor devices enables us to easily retrieve scene depth data with high frame rates. However, the resolution of the depth map captured from these devices is much lower than that of color images and the depth data suffers from the optical noise effects. In this paper, we propose an efficient algorithm that upsamples depth map captured by ToF depth cameras and reduces noise. The upsampling is carried out by applying plane based interpolation to the groups of points similar to planar structures and depth variance based joint bilateral upsampling to curved or bumpy surface points. For dividing the depth map into piecewise planar areas, we apply superpixel segmentation and graph component labeling. In order to distinguish planar areas and curved areas, we evaluate the reliability of detected plane structures. Compared with other state-of-the- art algorithms, our method is observed to produce an upsampled depth map that is smoothed and closer to the ground truth depth map both visually and numerically. Since the algorithm is parallelizable, it can work in real-time by utilizing highly parallel processing capabilities of modern commodity GPUs.

References

  1. Anderson, D., Herman, H., and Kelly, A. (2005). Experimental characterization of commercial flash ladar devices. In International Conference of Sensing and Technology, volume 2.
  2. Camplani, M. and Salgado, L. (2012). Adaptive spatiotemporal filter for low-cost camera depth maps. In Emerging Signal Processing Applications (ESPA), 2012 IEEE International Conference on, pages 33-36. IEEE.
  3. Chan, D., Buisman, H., Theobalt, C., Thrun, S., et al. (2008). A noise-aware filter for real-time depth upsampling. In Workshop on Multi-camera and Multimodal Sensor Fusion Algorithms and ApplicationsM2SFA2 2008.
  4. Chen, L., Lin, H., and Li, S. (2012). Depth image enhancement for kinect using region growing and bilateral filter. In Pattern Recognition (ICPR), 2012 21st International Conference on, pages 3070-3073. IEEE.
  5. Diebel, J. and Thrun, S. (2005). An application of markov random fields to range sensing. In Advances in neural information processing systems, pages 291-298.
  6. Dolson, J., Baek, J., Plagemann, C., and Thrun, S. (2010). Upsampling range data in dynamic environments. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 1141-1148. IEEE.
  7. Hawick, K. A., Leist, A., and Playne, D. P. (2010). Parallel graph component labelling with gpus and cuda. Parallel Computing, 36(12):655-678.
  8. Holzer, S., Rusu, R. B., Dixon, M., Gedikli, S., and Navab, N. (2012). Adaptive neighborhood selection for realtime surface normal estimation from organized point cloud data using integral images. In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on, pages 2684-2689. IEEE.
  9. Kopf, J., Cohen, M. F., Lischinski, D., and Uyttendaele, M. (2007). Joint bilateral upsampling. In ACM Transactions on Graphics (TOG), volume 26, page 96. ACM.
  10. Matsuo, K. and Aoki, Y. (2013). via smooth surface segmentation using tangent planes based on the superpixels of a color image. In Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on, pages 29-36. IEEE.
  11. Park, J., Kim, H., Tai, Y.-W., Brown, M. S., and Kweon, I. (2011). High quality depth map upsampling for 3d-tof cameras. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1623-1630. IEEE.
  12. Ren, C. Y. and Reid, I. (2011). gslic: a real-time implementation of slic superpixel segmentation. University of Oxford, Department of Engineering, Technical Report.
  13. Soh, Y., Sim, J.-Y., Kim, C.-S., and Lee, S.-U. (2012). Superpixel-based depth image super-resolution. In IS&T/SPIE Electronic Imaging, pages 82900D82900D. International Society for Optics and Photonics.
  14. Weikersdorfer, D., Gossow, D., and Beetz, M. (2012). Depth-adaptive superpixels. In Pattern Recognition (ICPR), 2012 21st International Conference on, pages 2087-2090. IEEE.
Download


Paper Citation


in Harvard Style

Matsumoto K., de Sorbier F. and Saito H. (2015). Plane Fitting and Depth Variance Based Upsampling for Noisy Depth Map from 3D-ToF Cameras in Real-time . In Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 2: ICPRAM, ISBN 978-989-758-077-2, pages 150-157. DOI: 10.5220/0005184801500157


in Bibtex Style

@conference{icpram15,
author={Kazuki Matsumoto and Francois de Sorbier and Hideo Saito},
title={Plane Fitting and Depth Variance Based Upsampling for Noisy Depth Map from 3D-ToF Cameras in Real-time},
booktitle={Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 2: ICPRAM,},
year={2015},
pages={150-157},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005184801500157},
isbn={978-989-758-077-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Pattern Recognition Applications and Methods - Volume 2: ICPRAM,
TI - Plane Fitting and Depth Variance Based Upsampling for Noisy Depth Map from 3D-ToF Cameras in Real-time
SN - 978-989-758-077-2
AU - Matsumoto K.
AU - de Sorbier F.
AU - Saito H.
PY - 2015
SP - 150
EP - 157
DO - 10.5220/0005184801500157