Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement

Takuya Matsuo, Norishige Fukushima, Yutaka Ishibashi

Abstract

In this paper, we propose a new refinement filter for depth maps. The filter convolutes a depth map by a jointly computed kernel on a natural image with a weight map. We call the filter weighted joint bilateral filter. The filter fits an outline of an object in the depth map to the outline of the object in the natural image, and it reduces noises. An additional filter of slope depth compensation filter removes blur across object boundary. The filter set’s computational cost is low and is independent of depth ranges. Thus we can refine depth maps to generate accurate depth map with lower cost. In addition, we can apply the filters for various types of depth map, such as computed by simple block matching, Markov random field based optimization, and Depth sensors. Experimental results show that the proposed filter has the best performance of improvement of depth map accuracy, and the proposed filter can perform real-time refinement.

References

  1. Boykov, Y., Veksler, O., and Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Trans. PAMI, 23(11):1222-1239.
  2. Eisemann, E. and Durand, F. (2004). Flash photography enhancement via intrinsic relighting. ACM Trans. on Graphics, 23(3):673-678.
  3. Fukushima, N., Fujii, T., Ishibashi, Y., Yendo, T., and Tanimoto, M. (2010). Real-time free viewpoint image rendering by using fast multi-pass dynamic programming. In 3DTV-Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), pages 1-4.
  4. Fukushima, N. and Ishibashi, Y. (2011). Client driven system of depth image based rendering. ECTI Trans. CIT, 5(2):15-23.
  5. Geiger, A., Roser, M., and Urtasun, R. (2010). Efficient large-scale stereo matching. In Asian Conference of Computer Vision, volume 6492, pages 25-38.
  6. Hirschmuller, H. (2008). Stereo processing by semiglobal matching and mutual information. IEEE Trans. PAMI, 30(2):328 -341.
  7. Kopf, J., Lischinski, M. F. C. D., and Uyttendaele, M. (2007). Joint bilateral upsampling. ACM Trans. on Graphics, 26(3):96.
  8. Lai, K., Bo, L., Ren, X., and Fox, D. (2011). A large-scale hierarchical multi-view rgb-d object dataset. In IEEE International Conference on Robotics and Automation (ICRA), pages 1817-1824.
  9. Lai, P. L., Tian, D., and Lopez, P. (2010). Depth map processing with iterative joint multilateral filtering. In Picture Coding Symposium, pages 9-12.
  10. Mori, Y., Fukushima, N., Yendo, T., Fujii, T., and Tanimoto, M. (2009). View generation 3d warping using depth information for ftv. Signal Processing: Image Communication, 24(1-2):65-72.
  11. Ohta, Y. and Kanade, T. (1985). Stereo by intra - and interscanline search using dynamic programming. IEEE Trans. PAMI, 7(2):139-154.
  12. Pestschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. (2004). Digital photography with flash and no-flash image pairs. ACM Trans. on Graphics, 23(3):664-672.
  13. Scharstein, D. and Szeliski, R. (2002). A taxonomy and evaluation of depth two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1):7-42.
  14. Sun, J., Zheng, N. N., and Shum, H. Y. (2003). Stereo matching using belief propagation. IEEE Trans. PAMI, 25(7):787-800.
  15. Tomasi, C. and Manduchi, R. (1998). Bilateral filtering for gray and color image. In International Conference of Computer Vision, pages 839-846.
  16. Wildeboer, M. O., Fukushima, N., Yendo, T., Tehrani, M. P., Fujii, T., and Tanimoto, M. (2010). A semi-automatic multi-view depth estimation method. In Proceedings of SPIE Visual Communications and Image Processing 2010, volume 7744.
  17. Yang, Q., Wang, L., and Ahuja, N. (2010). A constantspace belief propagation algorithm for stereo matching. In Computer Vision and Pattern Recognition, pages 1458-1465.
  18. Yang, Q., Wang, L., Yang, R., Stewenius, H., and Nister, D. (2008). Stereo matching with color-weighted correlation, hierarchial belief propagation and occlusion handling. IEEE Trans. PAMI.
  19. Yang, Q., Yang, R., Davis, J., and Nister, D. (2007). Spatialdepth super resolution for range images. In Computer Vision and Pattern Recognition, pages 1-8.
Download


Paper Citation


in Harvard Style

Matsuo T., Fukushima N. and Ishibashi Y. (2013). Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2013) ISBN 978-989-8565-48-8, pages 300-309. DOI: 10.5220/0004292203000309


in Bibtex Style

@conference{visapp13,
author={Takuya Matsuo and Norishige Fukushima and Yutaka Ishibashi},
title={Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2013)},
year={2013},
pages={300-309},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004292203000309},
isbn={978-989-8565-48-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2013)
TI - Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement
SN - 978-989-8565-48-8
AU - Matsuo T.
AU - Fukushima N.
AU - Ishibashi Y.
PY - 2013
SP - 300
EP - 309
DO - 10.5220/0004292203000309