Joint Color and Depth Segmentation based on Region Merging and Surface Fitting

Giampaolo Pagnutti, Pietro Zanuttigh

2016

Abstract

The recent introduction of consumer depth cameras has opened the way to novel segmentation approaches exploiting depth data together with the color information. This paper proposes a region merging segmentation scheme that jointly exploits the two clues. Firstly a set of multi-dimensional vectors is built considering the 3D spatial position, the surface orientation and the color data associated to each scene sample. Normalized cuts spectral clustering is applied to the obtained vectors in order to over-segment the scene into a large number of small segments. Then an iterative merging procedure is used to recombine the segments into the regions corresponding to the various objects and surfaces. The proposed algorithm tries to combine close compatible segments and uses a NURBS surface fitting scheme on the considered segments in order to understand if the regions candidate for the merging correspond to a single surface. The comparison with state-of-the-art methods shows how the proposed method provides an accurate and reliable scene segmentation.

References

  1. Arbelaez, P., Maire, M., Fowlkes, C., and Malik, J. (2011). Contour detection and hierarchical image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 33(5):898-916.
  2. Bleiweiss, A. and Werman, M. (2009). Fusing time-offlight depth and color for real-time segmentation and tracking. In Proc. of DAGM Workshop, pages 58-69.
  3. Dal Mutto, C., Zanuttigh, P., and Cortelazzo, G. (2011). Scene segmentation assisted by stereo vision. In Proceedings of 3DIMPVT 2011, Hangzhou, China.
  4. Dal Mutto, C., Zanuttigh, P., and Cortelazzo, G. (2012a). Fusion of geometry and color information for scene segmentation. IEEE Journal of Selected Topics in Signal Processing, 6(5):505-521.
  5. Dal Mutto, C., Zanuttigh, P., and Cortelazzo, G. M. (2012b). Time-of-Flight Cameras and Microsoft Kinect. SpringerBriefs. Springer.
  6. Erdogan, C., Paluri, M., and Dellaert, F. (2012). Planar segmentation of rgbd images using fast linear fitting and markov chain monte carlo. In Proc. of CRV.
  7. Felzenszwalb, P. and Huttenlocher, D. (2004). Efficient Graph-Based Image Segmentation. International Journal of Computer Vision, 59(2):167-181.
  8. Fowlkes, C., Belongie, S., Chung, F., and Malik, J. (2004). Spectral grouping using the nyström method. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):214-225.
  9. Gupta, S., Arbeláez, P., Girshick, R., and Malik, J. (2014). Indoor scene understanding with rgb-d images: Bottom-up segmentation, object detection and semantic segmentation. International Journal of Computer Vision, pages 1-17.
  10. Gupta, S., Arbelaez, P., and Malik, J. (2013). Perceptual organization and recognition of indoor scenes from RGB-D images. In Proceedings of CVPR.
  11. Hasnat, M. A., Alata, O., and Trmeau, A. (2014). Unsupervised rgb-d image segmentation using joint clustering and region merging. In Proceedings of BMVC.
  12. Pagnutti, G. and Zanuttigh, P. (2014). Scene segmentation from depth and color data driven by surface fitting. In IEEE International Conference on Image Processing (ICIP), pages 4407-4411. IEEE.
  13. Pagnutti, G. and Zanuttigh, P. (2015). Scene segmentation based on nurbs surface fitting metrics. In In proc. of STAG Workshop.
  14. Piegl, L. and Tiller, W. (1997). The NURBS Book (2Nd Ed.). Springer-Verlag, Inc., New York, USA.
  15. Ren, X., Bo, L., and Fox, D. (2012). Rgb-(d) scene labeling: Features and algorithms. In Proc. of CVPR.
  16. Shi, J. and Malik, J. (2000). Normalized Cuts and Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888-905.
  17. Silberman, N., Hoiem, D., Kohli, P., and Fergus, R. (2012). Indoor segmentation and support inference from rgbd images. In Proceedings of ECCV.
  18. Srinivasan, N. and Dellaert, F. (2014). A rao-blackwellized mcmc algorithm for recovering piecewise planar 3d model from multiple view rgbd images. In IEEE International Conference on Image Processing (ICIP).
  19. Taylor, C. J. and Cowley, A. (2013). Parsing indoor scenes using rgb-d imagery. In Robotics: Science and Systems, volume 8, pages 401-408.
  20. Wallenberg, M., Felsberg, M., Forssén, P.-E., and Dellen, B. (2011). Channel coding for joint colour and depth segmentation. In Proc. of DAGM, volume 6835, pages 306-315.
Download


Paper Citation


in Harvard Style

Pagnutti G. and Zanuttigh P. (2016). Joint Color and Depth Segmentation based on Region Merging and Surface Fitting . In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2016) ISBN 978-989-758-175-5, pages 93-100. DOI: 10.5220/0005672700930100


in Bibtex Style

@conference{visapp16,
author={Giampaolo Pagnutti and Pietro Zanuttigh},
title={Joint Color and Depth Segmentation based on Region Merging and Surface Fitting},
booktitle={Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2016)},
year={2016},
pages={93-100},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005672700930100},
isbn={978-989-758-175-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2016)
TI - Joint Color and Depth Segmentation based on Region Merging and Surface Fitting
SN - 978-989-758-175-5
AU - Pagnutti G.
AU - Zanuttigh P.
PY - 2016
SP - 93
EP - 100
DO - 10.5220/0005672700930100