Joint Large Displacement Scene Flow and Occlusion Variational Estimation

Roberto P. Palomares, Gloria Haro, Coloma Ballester

2017

Abstract

This paper presents a novel variational approach for the joint estimation of scene flow and occlusions. Our method does not assume that a depth sensor is available. Instead, we use a stereo sequence and exploit the fact that points that are occluded in time, might be visible from the other view and thus the 3D geometry can be densely reinforced in an appropriate manner through a simultaneous motion occlusion characterization. Moreover, large displacements are correctly captured thanks to an optimization strategy that uses a set of sparse image correspondences to guide the minimization process. We include qualitative and quantitative experimental results on several datasets illustrating that both proposals help to improve the baseline results.

References

  1. Ayvaci, A., Raptis, M., and Soatto, S. (2012). Sparse occlusion detection with optical flow. International Journal of Computer Vision, 97(3):322-338.
  2. Ballester, C., Garrido, L., Lazcano, V., and Caselles, V. (2012). A tv-l1 optical flow method with occlusion detection. In Pinz, A., Pock, T., Bischof, H., and Leberl, F., editors, DAGM/OAGM Symposium, volume 7476 of Lecture Notes in Computer Science, pages 31-40. Springer.
  3. Basha, T., Moses, Y., and Kiryati, N. (2013). Multi-view scene flow estimation: A view centered variational approach. International Journal of Computer Vision, 101(1):6-21.
  4. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In European Conference on Computer Vision (ECCV), volume 3024 of Lecture Notes in Computer Science, pages 25-36. Springer.
  5. Butler, D. J., Wulff, J., Stanley, G. B., and Black, M. J. (2012). A naturalistic open source movie for optical flow evaluation. In European Conference on Computer Vision, pages 611-625.
  6. Cech, J., Sanchez-Riera, J., and Horaud, R. P. (2011). Scene flow estimation by growing correspondence seeds. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3129-3136.
  7. Geiger, A., Lenz, P., and Urtasun, R. (2012). Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Computer Vision and Pattern Recognition (CVPR).
  8. Huguet, F. and Devernay, F. (2007). A variational method for scene flow estimation from stereo sequences. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, pages 1-7.
  9. Ince, S. and Konrad, J. (2008). Occlusion-aware optical flow estimation. IEEE Transactions on Image Processing, 17(8):1443-1451.
  10. Jaimez, M., Souiai, M., Stueckler, J., Gonzalez-Jimenez, J., and Cremers, D. (2015). Motion cooperation: Smooth piece-wise rigid scene flow from rgbd images. In Proc. of the Int. Conference on 3D Vision (3DV). ¡a href=”https://youtu.be/qjPsKb?kvE”target=”?blank”¿[video]¡/a¿.
  11. Menze, M. and Geiger, A. (2015). Object scene flow for autonomous vehicles. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
  12. Palomares, R. P., Meinhardt-Llopis, E., Ballester, C., and Haro, G. (2016). Faldoi: A new minimization strategy for large displacement variational optical flow. Journal of Mathematical Imaging and Vision, pages 1-20.
  13. Pons, J. P., Keriven, R., and Faugeras, O. (2007). Multiview stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal on Computer Vision, 72(2):179-193.
  14. Quiroga, J., Brox, T., Devernay, F., and Crowley, J. (2014). Dense semi-rigid scene flow estimation from rgbd images. In ECCV 2014, pages 567-582.
  15. Sand, P. and Teller, S. (2008). Particle video: Long-range motion estimation using point trajectories. International Journal of Computer Vision, 80(1):72-91.
  16. Sun, D., Sudderth, E. B., and Pfister, H. (2015). Layered rgbd scene flow estimation. In Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, pages 548-556.
  17. Vedula, S. and et al. (1999). Three-dimensional scene flow. In Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on. IEEE, volume 2, pages 722-729.
  18. Vedula, S., Rander, P., Collins, R., and Kanade, T. (2005). Three-dimensional scene flow. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 27(3):475-480.
  19. Vogel, C., Schindler, K., and Roth, S. (2011). 3d scene flow estimation with a rigid motion prior. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1291-1298.
  20. Vogel, C., Schindler, K., and Roth, S. (2015). 3d scene flow estimation with a piecewise rigid scene model. International Journal of Computer Vision, 115(1):1- 28.
  21. Wang, Y., Zhang, J., Liu, Z., Wu, Q., Chou, P. A., Zhang, Z., and Jia, Y. (2015). Handling occlusion and large displacement through improved rgb-d scene flow estimation. IEEE Transactions on Circuits and Systems for Video Technology, 26(7):1265-1278.
  22. Wedel, A., Brox, T., Vaudrey, T., Rabe, C., Franke, U., and Cremers, D. (2011). Stereoscopic scene flow computation for 3d motion understanding. International Journal of Computer Vision, 95(1):29-51.
  23. Weinzaepfel, P., Revaud, J., Harchaoui, Z., and Schmid, C. (2013). DeepFlow : Large displacement optical flow with deep matching. International Conference on Computer Vision.
  24. Young, D. M. (1971). Iterative solution of large linear systems. Computer science and applied mathematics. Academic Press, Orlando.
  25. Zanfir, A. and Sminchisescu, C. (2015). Large displacement 3d scene flow with occlusion reasoning. In Proceedings of the IEEE International Conference on Computer Vision, pages 4417-4425.
Download


Paper Citation


in Harvard Style

P. Palomares R., Haro G. and Ballester C. (2017). Joint Large Displacement Scene Flow and Occlusion Variational Estimation . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-227-1, pages 172-180. DOI: 10.5220/0006110601720180


in Bibtex Style

@conference{visapp17,
author={Roberto P. Palomares and Gloria Haro and Coloma Ballester},
title={Joint Large Displacement Scene Flow and Occlusion Variational Estimation},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={172-180},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006110601720180},
isbn={978-989-758-227-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)
TI - Joint Large Displacement Scene Flow and Occlusion Variational Estimation
SN - 978-989-758-227-1
AU - P. Palomares R.
AU - Haro G.
AU - Ballester C.
PY - 2017
SP - 172
EP - 180
DO - 10.5220/0006110601720180