FRAME INTERPOLATION WITH OCCLUSION DETECTION USING A TIME COHERENT SEGMENTATION

Rida Sadek, Coloma Ballester, Luis Garrido, Enric Meinhardt, Vicent Caselles

2012

Abstract

In this paper we propose an interpolation method to produce a sequence of plausible intermediate frames between two input images. The main feature of the proposed method is the handling of occlusions using a time coherent video segmentation into spatio-temporal regions. Occlusions and disocclusions are defined as points in a frame where a region ends or starts, respectively. Out of these points, forward and backward motion fields are used to interpolate the intermediate frames. After motion-based interpolation, there may still be some holes which are filled using a hole filling algorithm. We illustrate the proposed method with some experiments.

References

  1. Alvarez, L., Deriche, R., Papadopoulo, T., and Sánchez, J. (2007). Symmetrical dense optical flow estimation with occlusions detection. Int. Journal of Comp. Vis., 75:371-385.
  2. Arias, P., Facciolo, G., Caselles, V., and Sapiro, G. (2011). A variational framework for exemplar-based image inpainting. Int. Journal of Comp. Vis., 93:319-347.
  3. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., and Szeliski, R. (2011). A database and evaluation methodology for optical flow. Int. Journal of Comp. Vis., 92:1-31.
  4. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. (2009). PatchMatch: a randomized correspondence algorithm for structural image editing. In Proc. of SIGGRAPH, pages 1-11.
  5. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In ECCV, pages 25-36.
  6. Brox, T. and Malik, J. (2010). Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on PAMI, pages 500-513.
  7. Cafforio, C., Rocca, F., and Tubaro, S. (1990). Motion compensated image interpolation. IEEE Transactions on Communications, 38:215-222.
  8. Criminisi, A., Pérez, P., and Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. IEEE Trans. on IP, 13:1200-1212.
  9. Dane, G. and Nguyen, T. (2004). Motion vector processing for frame rate up conversion. In IEEE ICASSP, volume 3, pages 309-312.
  10. Herbst, E., Seitz, S., and Baker, S. (2009). Occlusion reasoning for temporal interpolation using optical flow. In Technical report UW-CSE-09-08-01. Dept. of Comp. Sci. and Eng., University of Washington.
  11. Ince, S. and Konrad, J. (2008). Occlusion-aware optical flow estimation. IEEE Trans. on IP, 17:1443-1451.
  12. Koepfler, G., Lopez, C., and Morel, J. (1994). A multiscale algorithm for image segmentation by variational method. SIAM Journal on Numerical Analysis, 31:282-299.
  13. Linz, C., Lipski, C., and Magnor, M. (2010). Multi-image interpolation based on graph-cuts and symmetric optical flow. In 15th International Workshop on Vision, Modeling and Visualization (VMV), pages 115-122.
  14. Mahajan, D., Huang, F., Matusik, W., Ramamoorthi, R., and Belhumeur, P. (2009). Moving gradients: a pathbased method for plausible image interpolation. In ACM SIGGRAPH, volume 28, pages 1-11.
  15. Sun, D., Roth, S., and Black, M. (2010). Secrets of optical flow estimation and their principles. In IEEE Conference on CVPR, pages 2432-2439.
Download


Paper Citation


in Harvard Style

Sadek R., Ballester C., Garrido L., Meinhardt E. and Caselles V. (2012). FRAME INTERPOLATION WITH OCCLUSION DETECTION USING A TIME COHERENT SEGMENTATION . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012) ISBN 978-989-8565-04-4, pages 367-372. DOI: 10.5220/0003830803670372


in Bibtex Style

@conference{visapp12,
author={Rida Sadek and Coloma Ballester and Luis Garrido and Enric Meinhardt and Vicent Caselles},
title={FRAME INTERPOLATION WITH OCCLUSION DETECTION USING A TIME COHERENT SEGMENTATION},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)},
year={2012},
pages={367-372},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003830803670372},
isbn={978-989-8565-04-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)
TI - FRAME INTERPOLATION WITH OCCLUSION DETECTION USING A TIME COHERENT SEGMENTATION
SN - 978-989-8565-04-4
AU - Sadek R.
AU - Ballester C.
AU - Garrido L.
AU - Meinhardt E.
AU - Caselles V.
PY - 2012
SP - 367
EP - 372
DO - 10.5220/0003830803670372