Online Non-rigid Structure-from-Motion based on a Keyframe Representation of History

Simon Donné, Ljubomir Jovanov, Bart Goossens, Wilfried Philips, Aleksandra Pižurica

2014

Abstract

Non-rigid structure-from-motion in an on-line setting holds many promises for useful applications, and off-line reconstruction techniques are already very advanced. Literature has only recently started focusing on on-line reconstruction, with only a handful of existing techniques available. Here we propose a novel method of history representation which utilizes the advances in off-line reconstruction. We represent the history as a set of keyframes, a representative subset of all past frames. This history representation is used as side-information in the estimation of individual frames. We expand the history as previously unseen frames arrive and compress it again when its size grows too large. We evaluate the proposed method on some test sequences, focusing on a human face in a conversation. While on-line algorithms can never perform as well as off-line methods as they have less information available, our method compares favourably to the state of the art off-line methods.

References

  1. Agudo, A., Calvo, B., and Montiel, J. (2012). 3d reconstruction of non-rigid surfaces in real-time using wedge elements. In Computer Vision ECCV 2012. Workshops and Demonstrations, volume 7583 of Lecture Notes in Computer Science, pages 113-122. Springer Berlin Heidelberg.
  2. Akhter, I., Sheikh, Y., Khan, S., and Kanade, T. (2011). Trajectory space: A dual representation for nonrigid structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).
  3. Akhter, I., Sheikh, Y. A., Khan, S., and Kanade, T. (2008). Nonrigid structure from motion in trajectory space. In Neural Information Processing Systems.
  4. Blanz, V. and Vetter, T. (1999). A morphable model for the synthesis of 3d faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques, SIGGRAPH 7899, pages 187-194, New York and NY and USA. ACM Press/AddisonWesley Publishing Co.
  5. Bregler, C., Hertzmann, A., and Biermann, H. (2000). Recovering non-rigid 3d shape from image streams. In Computer Vision and Pattern Recognition and 2000. Proceedings. IEEE Conference on, volume 2, pages 690 -696 vol.2.
  6. Fayad, J., Bue, A. D., de Agapito, L., and Aguiar, P. (2009). Non-rigid structure from motion using quadratic deformation models. In BMVC. British Machine Vision Association.
  7. Gotardo, P. and Martinez, A. (2011a). Kernel non-rigid structure from motion. In Computer Vision (ICCV) and 2011 IEEE International Conference on, pages 802 -809.
  8. Gotardo, P. F. and Martinez, A. M. (2011b). Computing smooth time trajectories for camera and deformable shape in structure from motion with occlusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33:2051-2065.
  9. Hartley, R. and Vidal, R. (2008). Perspective nonrigid shape and motion recovery. In In Proc. European Conference on Computer Vision, pages 276-289.
  10. Lee, S. J., Park, K. R., and Kim, J. (2011). A sfm-based 3d face reconstruction method robust to self-occlusion by using a shape conversion matrix. Pattern Recogn., 44(7):1470-1486.
  11. Qu, C., Gao, H., and Ekenel, H. K. (2012). Rotation update on manifold for non-rigid structure from motion. IEEE International Conference on Image Processing.
  12. Rabaud, V. and Belongie, S. (2008). Re-thinking non-rigid structure from motion. In Computer Vision and Pattern Recognition and 2008. CVPR 2008. IEEE Conference on, pages 1 -8.
  13. Rutanen, K. (2011). Tim, information-theoretic measures in matlab. http://www.cs.tut.fi/ timhome/tim/tim.htm.
  14. Shaji, A. and Chandran, S. (2008). Riemannian manifold optimisation for non-rigid structure from motion. In Computer Vision and Pattern Recognition Workshops and 2008. CVPRW 7808. IEEE Computer Society Conference on, pages 1 -6.
  15. Tao, L., Mein, S. J., Quan, W., and Matuszewski, B. J. (2013). Recursive non-rigid structure from motion with online learned shape prior. Computer Vision and Image Understanding, 117(10):1287 - 1298.
  16. Team, O. O. M. (2006). http://www.elephantsdream.org/.
  17. Torresani, L., Hertzmann, A., and Bregler, C. (2008). Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  18. Wang, S.-F. and Lai, S.-H. (2008). Estimating 3d face model and facial deformation from a single image based on expression manifold optimization. In Proceedings of the 10th European Conference on Computer Vision: Part I, ECCV 7808, pages 589-602, Berlin and Heidelberg. Springer-Verlag.
  19. Xiao, J. and Kanade, T. (2005). Uncalibrated perspective reconstruction of deformable structures. In Proceedings of the Tenth IEEE International Conference on Computer Vision - Volume 2, ICCV 7805, pages 1075- 1082, Washington and DC and USA. IEEE Computer Society.
  20. Zhou, H., Li, X., and Sadka, A. (2012). Nonrigid structurefrom-motion from 2-d images using markov chain monte carlo. Multimedia and IEEE Transactions on, 14(1):168 -177.
Download


Paper Citation


in Harvard Style

Donné S., Jovanov L., Goossens B., Philips W. and Pižurica A. (2014). Online Non-rigid Structure-from-Motion based on a Keyframe Representation of History . In Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 2: PANORAMA, (VISIGRAPP 2014) ISBN 978-989-758-004-8, pages 723-731. DOI: 10.5220/0004871407230731


in Bibtex Style

@conference{panorama14,
author={Simon Donné and Ljubomir Jovanov and Bart Goossens and Wilfried Philips and Aleksandra Pižurica},
title={Online Non-rigid Structure-from-Motion based on a Keyframe Representation of History},
booktitle={Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 2: PANORAMA, (VISIGRAPP 2014)},
year={2014},
pages={723-731},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004871407230731},
isbn={978-989-758-004-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 2: PANORAMA, (VISIGRAPP 2014)
TI - Online Non-rigid Structure-from-Motion based on a Keyframe Representation of History
SN - 978-989-758-004-8
AU - Donné S.
AU - Jovanov L.
AU - Goossens B.
AU - Philips W.
AU - Pižurica A.
PY - 2014
SP - 723
EP - 731
DO - 10.5220/0004871407230731