Bert De Decker, Tom Mertens, Philippe Bekaert


We present a novel way of interacting with a virtual 3D scene in the context of free-viewpoint video. Using a multi-camera setup, our technique detects collisions between virtual objects and real objects, including people. We perform collision computations directly on the image data, as opposed to reconstructing the full geometry of the subject. This reduces implementation complexity, and moreover, yields interactive performance. We demonstrate the effectiveness of our technique by incorporating it in a rigid body simulation. The subject can interact with virtual objects and observe his or her actions while being able to adjust the viewpoint, all in real-time.


  1. Allard, J., Franco, J., Menier, C., Boyer, E., and Raffin, B. (2006). The grimage platform: A mixed reality environment for interactions. In ICVS.
  2. Baraff, D. (1992). Dynamic simulation of non-penetrating rigid body simulation. PhD thesis.
  3. Bowman, D. and Hodges, L. (1997). Techniques for grabbing and manipulating remote objects in immersive virtual environments. In ACM Symposium on Interactive 3-D Graphics .
  4. Breen, D., Whitaker, R., and Tuceryan, M. (1996). Interactive occlusion and automatic object placement for augmented reality. In Computer Graphics Forum, Blackwell Publishers.
  5. Bridson, R., Fedkiw, R., , and Anderson, J. (2002). Robust treatment of collisions, contact and friction for cloth animation. In conference on Computer graphics and interactive techniques.
  6. Cheung, K. M., Kanade, T., Bouguet, J.-Y., and Holler, M. (2000). A real time system for robust 3d voxel reconstruction of human motions. In Proceedings of the 2000 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 7800), volume 2, pages 714 - 720.
  7. Debunne, G., Desbrun, M., Cani, M.-P., and Barr, A. H. (2001). Dynamic realtime deformations using space and time adaptive sampling. In ACM SIGGRAPH.
  8. Dewaele, G. and Cani, M.-P. (2004). Interactive global and local deformations for virtual clay. In Graphical Models.
  9. Govindaraju, N., Knott, D., Jain, N., Kabul, I., Tamstorf, R., Gayle, R., Lin, M., and Manocha, D. (2005). Interactive collision detection between deformable models using chromatic decomposition. In ACM SIGGRAPH.
  10. Grossman, T., Wigdor, D., and Balakrishnan, R. (2004). Multi-finger gestural interaction with 3d volumetric displays. In UIST 7804: Proceedings of the 17th annual ACM symposium on User interface software and technology, pages 61-70, New York, NY, USA. ACM Press.
  11. Guendelman, E., Bridson, R., , and Fedkiw, R. (2003). Nonconvex rigid bodies with stacking. In ACM SIGGRAPH.
  12. Hand, C. (1997). A survey of 3-d interaction techniques. In Computer Graphics Forum, Blackwell Publishers.
  13. Hasenfratz, J., Lapierre, M., Gascuel, J., and Boyer, E. (2003). Real-time capture, reconstruction and insertion into virtual world of human actors. In Vision, Video and Graphics Conference.
  14. Hasenfratz, J., Lapierre, M., and Sillion, F. (2004). A real-time system for full body interaction with virtual worlds. In Eurographics Symposium on Virtual Environments.
  15. Heidelberger, B., Teschner, M., and Gross, M. (2004). Detection of collisions and self-collisions using imagespace techniques. In WSCG.
  16. Kitamura, Y., Ogata, S., and Kishino, F. (2003). A manipulation environment of virtual and real objects using a magnetic metaphor. In ACM Symposium on Virtual Reality Software and Technology.
  17. Laurentini, A. (1994). The visual hull concept for silhouette based image understanding. In IEEE PAMI.
  18. Li, M., Magnor, M., and Seidel, H. (2004). Hardwareaccelerated rendering of photo hulls. In Computer Graphics Forum.
  19. Lindeman, R. W., Page, R., Yanagida, Y., and Sibert, J. L. (2004). Towards full-body haptic feedback: The design and deployment of a spatialized vibrotactile feedback system. In ACM Virtual Reality Software and Technology (VRST).
  20. Lok, B., Naik, S., Whitton, M., and Jr, F. B. (2003). Incorporating dynamic real objects into immersive virtual environments. In Proceedings of the 2003 symposium on Interactive 3D graphics.
  21. Losasso, F., Shinar, T., Selle, A., and Fedkiw, R. (2006). Multiple interacting liquids. In ACM SIGGRAPH.
  22. Matusik, W., Buehler, C., and McMillan, L. (2001). Polyhedral visual hulls for real-time rendering. In Eurographics Workshop on Rendering.
  23. ODE (2006). Open dynamics engine (ode).
  24. Pauly, M., Pai, D. K., and Guibas, L. J. (2004). Quasi-rigid objects in contact. In Eurographics/ACM SIGGRAPH Symposium on Computer Animation.
  25. Redon, S., Kim, Y. J., Lin, M. C., and Manocha, D. (2004). Fast continuous collision detection for articulated models. In ACM Symposium on Solid Modeling and Applications.
  26. Stam, J. (2000). Interacting with smoke and fire in real time. In Communications of the ACM.
  27. Svoboda, T., Hug, H., and Gool, L. V. (2002). Viroom - low cost synchronised multicamera system and its self-calibration. In Gool, L. V., editor, Pattern Recognition, 24th DAGM Symposium, LNCS, pages 512- 522. Springer.
  28. Teschner, M., Kimmerle, S., Heidelberger, B., Zachmann, G., Raghupathi, L., A. Fuhrmann, M.-P. C., Faure, F., Magnenat-Thalmann, N., Strasser, W., and Volino, P. (2005). Collision detection for deformable objects. In Computer Graphics Forum.
  29. Wang, X., Kotranza, A., Quarles, J., Lok, B., and Allen, B. D. (2005). Rapidly incorporating real objects for evaluation of engineering designs in a mixed reality environment. In 3D User Interfaces Workshop, IEEE Virtual Reality.

Paper Citation

in Harvard Style

De Decker B., Mertens T. and Bekaert P. (2007). INTERACTIVE COLLISION DETECTION FOR FREE-VIEWPOINT VIDEO . In Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP, ISBN 978-972-8865-72-6, pages 114-120. DOI: 10.5220/0002078901140120

in Bibtex Style

author={Bert De Decker and Tom Mertens and Philippe Bekaert},
booktitle={Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,},

in EndNote Style

JO - Proceedings of the Second International Conference on Computer Graphics Theory and Applications - Volume 2: GRAPP,
SN - 978-972-8865-72-6
AU - De Decker B.
AU - Mertens T.
AU - Bekaert P.
PY - 2007
SP - 114
EP - 120
DO - 10.5220/0002078901140120