NOVEL VIEW TELEPRESENCE WITH HIGH-SCALABILITY USING MULTI-CASTED OMNI-DIRECTIONAL VIDEOS

Tomoya Ishikawa, Kazumasa Yamazawa, Naokazu Yokoya

Abstract

The advent of high-speed network and high performance PCs has prompted research on networked telepresence, which allows a user to see virtualized real scenes in remote places. View-dependent representation, which provides a user with arbitrary view images using an HMD or an immersive display, is especially effective in creating a rich telepresence. The goal of our work is to realize a networked novel view telepresence system which enables multiple users to control the viewpoint and view-direction independently by virtualizing real dynamic environments. In this paper, we describe a novel view generation method from multiple omni-directional images captured at different positions. We mainly describe our prototype system with high-scalability which enables multiple users to use the system simultaneously and some experiments with the system. The novel view telepresence system constructs a virtualized environment from real live videos. The live videos are transferred to multiple users by using multi-cast protocol without increasing network traffic. The system synthesizes a view image for each user with a varying viewpoint and view-direction measured by a magnetic sensor attached to an HMD and presents the generated view on the HMD. Our system can generate the user’s view image in real-time by giving correspondences among omni-directional images and estimating camera intrinsic and extrinsic parameters in advance.

References

  1. Aliaga, D. G. and Carlbom, I. (2001). Plenoptic stitching: A scalable method for reconstructing 3D interactive walkthroughs. In Proc. of SIGGRAPH2001, pages 443-451.
  2. Chen, S. E. (1995). Quicktime VR: An image-based approach to virtual environment navigation. In Proceedings of SIGGRAPH'95, pages 29-38.
  3. Ishikawa, T., Yamazawa, K., Sato, T., Ikeda, S., Namamura, Y., Fujikawa, K., Sunahara, H., and Yokoya, N. (2005a). Networked telepresence system using web browsers and omni-directional video streams. In Proc. SPIE Electronic Imaging, volume 5664, pages 380- 387.
  4. Ishikawa, T., Yamazawa, K., and Yokoya, N. (2005b). Novel view generation from multiple omni-directional videos. In Proc. IEEE and ACM Int. Symp. on Mixed and Augmented Reality, pages 166-169.
  5. Kanade, T., Rander, P., and Narayanan, P. J. (1997). Virtualized reality: Constructing virtual worlds from real scenes. IEEE MultiMedia Magazine, 1(1):34-47.
  6. Koyama, T., Kitahara, I., and Ohta, Y. (2003). Live mixedreality 3D video in soccer stadium. In Proc. of IEEE and ACM Int. Symp. on Mixed and Augmented Reality, pages 178-187.
  7. Laurentini, A. (1994). The visual hull concept for silhouette-based image understanding. IEEE Trans. on Pattern Analysis and Machine Intelligence, 16(2):150-162.
  8. Matusik, W., Buehler, C., Raskar, R., Gortler, S. J., and McMillan, L. (2000). Image-based visual hulls. In Proc. of SIGGRAPH2000, pages 369-374.
  9. Morita, S., Yamazawa, K., and Yokoya, N. (2003). Networked video surveillance using multiple omnidirectional cameras. In Proc. 2003 IEEE Int. Symp. on Computational Intelligence in Robotics and Automation, pages 1245-1250.
  10. Negishi, Y., Miura, J., and Shirai, Y. (2004). Calibration of omnidirectional stereo for mobile robots. In Proc. of 2004 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pages 2600-2605.
  11. Onoe, Y., Yamazawa, K., Takemura, H., and Yokoya, N. (1998). Telepresence by real-time viewdependent image generation from omni-directional video streams. Computer Vision and Image Understanding, 71(2):154-165.
  12. Saito, H., Baba, S., and Kande, T. (2003). Appearancebased virtual view generation from multicamera videos captured in the 3-D room. IEEE Trans. on Multimedia, 5(3):303-316.
  13. Seitz, S. M. and Dyer, C. R. (1997). Photorealistic scene reconstruction by voxel coloring. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pages 1067-1073.
  14. Tomite, K., Yamazawa, K., and Yokoya, N. (2002). Arbitrary viewpoint rendering from multiple omnidirectional images for interactive walkthroughs. In Proc. of 16th IAPR Int. Conf. on Pattern Recognition, volume 3, pages 987-990.
  15. Uyttendaele, M., Criminisi, A., Kang, S. B., Winder, S., Szeliski, R., and Hartley, R. (2003). Imagebased interactive exploration of real-world environments. IEEE Computer Graphics and Applications, 24(3):52-63.
  16. Yamazawa, K., Yagi, Y., and Yachida, M. (1993). New real-time omni-directional image sensor with hyperboloidal mirror. In Proc. 8th Scandinavian Conf. on Image Analysis, volume 2, pages 1381-1387.
  17. Yamazawa, K. and Yokoya, N. (2003). Detecting moving objects from omni-directional dynamic images based on adaptive background subtraction. In Proc. 10th IEEE Int. Conf. on Image Processing, volume 3, pages 953-956.
  18. Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. (2004). High-quality video view interpolation using a layered representation. ACM Trans. on Graphics, 23(3):600-608.
Download


Paper Citation


in Harvard Style

Ishikawa T., Yamazawa K. and Yokoya N. (2007). NOVEL VIEW TELEPRESENCE WITH HIGH-SCALABILITY USING MULTI-CASTED OMNI-DIRECTIONAL VIDEOS . In Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 5: ICEIS, ISBN 978-972-8865-92-4, pages 148-156. DOI: 10.5220/0002359201480156


in Bibtex Style

@conference{iceis07,
author={Tomoya Ishikawa and Kazumasa Yamazawa and Naokazu Yokoya},
title={NOVEL VIEW TELEPRESENCE WITH HIGH-SCALABILITY USING MULTI-CASTED OMNI-DIRECTIONAL VIDEOS},
booktitle={Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 5: ICEIS,},
year={2007},
pages={148-156},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002359201480156},
isbn={978-972-8865-92-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 5: ICEIS,
TI - NOVEL VIEW TELEPRESENCE WITH HIGH-SCALABILITY USING MULTI-CASTED OMNI-DIRECTIONAL VIDEOS
SN - 978-972-8865-92-4
AU - Ishikawa T.
AU - Yamazawa K.
AU - Yokoya N.
PY - 2007
SP - 148
EP - 156
DO - 10.5220/0002359201480156