TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA

Tomoki Hayashi, Francois de Sorbier, Hideo Saito

Abstract

We present a method for overlaying a texture onto a non-rigid surface using a commodity depth camera. The depth cameras are able to capture 3-D data of a surface in real-time, and have several advantages compared with methods using only standard color cameras. However, it is not easy to register a 3-D deformable mesh to a point cloud of the non-rigid surface while keeping its geometrical topology. In order to solve this problem, our method starts by learning many representative meshes to generate surface deformation models. Then, while capturing 3-D data, we register a feasible 3-D mesh to the target surface and overlay a template texture onto the registered mesh. Even if the depth data are noisy or sparse, the learning-based method provides us with a smooth surface mesh. In addition, our method can be applied to real-time applications. In our experiments, we show some augmented reality results of texture overlay onto a non-textured T-shirt. In our experiments, we show some augmented reality results of texture overlay onto a non-textured T-shirt.

References

  1. Amberg, B., Romdhani, S., and Vetter, T. (2007). Optimal Step Nonrigid ICP Algorithms for Surface Registration. In Proc. CVPR, pages 1-8. Ieee.
  2. Azuma, R. (1997). A survey of augmented reality. PTVE, 6(4):355-385.
  3. Cai, Q., Gallup, D., Zhang, C., and Zhang, Z. (2010). 3D Deformable Face Tracking with a Commodity Depth Camera. In Proc. ECCV, number 2.
  4. Ehara, J. and Saito, H. (2006). Texture overlay for virtual clothing based on PCA of silhouettes. In Proc. ISMAR, pages 139-142. Ieee.
  5. Hilsmann, A. and Eisert, P. (2009). Tracking and Retexturing Cloth for Real-Time Virtual Clothing Applications. In Proc. MIRAGE, pages 1-12.
  6. Kim, Y. S., Lim, H., Kang, B., Choi, O., Lee, K., Kim, J. D. K., and Kim, C.-y. (2010). Realistic 3d face modeling using feature-preserving surface registration. In Proc. ICIP, pages 1821-1824.
  7. Li, H., Sumner, R. W., and Pauly, M. (2008). Global Correspondence Optimization for Non-Rigid Registration of Depth Scans. Computer Graphics Forum, 27(5):1421-1430.
  8. Papazov, C. and Burschka, D. (2011). Deformable 3D Shape Registration Based on Local Similarity Transforms. Computer Graphics Forum, 30(5):1493-1502.
  9. Perriollat, M., Hartley, R., and Bartoli, A. (2010). Monocular Template-based Reconstruction of Inextensible Surfaces. IJCV, 95(2):124-137.
  10. Pilet, J., Lepetit, V., and Fua, P. (2007). Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation. IJCV, 76(2):109-122.
  11. Salzmann, M. and Fua, P. (2011). Linear local models for monocular reconstruction of deformable surfaces. PAMI, 33(5):931-44.
  12. Salzmann, M., Pilet, J., Ilic, S., and Fua, P. (2007). Surface deformation models for nonrigid 3D shape recovery. PAMI, 29(8):1481-7.
  13. Shen, S., Shi, W., and Liu, Y. (2010). Monocular 3-D tracking of inextensible deformable surfaces under L(2) - norm. image processing, 19(2):512-21.
Download


Paper Citation


in Harvard Style

Hayashi T., de Sorbier F. and Saito H. (2012). TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012) ISBN 978-989-8565-04-4, pages 66-71. DOI: 10.5220/0003867200660071


in Bibtex Style

@conference{visapp12,
author={Tomoki Hayashi and Francois de Sorbier and Hideo Saito},
title={TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)},
year={2012},
pages={66-71},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003867200660071},
isbn={978-989-8565-04-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2012)
TI - TEXTURE OVERLAY ONTO NON-RIGID SURFACE USING COMMODITY DEPTH CAMERA
SN - 978-989-8565-04-4
AU - Hayashi T.
AU - de Sorbier F.
AU - Saito H.
PY - 2012
SP - 66
EP - 71
DO - 10.5220/0003867200660071