Non-rigid Surface Tracking for Virtual Fitting System

Naoki Shimizu, Takumi Yoshida, Tomoki Hayashi, Francois de Sorbier, Hideo Saito


In this paper, we describe a method for overlaying a texture onto a T-shirt, for improving current virtual fitting system. In such systems, users can try on clothes virtually. In order to realize such a system, a depth camera has been used. These depth cameras can capture 3D data in real time and have been used by some industrial virtual cloth fitting systems. However, these systems roughly, or just do not, consider the shape of the clothes that user is wearing. So the appearance of these virtual fitting systems looks unnaturally. For a better fitting, we need to estimate 3D shape of cloth surface, and overlay a texture of the cloth that the user wants to see onto the surface. There are some methods that register a 3D deformable mesh onto captured depth data of a target surface. Although those registration methods are very accurate, most of them require large amount of processing time or either manually-set markers or special rectangles. The main contribution of our method is to overlay a texture onto a texture of T-shirt in real-time without modifying the surface.


  1. Bay, H., Tuytelaars, T., and Van Gool, L. (2006). Surf: Speeded up robust features. Computer Vision-ECCV 2006, pages 404-417.
  2. Chum, O., Matas, J., and Kittler, J. (2003). Locally optimized ransac. Pattern Recognition, pages 236-243.
  3. Hayashi, T., de Sorbier, F., and Saito, H. (2012). Texture overlay onto non-rigid surface using commodity depth camera. In VISAPP.
  4. Hilsmann, A. and Eisert, P. (2009). Tracking and retexturing cloth for real-time virtual clothing applications. In Proceedings of the 4th International Conference on Computer Vision/Computer Graphics CollaborationTechniques, pages 94-105. Springer-Verlag.
  5. Lowe, D. (1999). Object recognition from local scaleinvariant features. In Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on, volume 2, pages 1150-1157. Ieee.
  6. Pilet, J., Lepetit, V., and Fua, P. (2007). Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation. IJCV, 76(2):109-122.
  7. Salzmann, M. and Fua, P. (2011). Linear local models for monocular reconstruction of deformable surfaces. PAMI, 33(5):931-44.
  8. Salzmann, M., Pilet, J., Ilic, S., and Fua, P. (2007). Surface deformation models for nonrigid 3D shape recovery. PAMI, 29(8):1481-7.
  9. Shen, S., Shi, W., and Liu, Y. (2010). Monocular 3-D tracking of inextensible deformable surfaces under L(2) - norm. image processing, 19(2):512-21.
  10. Yoshida, T., Saito, H., Shimizu, M., and Taguchi, A. (2013). Stable keypoint recognition using viewpoint generative learning. In VISAPP.

Paper Citation

in Harvard Style

Shimizu N., Yoshida T., Hayashi T., Sorbier F. and Saito H. (2013). Non-rigid Surface Tracking for Virtual Fitting System . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2013) ISBN 978-989-8565-48-8, pages 12-18. DOI: 10.5220/0004303000120018

in Bibtex Style

author={Naoki Shimizu and Takumi Yoshida and Tomoki Hayashi and Francois de Sorbier and Hideo Saito},
title={Non-rigid Surface Tracking for Virtual Fitting System},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2013)},

in EndNote Style

JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2013)
TI - Non-rigid Surface Tracking for Virtual Fitting System
SN - 978-989-8565-48-8
AU - Shimizu N.
AU - Yoshida T.
AU - Hayashi T.
AU - Sorbier F.
AU - Saito H.
PY - 2013
SP - 12
EP - 18
DO - 10.5220/0004303000120018