Authors:
Naoki Nozawa
1
;
Hubert P. H. Shum
2
;
Edmond S. L. Ho
2
and
Shigeo Morishima
3
Affiliations:
1
Department of Pure and Applied Physics, Waseda University, Tokyo, Japan
;
2
Department of Computer and Information Sciences, Northumbria University, Newcastle upon Tyne, U.K.
;
3
Waseda Research Institute for Science and Engineering, Waseda University, Tokyo, Japan
Keyword(s):
Deep Learning, Lazy Learning, 3D Reconstruction, Sketch-based Interface, Car.
Abstract:
Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketch image. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deep neural network that takes a 2D sketch and generates a set of multi-view depth and mask images, which form a more effective representation comparing to 3D meshes, and can be effectively fused to generate a 3D car shape. Since global models like deep learning have limited capacity to reconstruct fine-detail features, we propose a local lazy learning approach that constructs a small subspace based on a few relevant car samples in the database. Due to the small size of such a subspace, fine details can be represented effectively with a small number of parameters. With a low-cost optimization process, a high-quality car shape with detailed features is created. Experimen
tal results show that the system performs consistently to create highly realistic cars of substantially different shape and topology.
(More)