Combining Gesture and Voice Control for Mid-air Manipulation of CAD Models in VR Environments

Markus Friedrich, Stefan Langer, Fabian Frey

2021

Abstract

Modeling 3D objects in domains like Computer Aided Design (CAD) is time-consuming and comes with a steep learning curve needed to master the design process as well as tool complexities. In order to simplify the modeling process, we designed and implemented a prototypical system that leverages the strengths of Virtual Reality (VR) hand gesture recognition in combination with the expressiveness of a voice-based interface for the task of 3D modeling. Furthermore, we use the Constructive Solid Geometry (CSG) tree representation for 3D models within the VR environment to let the user manipulate objects from the ground up, giving an intuitive understanding of how the underlying basic shapes connect. The system uses standard mid-air 3D object manipulation techniques and adds a set of voice commands to help mitigate the deficiencies of current hand gesture recognition techniques. A user study was conducted to evaluate the proposed prototype. The combination of our hybrid input paradigm shows to be a promising step towards easier to use CAD modeling.

Paper Citation

in Harvard Style

Friedrich M., Langer S. and Frey F. (2021). Combining Gesture and Voice Control for Mid-air Manipulation of CAD Models in VR Environments. In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP; ISBN 978-989-758-488-6, SciTePress, pages 119-127. DOI: 10.5220/0010170501190127

in Bibtex Style

@conference{hucapp21,
author={Markus Friedrich and Stefan Langer and Fabian Frey},
title={Combining Gesture and Voice Control for Mid-air Manipulation of CAD Models in VR Environments},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP},
year={2021},
pages={119-127},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010170501190127},
isbn={978-989-758-488-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2021) - Volume 2: HUCAPP
TI - Combining Gesture and Voice Control for Mid-air Manipulation of CAD Models in VR Environments
SN - 978-989-758-488-6
AU - Friedrich M.
AU - Langer S.
AU - Frey F.
PY - 2021
SP - 119
EP - 127
DO - 10.5220/0010170501190127
PB - SciTePress