Magic Mirror: I and My Avatar - A Versatile Augmented Reality Installation Controlled by Hand Gestures

Alexander Seewald, Alexander Pfeiffer

2022

Abstract

Since 2012 we have been building the augmented reality system Magic Mirror based on Kinect V1 and V2’s native API. It relies on the magic mirror metaphor, where a large screen shows a mirrored camera view with overlaid graphical elements. In our case, it shows a different face mesh over the person’s face which reliably tracks face poses in real time while leaving the eyes and mouth of the person visible for interaction and to improve immersion; replaces the background with images that may be changed, smoothly zoomed and dragged; and allows to take screenshots which are automatically printed out on photo cards with an unique QR code linking to its digital twin. Control of the system is primarily via easily learned hand gestures very similar to multitouch screen gestures known from mobile phones and tablets. We have demonstrated the system to the public as well as in private over a wide variety of settings, faces and backgrounds. Here, we explain the challenges inherent in creating high-quality face meshes and textures from 2D images, and how we solved them; describe the different versions of the system, how they differ and their limitations; and demonstrate the usefulness of our system in several applications from people counting and tracking to obtaining height measurements without storing or processing personal data.

Paper Citation

in Harvard Style

Seewald A. and Pfeiffer A. (2022). Magic Mirror: I and My Avatar - A Versatile Augmented Reality Installation Controlled by Hand Gestures. In Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-547-0, pages 300-307. DOI: 10.5220/0010817500003116

in Bibtex Style

@conference{icaart22,
author={Alexander Seewald and Alexander Pfeiffer},
title={Magic Mirror: I and My Avatar - A Versatile Augmented Reality Installation Controlled by Hand Gestures},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2022},
pages={300-307},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010817500003116},
isbn={978-989-758-547-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Magic Mirror: I and My Avatar - A Versatile Augmented Reality Installation Controlled by Hand Gestures
SN - 978-989-758-547-0
AU - Seewald A.
AU - Pfeiffer A.
PY - 2022
SP - 300
EP - 307
DO - 10.5220/0010817500003116