Interactive Musical Setting with Deep Learning and Object Recognition

Mário Cardoso, Rui Lopes

2020

Abstract

The SeMI - Interactive Musical Setting, explores the possibilities of joining machine learning, the physical and the sound world. In this context, a machine learning algorithm and model was used to identify physical objects through image processing. Each physical object is associated with a student’s produced musical texture that starts playing when the object is recognized by the device. This allows defining use cases in which students have to develop diverse although interrelated sound textures and combine them with a physical world, in both a fake orchestra, that reacts to people and objects in front of it, and mood rooms, for example. The application was developed for iPad and iPhone, using Swift programming language and the iOS operating system and used in the classes of the masters on Teaching of Musical Education in the Basic School.

Paper Citation

in Harvard Style

Cardoso M. and Lopes R. (2020). Interactive Musical Setting with Deep Learning and Object Recognition.In Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSME, ISBN 978-989-758-417-6, pages 663-667. DOI: 10.5220/0009856406630667

in Bibtex Style

@conference{csme20,
author={Mário Cardoso and Rui Lopes},
title={Interactive Musical Setting with Deep Learning and Object Recognition},
booktitle={Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSME,},
year={2020},
pages={663-667},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0009856406630667},
isbn={978-989-758-417-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 12th International Conference on Computer Supported Education - Volume 1: CSME,
TI - Interactive Musical Setting with Deep Learning and Object Recognition
SN - 978-989-758-417-6
AU - Cardoso M.
AU - Lopes R.
PY - 2020
SP - 663
EP - 667
DO - 10.5220/0009856406630667