The Natural Interactive Walking Project and Emergence of Its Results in Research on Rhythmic Walking Interaction and the Role of Footsteps in Affecting Body Ownership

Erik Sikström, Justyna Maculewicz, Stefania Serafin

2015

Abstract

In this chapter we describe how the results of the Natural Interactive Project, which was funded within the 7th Framework Programme and ended in 2011, started several research directions concerning the role of auditory and haptic feedback in footstep simulations. We chose elements of the project which are interesting in a broader context of interactive walking with audio and haptic feedback to present and discuss the developed systems for gait analysis and feedback presentation, but also, what is even more interesting to show how it influence humans behavior and perception. We hope also to open a discussion on why we actually can manipulate our behavior and show the importance of explaining it from the neurological perspective. We start with a general introduction, moving on to more specific parts of the project, that are followed by the results of the research which were conducted after project’s termination but based on its results.

References

  1. Li, X., Logan, R.J., Pastore, R.E.: Perception of acoustic source characteristics: Walking sounds. The Journal of the Acoustical Society of America 90 (1991) 3036-3049
  2. Giordano, B., Bresin, R.: Walking and playing: Whats the origin of emotional expressiveness in music. In: Proc. Int. Conf. Music Perception and Cognition. (2006)
  3. Pastore, R.E., Flint, J.D., Gaston, J.R., Solomon, M.J.: Auditory event perception: The sourceperception loop for posture in human gait. Perception & psychophysics 70 (2008) 13-29
  4. Giordano, B. L., McAdams, S., Visell, Y., Cooperstock, J., Yao, H. Y., Hayward, V.: Nonvisual identification of walking grounds. The Journal of the Acoustical Society of America 123 (2008) 3412-3412
  5. Young, W., Rodger, M., Craig, C.M.: Perceiving and reenacting spatiotemporal characteristics of walking sounds. Journal of Experimental Psychology: Human Perception and Performance 39 (2013) 464
  6. Turchet, L., Serafin, S., Cesari, P.: Walking pace affected by interactive sounds simulating stepping on different terrains. ACM Transactions on Applied Perception (TAP) 10 (2013) 23
  7. Maculewicz, J., Jylha, A., Serafin, S., Erkut, C.: The effects of ecological auditory feedback on rhythmic walking interaction. MultiMedia, IEEE 22 (2015) 24-31
  8. Nordahl, R., Turchet, L., Serafin, S.: Sound synthesis and evaluation of interactive footsteps and environmental sounds rendering for virtual reality applications. Visualization and Computer Graphics, IEEE Transactions on 17 (2011) 1234-1244
  9. De Dreu, M., Van Der Wilk, A., Poppe, E., Kwakkel, G., Van Wegen, E.: Rehabilitation, exercise therapy and music in patients with parkinson's disease: a meta-analysis of the effects of music-based movement therapy on walking ability, balance and quality of life. Parkinsonism & related disorders 18 (2012) S114-S119
  10. Thaut, M.H., Abiru, M.: Rhythmic auditory stimulation in rehabilitation of movement disorders: a review of current research. (2010)
  11. McIntosh, G.C., Brown, S.H., Rice, R.R., Thaut, M.H.: Rhythmic auditory-motor facilitation of gait patterns in patients with parkinson's disease. Journal of Neurology, Neurosurgery & Psychiatry 62 (1997) 22-26
  12. Suteerawattananon, M., Morris, G., Etnyre, B., Jankovic, J., Protas, E.: Effects of visual and auditory cues on gait in individuals with parkinson's disease. Journal of the neurological sciences 219 (2004) 63-69
  13. Roerdink, M., Lamoth, C.J., Kwakkel, G., Van Wieringen, P.C., Beek, P.J.: Gait coordination after stroke: benefits of acoustically paced treadmill walking. Physical Therapy 87 (2007) 1009-1022
  14. Nieuwboer, A., Kwakkel, G., Rochester, L., Jones, D., van Wegen, E., Willems, A.M., Chavret, F., Hetherington, V., Baker, K., Lim, I.: Cueing training in the home improves gait-related mobility in parkinsons disease: the rescue trial. Journal of Neurology, Neurosurgery & Psychiatry 78 (2007) 134-140
  15. Thaut, M., Leins, A., Rice, R., Argstatter, H., Kenyon, G., McIntosh, G., Bolay, H., Fetter, M.: Rhythmic auditor y stimulation improves gait more than ndt/bobath training in nearambulatory patients early poststroke: a single-blind, randomized trial. Neurorehabilitation and neural repair 21 (2007) 455-459
  16. Bank, P.J., Roerdink, M., Peper, C.: Comparing the efficacy of metronome beeps and stepping stones to adjust gait: steps to follow! Experimental brain research 209 (2011) 159-169
  17. Storms, R., Zyda, M.: Interactions in perceived quality of auditory-visual displays. Presence 9 (2000) 557-580
  18. Sanders Jr, R.D.: The effect of sound delivery methods on a users sense of presence in a virtual environment. PhD thesis, Naval Postgraduate School (2002)
  19. Chueng, P., Marsden, P.: Designing auditory spaces to support sense of place: the role of expectation. In: CSCW Workshop: The Role of Place in Shaping Virtual Community. Citeseer, Citeseer (2002)
  20. Freeman, J., Lessiter, J.: Hear there & everywhere: the effects of multi-channel audio on presence. In: Proceedings of ICAD. (2001) 231-234
  21. Västfjäll, D.: The subjective sense of presence, emotion recognition, and experienced emotions in auditory virtual environments. CyberPsychology & Behavior 6 (2003) 181-188
  22. Larsson, P., Västfjäll, D., Kleiner, M.: Perception of self-motion and presence in auditory virtual environments. In: Proceedings of Seventh Annual Workshop Presence 2004. (2004) 252-258
  23. Kapralos, B., Zikovitz, D., Jenkin, M.R., Harris, L.R.: Auditory cues in the perception of self motion. In: Audio Engineering Society Convention 116, Audio Engineering Society (2004)
  24. Väljamäe, A., Larsson, P., Västfjäll, D., Kleiner, M.: Travelling without moving: Auditory scene cues for translational self-motion. In: Proceedings of ICAD05. (2005)
  25. Tonetto, P.L.M., Klanovicz, C.P., Spence, P.C.: Modifying action sounds influences peoples emotional responses and bodily sensations. i-Perception 5 (2014) 153-163
  26. Bresin, R., de Witt, A., Papetti, S., Civolani, M., Fontana, F.: Expressive sonification of footstep sounds. In: Proceedings of ISon 2010: 3rd Interactive Sonification Workshop. (2010) 51-54
  27. Rocchesso, D., Bresin, R., Fernstrom, M.: Sounding objects. MultiMedia, IEEE 10 (2003) 42-52
  28. Avanzini, F., Rocchesso, D.: Modeling collision sounds: Non-linear contact force. In: Proc. COST-G6 Conf. Digital Audio Effects (DAFx-01). (2001) 61-66
  29. Cook, P.: Physically Informed Sonic Modeling (PhISM): Synthesis of Percussive Sounds. Computer Music Journal 21 (1997) 38-49
  30. Gaver, W.W.: What in the world do we hear?: An ecological approach to auditory event perception. Ecological psychology 5 (1993) 1-29
  31. Turchet, L., Serafin, S., Dimitrov, S., Nordahl, R.: Physically based sound synthesis and control of footsteps sounds. In: Proceedings of Digital Audio Effects Conference. (2010) 161-168
  32. Nordahl, R., Serafin, S., Turchet, L.: Sound synthesis and evaluation of interactive footsteps for virtual reality applications. In: Proceedings of the IEEE Virtual Reality Conference. (2010) 147-153
  33. Turchet, L., Serafin, S., Nordahl, R.: Examining the role of context in the recognition of walking sounds. In: Proceedings of Sound and Music Computing Conference. (2010)
  34. Turchet, L., Nordahl, R., Berrezag, A., Dimitrov, S., Hayward, V., Serafin, S.: Audio-haptic physically based simulation of walking on different grounds. In: Proceedings of IEEE International Workshop on Multimedia Signal Processing, IEEE Press (2010) 269-273
  35. Maculewicz, J., Cumhur, E., Serafin, S.: An investigation on the impact of auditory and haptic feedback on rhythmic walking interactions. International Journal of Human-Computer Studies 2015submitted.
  36. Maculewicz, J., Cumhur, E., Serafin, S.: The influence of soundscapes and footsteps sounds in affecting preferred walking pace. International Conference on Auditory Display 2015submitted.
  37. Maculewicz, J., Nowik, A., Serafin, S., Lise, K., Króliczak, G.: The effects of ecological auditory cueing on rhythmic walking interaction: Eeg study. (2015)
  38. Sikstr öm, E., de Götzen, A., Serafin, S.: Self-characteristics and sound in immersive virtual reality - estimating avatar weight from footstep sounds. In: Virtual Reality (VR), 2015 IEEE, IEEE (2015)
  39. Tajadura-Jiménez, A., Basia, M., Deroy, O., Fairhurst, M., Marquardt, N., BianchiBerthouze, N.: As light as your footsteps: altering walking sounds to change perceived body weight, emotional state and gait. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, ACM (2015) 2943-2952
  40. Zielinski, S.K., Rumsey, F., Bech, S., De Bruyn, B., Kassier, R.: Computer games and multichannel audio quality-the effect of division of attention between auditory and visual modalities. In: Audio Engineering Society Conference: 24th International Conference: Multichannel Audio, The New Reality, Audio Engineering Society (2003)
  41. Kassier, R., Zielinski, S.K., Rumsey, F.: Computer games and multichannel audio quality part 2'evaluation of time-variant audio degradations under divided and undivided attention. In: Audio Engineering Society Convention 115, Audio Engineering Society (2003)
  42. Reiter, U., Weitzel, M.: Influence of interaction on perceived quality in audiovisual applications: evaluation of cross-modal influence. In: Proc. 13th International Conference on Auditory Displays (ICAD), Montreal, Canada. (2007)
  43. Rumsey, F., Ward, P., Zielinski, S.K.: Can playing a computer game affect perception of audio-visual synchrony? In: Audio Engineering Society Convention 117, Audio Engineering Society (2004)
  44. Reiter, U., Weitzel, M.: Influence of interaction on perceived quality in audio visual applications: subjective assessment with n-back working memory task, ii. In: Audio Engineering Society Convention 122, Audio Engineering Society (2007)
  45. Reiter, U.: Toward a salience model for interactive audiovisual applications of moderate complexity. (audio mostly) 101
  46. Larsson, P., Vastfjall, D., Kleiner, M.: Ecological acoustics and the multi-modal perception of rooms: real and unreal experiences of auditory-visual virtual environments. (2001)
  47. Larsson, P., Västfjäll, D., Kleiner, M.: Auditory-visual interaction in real and virtual rooms. In: Proceedings of the Forum Acusticum, 3rd EAA European Congress on Acoustics, Sevilla, Spain. (2002)
Download


Paper Citation


in Harvard Style

Serafin S., Sikström E. and Maculewicz J. (2015). The Natural Interactive Walking Project and Emergence of Its Results in Research on Rhythmic Walking Interaction and the Role of Footsteps in Affecting Body Ownership . In European Project Space on Intelligent Systems, Pattern Recognition and Biomedical Systems - EPS Lisbon, ISBN 978-989-758-095-6, pages 3-24. DOI: 10.5220/0006162100030024


in Bibtex Style

@conference{eps lisbon15,
author={Stefania Serafin and Erik Sikström and Justyna Maculewicz},
title={The Natural Interactive Walking Project and Emergence of Its Results in Research on Rhythmic Walking Interaction and the Role of Footsteps in Affecting Body Ownership},
booktitle={European Project Space on Intelligent Systems, Pattern Recognition and Biomedical Systems - EPS Lisbon,},
year={2015},
pages={3-24},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006162100030024},
isbn={978-989-758-095-6},
}


in EndNote Style

TY - CONF
JO - European Project Space on Intelligent Systems, Pattern Recognition and Biomedical Systems - EPS Lisbon,
TI - The Natural Interactive Walking Project and Emergence of Its Results in Research on Rhythmic Walking Interaction and the Role of Footsteps in Affecting Body Ownership
SN - 978-989-758-095-6
AU - Serafin S.
AU - Sikström E.
AU - Maculewicz J.
PY - 2015
SP - 3
EP - 24
DO - 10.5220/0006162100030024