Video-based Feedback for Assisting Physical Activity

Renato Baptista, Michel Antunes, Djamila Aouada, Björn Ottersten

2017

Abstract

In this paper, we explore the concept of providing feedback to a user moving in front of a depth camera so that he is able to replicate a specific template action. This can be used as a home based rehabilitation system for stroke survivors, where the objective is for patients to practice and improve their daily life activities. Patients are guided in how to correctly perform an action by following feedback proposals. These proposals are presented in a human interpretable way. In order to align an action that was performed with the template action, we explore two different approaches, namely, Subsequence Dynamic Time Warping and Temporal Commonality Discovery. The first method aims to find the temporal alignment and the second one discovers the interval of the subsequence that shares similar content, after which standard Dynamic Time Warping can be used for the temporal alignment. Then, feedback proposals can be provided in order to correct the user with respect to the template action. Experimental results show that both methods have similar accuracy rate and the computational time is a decisive factor, where Subsequence Dynamic Time Warping achieves faster results.

References

  1. Antunes, M., Aouada, D., and Ottersten, B. (2016a). A revisit to human action recognition from depth sequences: Guided svm-sampling for joint selection.
  2. Antunes, M., Baptista, R., Demisse, G., Aouada, D., and Ottersten, B. (2016b). Visual and human-interpretable feedback for assisting physical activity. In European Conference on Computer Vision (ECCV) Workshop on Assistive Computer Vision and Robotics Amsterdam,.
  3. Burke, J. W., McNeill, M., Charles, D., Morrow, P., Crosbie, J., and McDonough, S. (2009). Serious games for upper limb rehabilitation following stroke. In Proceedings of the 2009 Conference in Games and Virtual Worlds for Serious Applications, VS-GAMES 7809, pages 103-110, Washington, DC, USA. IEEE Computer Society.
  4. Chaaraoui, A. A., Climent-Pérez, P., and Fl órez-Revuelta, F. (2012). A review on vision techniques applied to human behaviour analysis for ambient-assisted living. Expert Systems with Applications.
  5. Chu, W.-S., Zhou, F., and De la Torre, F. (2012). Unsupervised temporal commonality discovery. In ECCV.
  6. Gupta, A., He, J., Martinez, J., Little, J. J., and Woodham, R. J. (2016). Efficient video-based retrieval of human motion with flexible alignment. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).
  7. Hondori, H. M., Khademi, M., Dodakian, L., Cramer, S. C., and Lopes, C. V. (2013). A spatial augmented reality rehab system for post-stroke hand rehabilitation. In MMVR.
  8. Kato, P. M. (2010). Video Games in Health Care: Closing the Gap. Review of General Psychology, 14:113-121.
  9. Kulkarni, K., Evangelidis, G., Cech, J., and Horaud, R. (2015). Continuous action recognition based on sequence alignment. International Journal of Computer Vision, 112(1):90-114.
  10. Kwakkel, G., Kollen, B. J., and Krebs, H. I. (2007). Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review. Neurorehabilitation and neural repair.
  11. Langhorne, P., Taylor, G., Murray, G., Dennis, M., Anderson, C., Bautz-Holter, E., Dey, P., Indredavik, B., Mayo, N., Power, M., et al. (2005). Early supported discharge services for stroke patients: a meta-analysis of individual patients' data. The Lancet.
  12. Mousavi Hondori, H. and Khademi, M. (2014). A review on technical and clinical impact of microsoft kinect on physical therapy and rehabilitation. Journal of Medical Engineering, 2014.
  13. M üller, M. (2007). Dynamic Time Warping. Springer.
  14. Ofli, F., Kurillo, G., Obdrz álek, S., Bajcsy, R., Jimison, H. B., and Pavel, M. (2016). Design and evaluation of an interactive exercise coaching system for older adults: Lessons learned. IEEE J. Biomedical and Health Informatics.
  15. Pirsiavash, H., Vondrick, C., and Torralba, A. (2014). Assessing the quality of actions. In Computer VisionECCV 2014, pages 556-571. Springer.
  16. Rakthanmanon, T., Campana, B., Mueen, A., Batista, G., Westover, B., Zhu, Q., Zakaria, J., and Keogh, E. (2012). Searching and mining trillions of time series subsequences under dynamic time warping. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 7812, pages 262-270, New York, NY, USA. ACM.
  17. Sucar, L. E., Luis, R., Leder, R., Hernandez, J., and Sanchez, I. (2010). Gesture therapy: a visionbased system for upper extremity stroke rehabilitation. In Engineering in Medicine and Biology Society (EMBC), 2010 Annual International Conference of the IEEE.
  18. Sun, F., Norman, I. J., and While, A. E. (2013). Physical activity in older people: a systematic review. BMC Public Health.
  19. Tao, L., Paiement, A., Damen, D., Mirmehdi, M., Hannuna, S., Camplani, M., Burghardt, T., and Craddock, I. (2016). A comparative study of pose representation and dynamics modelling for online motion quality assessment. Computer Vision and Image Understanding.
  20. Veerbeek, J. M., van Wegen, E., van Peppen, R., van der Wees, P. J., Hendriks, E., Rietberg, M., and Kwakkel, G. (2014). What is the evidence for physical therapy poststroke? a systematic review and meta-analysis. PloS one.
  21. Vemulapalli, R., Arrate, F., and Chellappa, R. (2014). Human action recognition by representing 3d skeletons as points in a lie group. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
  22. Wang, R., Medioni, G., Winstein, C., and Blanco, C. (2013). Home monitoring musculo-skeletal disorders with a single 3d sensor. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 521-528.
  23. Xia, L., Chen, C., and Aggarwal, J. (2012). View invariant human action recognition using histograms of 3d joints. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2012 IEEE Computer Society Conference on, pages 20-27. IEEE.
  24. Zhou, F. and Torre, F. (2009). Canonical time warping for alignment of human behavior. In Bengio, Y., Schuurmans, D., Lafferty, J. D., Williams, C. K. I., and Culotta, A., editors, Advances in Neural Information Processing Systems 22, pages 2286-2294. Curran Associates, Inc.
  25. Zhou, H. and Hu, H. (2008). Human motion tracking for rehabilitationa survey. Biomedical Signal Processing and Control, 3(1):1-18.
Download


Paper Citation


in Harvard Style

Baptista R., Antunes M., Aouada D. and Ottersten B. (2017). Video-based Feedback for Assisting Physical Activity . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-226-4, pages 274-280. DOI: 10.5220/0006132302740280


in Bibtex Style

@conference{visapp17,
author={Renato Baptista and Michel Antunes and Djamila Aouada and Björn Ottersten},
title={Video-based Feedback for Assisting Physical Activity},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={274-280},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006132302740280},
isbn={978-989-758-226-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 5: VISAPP, (VISIGRAPP 2017)
TI - Video-based Feedback for Assisting Physical Activity
SN - 978-989-758-226-4
AU - Baptista R.
AU - Antunes M.
AU - Aouada D.
AU - Ottersten B.
PY - 2017
SP - 274
EP - 280
DO - 10.5220/0006132302740280