Multisensory Analytics: Case of Visual-auditory Analysis of Scalar Fields

E. Malikova, V. Pilyugin, V. Adzhiev, G. Pasko, A. Pasko

2017

Abstract

A well-known definition of visualization is the mapping of initial data to a visual representation, which can be perceived and interpreted by humans. Human senses include not only vision, but also hearing, sense of touch, smell and others including their combinations. Visual analytics and its more general version that we call Multisensory Analytics are areas that consider visualization as one of its components. We present a particular case of the multisensory analytics with a hybrid visual-auditory representation of data to show how auditory display can be used in the context of data analysis. Some generalizations based on using real-valued vector functions for solving data analysis problems by means of multisensory analytics are proposed. These generalizations might be considered as a first step to formalization of the correspondence between the initial data and various sensory stimuli. An illustration of our approach with a case study of analysis of a scalar field using both visual and auditory data representations is given.

References

  1. Wong, P. C., Thomas J., 2004. Visual analytics, IEEE Computer Graphics and Applications, vol. 24, No. 5, pp. 20-21.
  2. Keim, D., Mansmann, F., Schneidewind, J., Thomas, J., Ziegler, H., 2008. Visual analytics: scope and challenges, Visual Data Mining, Lecture Notes in Computer Science, volume 4404, Springer, pp 76-90.
  3. Foley, J., Ribarsky, B., 1994. Next-generation data visualization tools, in Scientific Visualization, Advances and Challenges, L. Rosenblum et al. (Eds.), Academic Press.
  4. McCormick, B., DeFanti, T., Brown, M. (Eds.), 1987. Visualization in Scientific Computing, Computer Graphics, vol. 21, No. 6.
  5. Pilyugin, V., Malikova, E., Adzhiev, V., Pasko, A., 2013. Some theoretical issues of scientific visualization as a method of data analysis, Transactions on Computational Science XIX, Lecture Notes in Computer Science, vol. 7870, Springer-Verlag, pp. 131-142.
  6. Yeung, E., 1980. Pattern Recognition by Audio Representation of Multivariate Analytical Data, Analytical Chemistry, vol. 52, No.7, pp. 1120- 1123.
  7. Bly, S., 1982. Presenting information in sound, Proceedings of the CHI 7882 Conference on Human Factors in Computer Systems, ACM, pp. 371-375.
  8. Kaper, H., Wiebel, E., Tipei, S., 1999. Data Sonification and Sound Visualization, Computing in science and Engineering, vol.1, No.4, pp.48-58.
  9. Scaletti, C., Craig, A.B., 1991. Using Sound to Extract Meaning from Complex Data, In Proceedings SPIE, 1459, pp. 207-219.
  10. Mezrich, J. J., Frysinger, S., Slivjanovski, R., 1984. Dynamic representation of multivariate. Time Series data, Journal of the American Statistical Association, Vol. 79, N. 385. pp. 34-40.
  11. Lodha Suresh, K., Beahan, J., Heppe, T. and etc., 1997. MUSE: A Musical Data Sonification Toolkit, In Proceedings of International Conference on Auditory Display (ICAD), pp. 36-40.
  12. Grinstein, G., Smith S., 1990. Perceptualization of scientific data, Proc. SPIE 1259, Extracting Meaning from Complex Data: Processing, Display, Interaction, pp. 190-199.
  13. Ebert, D., 2004. Extending Visualization to Perceptualization: the Importance of Perception in Effective Communication of Information, in The Visualization Handbook, C. Hansen and C. Johnson (Eds.), Academic Press, pp. 771-780.
  14. Ogi, T., Hirose M., 1996. Multisensory Data Sensualization based on Human Perception, VRAIS 7896 Proceedings of the 1996 Virtual Reality Annual International Symposium, pp. 66-71.
  15. Jovanov, E., Starcevic, D., Radivojevic, V., Samardzic, A., Simeunovic, V., 1999. Perceptualization of Biomedical Data. An Experimental Environment for Visualization and Sonification of Brain Electrical activity, IEEE Engineering in Medicine and Biology Magazine, vol. 18, No. 1, pp. 50-55.
  16. Maciejewski, R., Choi, S., Ebert, D., Tan, H., 2005. MultiModal Perceptualization of Volumetric Data and its Application to Molecular Docking, WHC 7805 Proceedings of the First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 511-514.
  17. Adzhiev, V., Ossipov, A., Pasko, A., 1999. Multidimensional shape modeling in multimedia Applications, in MultiMedia Modeling: Modeling Multimedia Information and Systems, ed. A.Karmouch, World Scientific, pp. 39-60.
  18. Pasko, A., Adzhiev, V., Sourin, A., Savchenko, V., 1995. Function Representation in Geometric Modeling: Concepts, Implementation and Applications, The Visual Computer, vol.11, No.8, pp.429-446.
  19. Pasko, A., Adzhiev, V., Schmitt, B., Schlick, C., 2001. Constructive Hypervolume Modeling, Graphical Models, vol. 63, No. 6, pp. 413-442.
  20. Zavadska, G., Davidova, J., 2015. The Development of Prospective Music Teachers' Harmonic Hearing at Higher Education Establishments, Pedagogika / Pedagogy Vol. 117, No. 1, pp. 72-85, Lietovus Edukologijos Universitetas, Lituania.
  21. Wong, P.C., Bergeron, R.D., 1997. 30 Years of Multidimensional Multivariate Visualization, Proceeding Scientific Visualization, Overviews, Methodologies, and Techniques, EEE Computer Society Washington, DC, USA, pp. 3-33.
  22. OpenAL, 2016. Programmers Guide. Available at: http://connect.creativelabs.com/openal/Documentation /OpenAL_Programmers_Guide.pdf
Download


Paper Citation


in Harvard Style

Malikova E., Pilyugin V., Adzhiev V., Pasko G. and Pasko A. (2017). Multisensory Analytics: Case of Visual-auditory Analysis of Scalar Fields . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, (VISIGRAPP 2017) ISBN 978-989-758-228-8, pages 322-329. DOI: 10.5220/0006255003220329


in Bibtex Style

@conference{ivapp17,
author={E. Malikova and V. Pilyugin and V. Adzhiev and G. Pasko and A. Pasko},
title={Multisensory Analytics: Case of Visual-auditory Analysis of Scalar Fields},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, (VISIGRAPP 2017)},
year={2017},
pages={322-329},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006255003220329},
isbn={978-989-758-228-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, (VISIGRAPP 2017)
TI - Multisensory Analytics: Case of Visual-auditory Analysis of Scalar Fields
SN - 978-989-758-228-8
AU - Malikova E.
AU - Pilyugin V.
AU - Adzhiev V.
AU - Pasko G.
AU - Pasko A.
PY - 2017
SP - 322
EP - 329
DO - 10.5220/0006255003220329