VESPa: A Pattern-based Visual Query Language for Event Sequences

Florian Haag, Robert Krüger, Thomas Ertl

2016

Abstract

Movement data can often be enriched with additional information that enables analysts to ask new questions, for instance about POIs visited and meetings that imply interactions between persons. Information on spatio-temporal events such as visits or meetings can be especially valuable for digital forensics, marketing analysis, and urban planning. Most existing query languages for movement data, however, do not take that additional information into account. We address this gap by proposing VESPa, a pattern-based graphical query language to express, check, and refine hypotheses about spatio-temporal event sequences. Using VESPa, the analyst can sketch abstract assumptions and use the pattern to query the data for matches. The applicability of our approach is demonstrated in two case studies with different datasets. We also report on a small user study in which several construction and comprehension tasks were successfully solved in an interactive implementation of the concept.

References

  1. Abela, J., Debeaupuis, T., and Herve Schauer Consultants (1999). Universal format for logger messages. http://tools.ietf.org/html/draft-abela-ulm-05.
  2. Andrienko, N., Andrienko, G., and Fuchs, G. (2013). Towards privacy-preserving semantic mobility analysis. In EuroVis Workshop on Visual Analytics, pages 19- 23. Eurographics Association.
  3. Atrey, P., Maddage, M., and Kankanhalli, M. (2006). Audio based event detection for multimedia surveillance. In ICASSP 7806 Proc., volume 5, pages 813-816. IEEE.
  4. Atrey, P. K., Kankanhalli, M. S., and Jain, R. (2005). Timeline-based information assimilation in multimedia surveillance and monitoring systems. In Proc. VSSN 7805, pages 103-112. ACM.
  5. Bonhomme, C., Trépied, C., Aufaure, M.-A., and Laurini, R. (1999). A visual language for querying spatiotemporal databases. In Proc. GIS 7899, pages 34-39. ACM.
  6. Boyandin, I., Bertini, E., Bak, P., and Lalanne, D. (2011). Flowstrates: An approach for visual exploration of temporal origin-destination data. Comput. Graphics Forum, 30(3):971-980.
  7. Bracciale, L., Bonola, M., Loreti, P., Bianchi, G., Amici, R., and Rabuffi, A. (2014). CRAWDAD data set roma/- taxi (v. 2014-07-17). http://crawdad.org/roma/taxi/.
  8. Certo, L., Galva˜o, T., and Borges, J. (2013). Time Automaton: A visual mechanism for temporal querying. J. Visual Lang. Comput., 24(1):24-36.
  9. Dionisio, J. D. and Cárdenas, A. F. (1996). MQuery: A visual query language for multimedia, timeline and simulation data. J. Visual Lang. Comput., 7(4):377 - 401.
  10. Do, Q. X., Lu, W., and Roth, D. (2012). Joint inference for event timeline construction. In Proc. EMNLP-CoNLL 7812, pages 677-687. ACL.
  11. D'Ulizia, A., Ferri, F., and Grifoni, P. (2012). Moving GeoPQL: A pictorial language towards spatiotemporal queries. GeoInformatica, 16(2):357-389.
  12. Fails, J., Karlson, A., Shahamat, L., and Shneiderman, B. (2006). A visual interface for multivariate temporal data: Finding patterns of events across multiple histories. In VAST 7806, pages 167-174.
  13. Fegeras, L. (1999). VOODOO: A visual object-oriented database language for odmg oql. In W13. The First ECOOP Workshop on Object-Oriented Databases.
  14. Fischer, F., Mansmann, F., and Keim, D. A. (2012). Realtime visual analytics for event data streams. In Proc. SAC 7812, pages 801-806. ACM.
  15. Gaaloul, W., Bhiri, S., and Godart, C. (2004). Discovering workflow transactional behavior from event-based log. In On the Move to Meaningful Internet Systems 2004: CoopIS, DOA, and ODBASE, volume 3290 of LNCS, pages 3-18. Springer.
  16. Gotz, D. and Stavropoulos, H. (2014). DecisionFlow: Visual analytics for high-dimensional temporal event sequence data. IEEE TVCG, 20(12):1783-1792.
  17. Guo, H., Wang, Z., Yu, B., Zhao, H., and Yuan, X. (2011). TripVista: Triple perspective visual trajectory analytics and its application on microscopic traffic data at a road intersection. In Proc. PacificVis 7811, pages 163- 170. IEEE.
  18. Guo, X., Li, J., Yang, R., and Ma, X. (2014). NEI: A framework for dynamic news event exploration and visualization. In Proc. VINCI 7814, pages 121-128. ACM.
  19. Havre, S., Hetzler, B., and Nowell, L. (2000). ThemeRiver: Visualizing theme changes over time. In Proc. InfoVis 7800, pages 115-123. IEEE.
  20. Heydekorn, J., Nitsche, M., Dachselt, R., and Nürnberger, A. (2011). On the interactive visualization of a logistics scenario: Requirements and possible solutions. In Proc. IWDE 7811, Technical report (Internet): Elektronische Zeitschriftenreihe der Fakultät f ür Informatik der OVGU Magdeburg, pages 1-7.
  21. Huang, Y., Zhang, L., and Zhang, P. (2008). A framework for mining sequential patterns from spatio-temporal event data sets. IEEE Trans. Knowl. Data Eng., 20(4):433-448.
  22. Jiang, F., Yuan, J., Tsaftaris, S. A., and Katsaggelos, A. K. (2011). Anomalous video event detection using spatiotemporal context. Comput. Vision Image Understanding, 115(3):323-333.
  23. Jin, J. and Szekely, P. (2009). QueryMarvel: A visual query language for temporal patterns using comic strips. In Proc. VL/HCC 7809, pages 207-214.
  24. Kapler, T. and Wright, W. (2005). Geotime information visualization. Information Visualization, 4(2):136-146.
  25. Kim, P. H. and Giunchiglia, F. (2012). Life logging practice for human behavior modeling. In Proc. SMC 7812, pages 2873-2878.
  26. Krstajic, M., Bertini, E., and Keim, D. (2011). CloudLines: Compact display of event episodes in multiple timeseries. IEEE TVCG, 17(12):2432-2439.
  27. Krüger, R., Herr, D., Haag, F., and Ertl, T. (2015). Inspector Gadget: Integrating data preprocessing and orchestration in the visual analysis loop. In EuroVis Workshop on Visual Analytics (EuroVA). The Eurographics Association.
  28. Krüger, R., Thom, D., and Ertl, T. (2014). Visual analysis of movement behavior using web data for context enrichment. In Proc. PacificVis 7814, pages 193-200. IEEE.
  29. Krüger, R., Thom, D., Wörner, M., Bosch, H., and Ertl, T. (2013). TrajectoryLenses - A set-based filtering and exploration technique for long-term trajectory data. Comput. Graphics Forum, 2013(3):451-460.
  30. Kumar, C., Heuten, W., and Boll, S. (2013). Geographical queries beyond conventional boundaries: Regional search and exploration. In Proc. GIR 7813, pages 84- 85. ACM.
  31. Kumar, V., Furuta, R., and Allen, R. B. (1998). Metadata visualization for digital libraries: Interactive timeline editing and review. In Proc. DL 7898, pages 126-133. ACM.
  32. Makanju, A., Zincir-Heywood, A. N., and Milios, E. E. (2011). Storage and retrieval of system log events using a structured schema based on message type transformation. In Proc. SAC 7811, pages 528-533. ACM.
  33. Marcus, A., Bernstein, M. S., Badar, O., Karger, D. R., Madden, S., and Miller, R. C. (2011). Twitinfo: Aggregating and visualizing microblogs for event exploration. In Proc. CHI 7811, pages 227-236. ACM.
  34. Monroe, M., Lan, R., Morales del Olmo, J., Shneiderman, B., Plaisant, C., and Millstein, J. (2013). The challenges of specifying intervals and absences in temporal queries: A graphical language approach. In Proc. CHI 7813, pages 2349-2358. ACM.
  35. Morris, A., Abdelmoty, A., El-Geresy, B., and Jones, C. (2004). A filter flow visual querying language and interface for spatial databases. GeoInformatica, 8(2):107-141.
  36. Nguyen, T., Loke, S., and Torabi, T. (2007). The Community Stack: Concept and prototype. In Proc. AINAW 7807, volume 2, pages 52.-58.
  37. Parent, C., Spaccapietra, S., Renso, C., Andrienko, G., Andrienko, N., Bogorny, V., Damiani, M. L., GkoulalasDivanis, A., Macedo, J., Pelekis, N., Theodoridis, Y., and Yan, Z. (2013). Semantic trajectories modeling and analysis. ACM Comput. Surv., 45(4):42:1-42:32.
  38. Peuquet, D. J. and Duan, N. (1995). An event-based spatiotemporal data model (ESTDM) for temporal analysis of geographical data. Int. J. Geogr. Inf. Syst., 9(1):7-24.
  39. Pirolli, P. and Card, S. (2005). The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proc. Int'l Conf. on Intelligence Analysis, pages 2-4. MITRE.
  40. Plaisant, C., Milash, B., Rose, A., Widoff, S., and Shneiderman, B. (1996). LifeLines: Visualizing personal histories. In Proc. CHI 7896, pages 221-227. ACM.
  41. Russell, A., Smart, P., Braines, D., and Shadbolt, N. (2008). NITELIGHT: A graphical tool for semantic query construction. In Proc. SWUI 7808, volume 543 of CEUR-WS.
  42. Seifert, I. (2011). A pool of queries: Interactive multidimensional query visualization for information seeking in digital libraries. Information Visualization, 10(2):97- 106.
  43. Shneiderman, B. (1994). Dynamic Queries for Visual Information Seeking. IEEE Software, 11(6):70-77.
  44. Soylu, A., Giese, M., Jimenez-Ruiz, E., Kharlamov, E., Zheleznyakov, D., and Horrocks, I. (2013). OptiqueVQS: Towards an ontology-based visual query system for big data. In Proc. MEDES 7813, pages 119- 126. ACM.
  45. Sun, G., Liu, Y., Wu, W., Liang, R., and Qu, H. (2014). Embedding temporal display into maps for occlusionfree visualization of spatio-temporal data. In Proc. PacificVis 7814, pages 185-192. IEEE.
  46. Tao, C., Wongsuphasawat, K., Clark, K., Plaisant, C., Shneiderman, B., and Chute, C. G. (2012). Towards event sequence representation, reasoning and visualization for EHR data. In Proc. IHI 7812, pages 801- 806. ACM.
  47. Tominski, C., Schumann, H., Andrienko, G., and Andrienko, N. (2012). Stacking-based visualization of trajectory attribute data. IEEE TVCG, 18(12):2565- 2574.
  48. Visual Analytics Community (2014). VAST 2014 Challenge - the Kronos incident. http://va community.org/VAST+Challenge+2014.
  49. Westermann, U. and Jain, R. (2007). Toward a common event model for multimedia applications. IEEE MultiMedia, 14(1):19-29.
  50. Wongsuphasawat, K., Plaisant, C., Taieb-Maimon, M., and Shneiderman, B. (2012). Querying event sequences by exact match or similarity search: Design and empirical evaluation. Interact. Comput., 24(2):55-68.
  51. Wu, S., Otmane, S., Moreau, G., and Servières, M. (2013). Design of a visual query language for geographic information system on a touch screen. In HumanComputer Interaction. Interaction Modalities and Techniques, volume 8007 of LNCS, pages 530-539. Springer.
  52. Zgraggen, E., Drucker, S. M., Fisher, D., and DeLine, R. (2015). (s-qu)eries: Visual regular expressions for querying and exploring event sequences. In Proc. CHI 7815, pages 2683-2692. ACM.
  53. Zhu, X. Y., Guo, W., Huang, L., Hu, T., and Gao, W. X. (2013). Pan-information location map. ISPRS Archives, XL-4(4):57-62.
Download


Paper Citation


in Harvard Style

Haag F., Krüger R. and Ertl T. (2016). VESPa: A Pattern-based Visual Query Language for Event Sequences . In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: IVAPP, (VISIGRAPP 2016) ISBN 978-989-758-175-5, pages 48-59. DOI: 10.5220/0005716900480059


in Bibtex Style

@conference{ivapp16,
author={Florian Haag and Robert Krüger and Thomas Ertl},
title={VESPa: A Pattern-based Visual Query Language for Event Sequences},
booktitle={Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: IVAPP, (VISIGRAPP 2016)},
year={2016},
pages={48-59},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005716900480059},
isbn={978-989-758-175-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: IVAPP, (VISIGRAPP 2016)
TI - VESPa: A Pattern-based Visual Query Language for Event Sequences
SN - 978-989-758-175-5
AU - Haag F.
AU - Krüger R.
AU - Ertl T.
PY - 2016
SP - 48
EP - 59
DO - 10.5220/0005716900480059