A Personal Analytics Platform for the Internet of Things - Implementing Kappa Architecture with Microservice-based Stream Processing

Theo Zschörnig, Robert Wehlitz, Bogdan Franczyk

2017

Abstract

The foundation of the Internet of Things (IoT) consists of different devices, equipped with sensors, actuators and tags. With the emergence of IoT devices and home automation, advantages from data analysis are not limited to businesses and industry anymore. Personal analytics focus on the use of data created by individuals and used by them. Current IoT analytics architectures are not designed to respond to the needs of personal analytics. In this paper, we propose a lightweight flexible analytics architecture based on the concept of the Kappa Architecture and microservices. It aims to provide an analytics platform for huge numbers of different scenarios with limited data volume and different rates in data velocity. Furthermore, the motivation for and challenges of personal analytics in the IoT are laid out and explained as well as the technological approaches we use to overcome the shortcomings of current IoT analytics architectures.

References

  1. Accenture. (2016). Igniting Growth in Consumer Technology. Retrieved from https://www. accenture.com/_acnmedia/PDF-3/Accenture-IgnitingGrowth-in-Consumer-Technology.pdf.
  2. Alrehamy, H., & Walker, C. (2015). Personal Data Lake With Data Gravity Pull. In 2015 IEEE Fifth International Conference on Big Data and Cloud Computing (BDCloud) (pp. 160-167).
  3. Chang, H.-T., Mishra, N., & Lin, C.-C. (2015). IoT BigData Centred Knowledge Granule Analytic and Cluster Framework for BI Applications: A Case Base Analysis. PloS one, 10(11).
  4. Cheng, B., Longo, S., Cirillo, F., Bauer, M., & Kovacs, E. (2015). Building a Big Data Platform for Smart Cities: Experience and Lessons from Santander. In B. Carminati (Ed.), 2015 IEEE International Congress on Big Data (BigData Congress). New York, USA (pp. 592-599). Piscataway, NJ: IEEE.
  5. Choe, E. K., Lee, N. B., Lee, B., Pratt, W., & Kientz, J. A. (2014). Understanding quantified-selfers' practices in collecting and exploring personal data. In M. Jones, P. Palanque, A. Schmidt, & T. Grossman (Eds.), The 32nd Annual ACM Conference on Human Factors in Computing Systems (pp. 1143-1152).
  6. Fang, H. (2015). Managing data lakes in big data era: What's a data lake and why has it became popular in data management ecosystem. In 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), (pp. 820-824).
  7. Fetzer, C. (2016). Building Critical Applications Using Microservices. IEEE Security & Privacy, 14(6), 86-89.
  8. Greenough, J. (2016). How the 'Internet of Things' will impact consumers, businesses, and governments in 2016 and beyond. Retrieved from http://www.businessinsider.com/how-the-internet-ofthings-market-will-grow-2014-10?IR=T.
  9. Hasan, T., Kikiras, P., Leonardi, A., Ziekow, H., & Daubert, J. (2015). Cloud-based IoT Analytics for the Smart Grid: Experiences from a 3-year Pilot. In D. G. Michelson, A. L. Garcia, W.-B. Zhang, J. Cappos, & M. E. Darieby (Eds.), Proceedings of the 10th International Conference on Testbeds and Research Infrastructures for the Development of Networks & Communities.
  10. Jaramillo, D., Nguyen, D. V., & Smart, R. (2016). Leveraging microservices architecture by using Docker technology. In SoutheastCon 2016. (pp. 1-5).
  11. Kreps, J. (2014). Questioning the Lambda Architecture. Retrieved from https://www.oreilly.com/ ideas/questioning-the-lambda-architecture.
  12. Lewis, J., & Fowler, M. (2014). Microservices: a definition of this new architectural term. Retrieved from http:// www.martinfowler.com/articles/microservices.html.
  13. Mineraud, J., Mazhelis, O., Su, X., & Tarkoma, S. (2015). A gap analysis of Internet-of-Things platforms. arXiv preprint arXiv:1502.01181.
  14. Mishra, N., Chang, H.-T., & Lin, C.-C. (2015). An IoT Knowledge Reengineering Framework for Semantic Knowledge Analytics for BI-Services. Mathematical Problems in Engineering, 2015(1), 1-12.
  15. Naqishbandi, T., Sheriff, I. C., & Sama, Q. (2015). Big Data, CEP and IoT: Redefining Holistic Healthcare Information Systems and Analytics. International Journal of Engineering Research & Technology, 4(1).
  16. Qanbari, S., Behinaein, N., Rahimzadeh, R., & Dustdar, S. (2015). Gatica: Linked Sensed Data Enrichment and Analytics Middleware for IoT Gateways. In 2015 3rd International Conference on Future Internet of Things and Cloud (FiCloud) (pp. 38-43).
  17. Pasupuleti, P., & Purra, B. S. (2015). Data Lake Development with Big Data: Packt Publishing.
  18. Ramakrishnan, R., & Gaur, L. (2016). Smart electricity distribution in residential areas: Internet of Things (IoT) based advanced metering infrastructure and cloud analytics. In 2016 International Conference on Internet of Things and Applications (IOTA) (pp. 46-51).
  19. Riggins, F. J., & Wamba, S. F. (2015). Research Directions on the Adoption, Usage, and Impact of the Internet of Things through the Use of Big Data Analytics. In 2015 48th Hawaii International Conference on System Sciences (HICSS) (pp. 1531-1540).
  20. Rozik, A. S., Tolba, A. S., & El-Dosuky, M. A. (2016). Design and Implementation of the Sense Egypt Platform for Real-Time Analysis of IoT Data Streams. Advances in Internet of Things, 06(04), 65-91.
  21. Ruckenstein, M. (2014). Visualized and Interacted Life: Personal Analytics and Engagements with Data Doubles. Societies, 4(1), 68-84.
  22. Stolpe, M. (2016). The Internet of Things: Opportunities and Challenges for Distributed Data Analysis. ACM SIGKDD Explorations Newsletter, 18(1), 15-34.
  23. Swan, M. (2012). Sensor Mania!: The Internet of Things, Wearable Computing, Objective Metrics, and the Quantified Self 2.0. Journal of Sensor and Actuator Networks, 1(3), 217-253.
  24. Ueda, T., Nakaike, T., & Ohara, M. (2016). Workload characterization for microservices. In 2016 IEEE International Symposium on Workload Characterization (IISWC) (pp. 1-10). IEEE.
  25. Wingerath, W., Gessert, F., Friedrich, S., & Ritter, N. (2016). Real-time stream processing for Big Data. it - Information Technology, 58(4).
  26. Xu, Q., Aung, K. M. M., Zhu, Y., & Yong, K. L. (2016). Building a large-scale object-based active storage platform for data analytics in the internet of things. The Journal of Supercomputing, 72(7), 2796-2814.
Download


Paper Citation


in Harvard Style

Zschörnig T., Wehlitz R. and Franczyk B. (2017). A Personal Analytics Platform for the Internet of Things - Implementing Kappa Architecture with Microservice-based Stream Processing . In Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS, ISBN 978-989-758-248-6, pages 733-738. DOI: 10.5220/0006355407330738


in Bibtex Style

@conference{iceis17,
author={Theo Zschörnig and Robert Wehlitz and Bogdan Franczyk},
title={A Personal Analytics Platform for the Internet of Things - Implementing Kappa Architecture with Microservice-based Stream Processing},
booktitle={Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS,},
year={2017},
pages={733-738},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006355407330738},
isbn={978-989-758-248-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 19th International Conference on Enterprise Information Systems - Volume 2: ICEIS,
TI - A Personal Analytics Platform for the Internet of Things - Implementing Kappa Architecture with Microservice-based Stream Processing
SN - 978-989-758-248-6
AU - Zschörnig T.
AU - Wehlitz R.
AU - Franczyk B.
PY - 2017
SP - 733
EP - 738
DO - 10.5220/0006355407330738