GROWING HIERARCHICAL SELF-ORGANISING MAPS FOR ONLINE ANOMALY DETECTION BY USING NETWORK LOGS

Mikhail Zolotukhin, Timo Hämäläinen, Antti Juvonen

Abstract

In modern networks HTTP clients request and send information using queries. Such queris are easy to manipulate to include malicious attacks which can allow attackers to corrupt a server or collect confidential information. In this study, the approach based on self-organizing maps is considered to detect such attacks. Feature matrices are obtained by applying n-gram model to extract features from HTTP requests contained in network logs. By learning on basis of these matrices, growing hierarchical self-organizing maps are constructed and by using these maps new requests received by the web-server are classified. The technique proposed allows to detect online HTTP attacks in the case of continuous updated web-applications. The algorithm proposed was tested using Logs, which were aquire acquired from a large real-life web-service and include normal and intrusive requests. As a result, almost all attacks from these logs have been detected, and the number of false alarms was very low at the same time.

References

  1. Apache 2.0 Documentation http://www.apache.org/.
  2. Axelsson, S. (1998). Automatically hardening web applications using precise tainting. Technical report, Department of Computer Engineering, Chalmers University of Technology, Goteborg, Sweden.
  3. Chan, A. and Pampalk, E. (2002). Growing hierarchical self organising map (ghsom) toolbox: visualisations and enhancements. In 9-th International Conference Neural Information Processing, ICONIP 7802, volume 5, pages 2537-2541.
  4. Gollmann, D. (2006). Computer Security. Wiley, 2nd edition.
  5. Hirsimaki, T., Pylkkonen, J., and Kurimo, M. (2009). Importance of high-order n-gram models in morph-based speech recognition. Audio, Speech, and Language Processing, IEEE Transactions, 17:724-732.
  6. Ippoliti, D. and Xiaobo, Z. (2010). An adaptive growing hierarchical self organizing map for network intrusion detection. In 19th IEEE International Conference Computer Communications and Networks (ICCCN), pages 1-7.
  7. Jiang, D., Yang, Y., and Xia, M. (2009). Research on intrusion detection based on an improved som neural network. In Fifth International Conference on Information Assurance and Security, pages 400-403.
  8. Johnson, R. W. (1994). Estimating the size of a population. Teaching Statistics, 16:50-52.
  9. Kayacik, H. G., Nur, Z.-H., and Heywood, M. I. (2007). A hierarchical som-based intrusion detection system. Engineering Applications of Artificial Intelligence, 20:439-451.
  10. Kemmerer, R. and Vigna, G. (2002). Intrusion detection: A brief history and overview. Computer, 35:27-30.
  11. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological cybernetics, 43:59-69.
  12. Kohonen, T. (2001). Self-organizing map. Springer-Verlag, Berlin, 2rd edition.
  13. Mukkamala, S. and Sung, A. (2003). A comparative study of techniques for intrusion detection. In Tools with Artificial Intelligence, 15th IEEE International Conference.
  14. Nguyen-Tuong, A., Guarnieri, S., Greene, D., Shirley, J., and Evans, D. (2005). Automatically hardening web applications using precise tainting. In 20th IFIP International Information Security Conference.
  15. Palomo, E. J., Domínguez, E., Luque, R. M., and Mun˜oz, J. (2008). A New GHSOM Model Applied to Network Security, volume 5163 of Lecture Notes in Computer Science. Springer, Berlin, Germany.
  16. Patcha, A. and Park, J. (2007). An overview of anomaly detection techniques: Existing solutions and latest technological trends. Computer Networks: The International Journal of Computer and Telecommunications Networking, 51.
  17. Rauber, A., Merkl, D., and Dittenbach, M. (2002). The growing hierarchical self-organizing map: exploratory analysis of high-dimensional data. Neural Networks, IEEE Transactions, 13:1331-1341.
  18. Shehab, M., Mansour, N., and Faour, A. (2008). Growing hierarchical self-organizing map for filtering intrusion detection alarms. In International Symposium Parallel Architectures, Algorithms, and Networks, I-SPAN, pages 167-172.
  19. Suen, C. Y. (1979). n-gram statistics for natural language understanding and text processing. Pattern Analysis and Machine Intelligence, IEEE Transactions, PAMI1:162-172.
  20. Ultsch, A. (2003a). Maps for the visualization of highdimensional data spaces. In Workshop on SelfOrganizing Maps (WSOM 2003),, pages 225-230.
  21. Ultsch, A. (2003b). Pareto density estimation: A density estimation for knowledge discovery. In Innovations in Classification, Data Science, and Information Systems - Proc. 27th Annual Conference of the German Classification Siciety, pages 91-100.
  22. Ultsch, A. and Siemon, H. P. (1990). Kohonen's self organizing feature maps for exploratory data analysis. In Proc. Intern. Neural Networks, Kluwer Academic Press, pages 305-308.
  23. Ultschk, A. (2005). Clustering with som: U*c. In U*C. Proc. Workshop on Self-Organizing Maps (WSOM 2005), pages 75-82.
  24. Verwoerd, T. and Hunt, R. (2002). Intrusion detection techniques and approaches. Computer Communications, 25:1356-1365.
Download


Paper Citation


in Harvard Style

Zolotukhin M., Hämäläinen T. and Juvonen A. (2012). GROWING HIERARCHICAL SELF-ORGANISING MAPS FOR ONLINE ANOMALY DETECTION BY USING NETWORK LOGS . In Proceedings of the 8th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-989-8565-08-2, pages 633-642. DOI: 10.5220/0003936606330642


in Bibtex Style

@conference{webist12,
author={Mikhail Zolotukhin and Timo Hämäläinen and Antti Juvonen},
title={GROWING HIERARCHICAL SELF-ORGANISING MAPS FOR ONLINE ANOMALY DETECTION BY USING NETWORK LOGS},
booktitle={Proceedings of the 8th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2012},
pages={633-642},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003936606330642},
isbn={978-989-8565-08-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - GROWING HIERARCHICAL SELF-ORGANISING MAPS FOR ONLINE ANOMALY DETECTION BY USING NETWORK LOGS
SN - 978-989-8565-08-2
AU - Zolotukhin M.
AU - Hämäläinen T.
AU - Juvonen A.
PY - 2012
SP - 633
EP - 642
DO - 10.5220/0003936606330642