Authors:
Ilan Ben-Bassat
1
and
Erez Rokah
2
Affiliations:
1
School of Computer Science, Tel-Aviv University, Tel-Aviv and Israel
;
2
IBM, Herzliya and Israel
Keyword(s):
Security Testing, Automated Crawling, Rich Internet Applications, DOM Similarity, Locality-Sensitive Hashing, MinHash.
Related
Ontology
Subjects/Areas/Topics:
Computer-Supported Education
;
Enterprise Information Systems
;
Information Systems Analysis and Specification
;
Information Technologies Supporting Learning
;
Security
;
Security and Privacy
Abstract:
Web application security has become a major concern in recent years, as more and more content and services are available online. A useful method for identifying security vulnerabilities is black-box testing, which relies on an automated crawling of web applications. However, crawling Rich Internet Applications (RIAs) is a very challenging task. One of the key obstacles crawlers face is the state similarity problem: how to determine if two client-side states are equivalent. As current methods do not completely solve this problem, a successful scan of many real-world RIAs is still not possible. We present a novel approach to detect redundant content for security testing purposes. The algorithm applies locality-sensitive hashing using MinHash sketches in order to analyze the Document Object Model (DOM) structure of web pages, and to efficiently estimate similarity between them. Our experimental results show that this approach allows a successful scan of RIAs that cannot be crawled other
wise.
(More)