Authors:
Roberto Milanese
1
;
2
;
3
;
Michele Guerra
1
;
3
;
Michele Daniele
3
;
1
;
Giovanni Fabbrocino
3
and
Fausto Fasano
1
;
3
Affiliations:
1
Mosaic Research Center, University of Molise, Italy
;
2
Department of Control and Computer Engineering, Politecnico di Torino, Italy
;
3
Department of Biosciences and Territory, University of Molise, Italy
Keyword(s):
LLM, MLLM, AI, UI, Security and Privacy, Application Security, App Permission, Android, Android Permission Model.
Abstract:
With the widespread use of mobile apps, users are frequently required to make decisions about app permissions. However, most people lack the knowledge to fully understand the consequences of their choices. Apps often request access to sensitive data, sometimes in the background and without clear justification, making users the weakest link in the security chain. This inadvertently exposes them to privacy breaches and malicious activities. Despite improvements, Android’s permission system remains inadequate in helping users make informed, real-time decisions. In this paper, we investigate the feasibility of an approach to address this critical gap that leverages the power of Large Language Models (LLMs) and Multi-Modal Large Language Models (MLLMs). We propose a system that dynamically evaluates permission requests by analyzing the full context of the UI on mobile app screens. Unlike traditional permission models, which rely on static rules or user input, our approach integrates seaml
essly into existing systems, interpreting the relationships between UI elements and requested permissions to make aware, real-time decisions about whether the request is necessary or potentially harmful. Our evaluation on 123,552 UI screens from 70 popular Android apps revealed promising results, reaching 81% accuracy. By reducing the cognitive load on users and offering real-time protection against security threats or supporting a more informed choice by the user, our system can enhance existing permission models, providing a step towards smarter and safer mobile ecosystems. This solution paves the way for integrating intelligent permission systems that proactively shield users from risks while ensuring data security without overwhelming them with complex decisions.
(More)