Automatic Generation of UIs for Disabled Users using Context-aware

Techniques and Reasoning

Lamia Zouhaier, Yousra Hlaoui Ben Daly and Leila Jemni Ben Ayed

Laboratory LaTICE, Higher School of Sciences and Technologies of Tunis, University of Tunis, Tunis, Tunisia

Keywords: User Interface, Adaptation, Context-awareness, Model-based, Context Modelling, Ontology, Impaired User,

Task Model.

Abstract: Today, users need to interact with the UIs of the computer systems at any time and in any place. In fact,

users have to deal with diverse devices supporting diverse interfaces and used in diverse environments.

Thus, research must to be devoted to adapting the content, presentation and also the navigation scheme of

the user interface not only for people without disabilities but also to impaired users according to updated

context of use. In fact, context is captured from the surrounding environment in which the user is interacting

with the application, gathered from a variety of sources and changed dynamically over time. For that reason,

we propose to include context awareness system as a solution for adaptation of user interface tailored to user

with special needs. In this paper, we try to introduce our contribution in how to build a user interface which

is aware and capable to adapt depending of context. Our contribution is demonstrated through the behaviour

of task model.

1 INTRODUCTION

Due to the diversity in human society and the

increasing numbers of user’s profile (handicapped

users, normal users, young/elderly people, etc), the

interface of computational systems faces to be used

by different users at same time but also in different

environments. UI must to be flexible and

autonomous in order to match the big numbers of

users’ needs (Lopez, 2003).

With the diversity of computing devices (mobile

phones, PDAs, PC, interactive kiosks, etc.)

increasingly present to support the daily activities of

any individual, engineering of interactive systems is

facing the challenge of not only produce quality

systems but also systems which can be customized

or adapted to any type of device with respect to the

personal characteristics of user specially whose are

with special needs.

These problems have motivated, recently,

researches to define features for interface‘s

adaptation according to the context change. The

application must to be able to both detect the current

state of context and the new context in the ambient

environment and to determine what actions to take

based on this contextual information.

In pervasive computing environments, context

consists of any information that can be captured

from the surrounding environment in which the user

is interacting with the application, gathered from a

variety of sources. Context-aware systems offer

entirely new opportunities for application developers

the possibility to gather context data and adapting

systems behavior accordingly (Baldauf, 2007).

Adaptation of user interface in mobile contexts is a

topic that has recently stimulated various research

contributions (Manco, 2013).

Existing work in the area of context awareness

focused on all aspects of capture, interpretation,

modeling, storage and dissemination of context but

there are no generic and global solutions that include

all steps of adaptation from context acquisition to

generated final interface. Context aware application

is a new computing paradigm. It gives the possibility

to explore the dynamic context of use and to take

advantage of contextual information in order to

adapt to user needs. Many context-aware

applications are built to demonstrate the utility of

this new technology. The majority of existing

frameworks of context sensitive use a layered

architecture supporting the important aspects of

sensor capture, context extraction and reasoning.

Building context-aware applications is one of the

413

Zouhaier L., Hlaoui Ben Daly Y. and Jemni Ben Ayed L..

Automatic Generation of UIs for Disabled Users using Context-aware Techniques and Reasoning.

DOI: 10.5220/0004635504130418

In Proceedings of the International Conference on Knowledge Engineering and Ontology Development (KEOD-2013), pages 413-418

ISBN: 978-989-8565-81-5

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

solutions adopted but also still a complicated task

due to the lack of an adequate and generic

infrastructure that support in pervasive computing

environments (Lopez, 2003).

The goal of this paper is to introduce an

approach for the adaptation of UI to the context

(user preferences, environment, terminal, etc). The

populations that we target are people with

disabilities.

We try to give a plan of our contribution

in how to build a UI which is aware and capable to

adapt depending of context.

In section two, we list the related work in user

interface adaptation. In section three, we introduce

our approach. In section four, we discuss our context

acquisition and management strategy. In section

five, we show how the adaptation affects the

behaviour of task model illustrating by Petri-nets

diagram when the context varies.

2 RELATED WORK

This section is dedicated to existing works in

accessible user interface adaptation. Little work has

focused on identifying generic solutions able to

adapt any user interface of applications to various

combinations of context of use including disable

people using context awareness mechanism.

In the early period, accessibility problems were

primarily considered to concern only the field of

Assistive Technology (AT), and consequently,

accessibility entailed meeting prescribed

requirements for the use of a product by people with

disabilities (Stephandis and Savidis, 20001).

For some developments, it is necessary to apply

also more specific guidelines such as a set of

guidelines for specific application type (e-learning,

tele-working), access device (mobile devices) or

user type (elderly, children, blind, deaf) (Leonidis,

2011). Web developers and designers can leverage

standards such as WCAG (Henry, 2012) to ensure

the overall accessibility of a given Web application.

This type of guidelines devises a set of conformance

levels based on how loose or strict is a Web page’s

support on accessibility issues, independently from

any particular disability (Calvary, 2002).

There are a wide variety of applications to which

personalization can be applied and a wide variety of

devices available on which to deliver the

personalized information. Most personalization

systems are based on some type of user profile, a

data instance of a user model that is applied to

adaptive interactive systems.

Leonidis et al.

(Leonidis, 2011) propose a toolkit for rapid

prototyping in order to ease the design of adaptive

widget-based interfaces.

Adaptive and adaptable interactions techniques

are increasingly emerged in recent research. There

are no generic solutions oriented towards

accessibility of user interface, but different

terminologies are employed as Universal Access

(Stephanidis, 1998), User Interfaces for All

(Akoumianakis, 1999), Design For All (Lopez,

2003) Unified User Interfaces (Minon, 2011)

because of the range of the population which may

gradually be confronted with accessibility problems

extends beyond the population of disabled and

elderly users to include all people (Stephandis &

Savidis, 2001). Universal Access refers to the global

requirement of coping with diversity in: (i) the

characteristics of the target user population

(including people with disabilities); (ii) the scope

and nature of tasks; and (iii) the different contexts of

use and the effects of their proliferation into

business and social endeavors (Stephandis and

Savidis, 20001).

Universal accessibility system should be

accessible for all users, although the design is

focused on people with special needs. But none of

these projects resulted in any concrete solutions for

users with special needs. The scope of User

Interfaces for All, as a perspective on HCI, is

necessarily broad and complex, involving

challenges, which pertain to issues such as context

oriented design, diverse user requirements and

adaptable and adaptive interactive behaviors. This

diversity of needs is generally ignored at the present

time. Occasionally, it is addressed in one of several

ways: manual redesign of the interface, limited

customization support, or by supplying an external

assistive technology.

AVANTI (Stephanidis, 1998) is the first project

to employ adaptive techniques in order to ensure

accessibility and high quality of interaction for all

potential users. It put forward a conceptual

framework for the construction of systems that

support adaptability and adaptivity at both the

content and the user interface levels (Stephandis &

Savidis, 20001). The distinctive characteristic of the

AVANTI browser is its ability to dynamically tailor

itself to the abilities, skills, requirements, and

preferences of the end-users, to the different

contexts of use, and to the changing characteristics

of users as they interact with the system.

EGOKI (Abascal, 2011) is a system that

generates accessible mobile user interfaces adapted

for people with disabilities in order to grant them

access to ubiquitous services. These interfaces are

KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment

414

intended to provide access to ubiquitous services in

intelligent environments. EGOKI dynamically

creates an instance of the interface running on the

user device. To adapt the interface to the user

characteristics, it is necessary to take into account

what the most suitable communication modalities

are for each user, mapping them to the appropriate

media.

Plasticity (Calvary, 2001) is a recent and

emerged technique of adaptation which is the

capacity of an interactive system to withstand to

context variations while preserving usability. In

order to support the end-user preferences,

adaptations rules can be changed according to user’s

order (Thevenin, 2001); (Calvary, 2002). It results

from a SituationReaction process where the

situation denotes a context change that needs a

reaction, and reaction denotes the procedures that

the system and/or the user executes to preserve

usability.

In literature, few works deal with adaptation of

the content, presentation or the navigation scheme of

the user interface to users with special needs

(Minon, 2011). Ubiquitous services are usually

provided by means of generic interfaces that may

contain barriers for people with disabilities (Gajos,

2011). To overcome this problem, the use of

adaptable or adaptive user interfaces is

recommended (Ay, 2007).

3 OUR APPROACH FOR

ADAPTATION

Based on context awareness systems architecture, it

is easy to deal with context from capture to

management steps. That is why, we propose to use

some of existing infrastructures that support context-

awareness to adapt user interface accordingly to

disabled people profile, preferences and surrounding

environment.

Our contribution is to propose a novel approach

of user interface adaptation targeted to impaired

people. This approach must to be aware of user

context, plateform context and environment context.

Consequently, we need to follow these objectives:

- Provide a generic and scalable architecture for

adapting applications to new context of use.

- Provide an overall adaptation strategy (features,

data and presentation) of an application to new

contexts of use.

- Ensure scalability of available adaptive

mechanisms that we can apply to an application.

To ensure these described objectives, we propose to

include recent techniques employed by context-

aware systems for context acquisition and

management for, afterwards, enabling the

application of adaptation rules in order to generate

the final interface using model-based technique

following the approach Model Driven Architecture

MDA. Three principal steps that characterize our

approach and scheduled as depicted in figure 1:

- First: Context acquisition and Management

- Second: Application of adaptation rules

- Finally: Generation of final interface based on

model based development using the paradigm

Model Driven Architecture (MDA).

Figure 1: A global overview of our strategy.

Figure 2: Class diagram of our context model.

ContextAcquisition

andmanagement

Adaptation

Generationofuser

interface

AutomaticGenerationofUIsforDisabledUsersusingContext-awareTechniquesandReasoning

415

4 CONTEXT ACQUISITION AND

MANAGEMENT

Architectures presented in existing work to ensure

sensitivity context (context-aware architectures)

accord considerable importance to the management

of context without showing how to modify the

behavior of the application to fit the context.

Context modeling includes a variety of context

information types, their relationships, different

situations (e.g. abstractions of context information

facts, etc), histories of context information, and

uncertainty of context information (Bettini, 2010).

The different entities must have a common

structure for representing information. Each context

expression must contain at least Context type and

Context value:

- Context type: Each context must belong to a

category (Sound, time, temperature, etc.). These

types will be used in a context subscription or

query. Context type concepts form a tree

structure.

- Context value refers to the semantic or absolute

“value” of context type and is usually used

together with context type, forming a verbal

description. In some cases, context value might

contain an absolute numerical value or feature

describing context.

- Attributes specify the context expression and

might contain any additional details not included

in the other properties e.g Timestamp describing

the date/time when context was sensed, Source

containing how information was gathered, etc.

Existing approaches of context modeling differ in

the ease with which real world concepts can be

captured by software engineers, in the expressive

power of the context information model (see figure

2), in the support they can provide for reasoning

about context information, and in the computational

performance of the reasoning (Bettini, 2010).

Context management layer as depicted in figure 3 is

highly dependent on the two principal’s layers:

Context Provider (CP) and Context Interpreter (CI).

a. Context Provider: is responsible of collecting

context from sources and managing it.

b. Context Interpreter which translate the low level

context into high level representation.

The context Manager is the middleware between the

application and the context management layer. It

carries only pertinent data to application.

Context repository maintain only context which

is non-volatile, its value survive different execution

sessions as user profile, but dynamic context (time,

location, temperature, noise, etc) that rapidly

changes will be maintained in internal data

structures.

Figure 3: Context acquisition and management.

In the remainder of this paper, we demonstrate

the impact of context on the task model task.

5 TASK MODELLING WITH

CONTEXT CHANGE

We demonstrate our approach based on task model

behavior at the arrival of context event.

In (Nathalie, 2002), authors provide a formal

notation of task model in order to support the

variation of conditions depending on multiple

contexts of use. We will use the same formulation.

If we consider that a context is a triplet of three

components user model, platform model and

environment model, we can represent each

component as a set of variables.

Let‘s U, a user Model described by a set of finite

parameters, {u

, u

,..., u

}, u

represents a specific

profile of a given user. A concrete User context is

represented by identity, preference, activity,

location, disability, etc.

P= {p

, p

,..., p

} is a finite set of plateform, p

represents any property of the computing plateform

such as a screen resolution, screen size, processor

speed, location, operating system, network

bandwidth, etc.

E= {e

, e

,.., e

} is a finite set of environment

attribute, e

represents a specific configuration of

physical conditions (light or pressure), location,

social and organizational environment (stress level

or social interactions) in which a task is carried out.

So, a given Context C

is a triplet < u

i ,

>. A

context variation appears when at least one element

is modified. So, we consider C a matrix that contains

all the different contexts of use.

Context

Repository

ContextProvider

Context

Inter

reter

ContextManager

Generate

KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment

416

With the multiplicity of contexts, the highest level of

a user interface which is the task model must to

support the variation of conditions. It is needed in

order to create a boundary between two different set

of tasks; those dependent and those which do not

depend on context. According to (Nathalie, 2002),

we will consider in our work (see figure 1):

- Task model Independent of Context which is a

set of task that carried out in all different context

of use.

- Task model dependent of Context which is a set

of tasks that are valid in a subset of considered

context of use.

We have used a formal technique which is Petri Nets

(Elkoutbi, 2000). Petri nets are a basic model to

describe state changes in a system with transitions.

Based on this formalism, we consider that transitions

represent the tasks that will be carried out by the

user in interaction with the system, could be

dependent or independent of context, noted:

TASK= {Task

}

i=1...n

= {Task

dep

}



{Task

Indep

}

Places are the set of context attributes C= {C

}

j=1..m

Based on (Calvary, 2002) triggers, we will use

the following statements (1) and (2) in order to

demonstrate the behaviour infection of Task Model:

If ContextEntering (C

) Then perform (Task

)

(1)

If ContextLeaving (C

) Then perform (Task

Indep

)

(2)

The Task Model (Limbourg, 2002) behaviour can be

infected due to the change of context. With the Petri

net, we want to show how the execution of some

task depends on the new context that recently has

entered. In this case, the transition from task

task

i+2

(task

, task

i+1

and taski

are ordered using

temporal operators in Concur Task Tree (CTT)

(Limbourg, 2002)) is verified if and only if the two

Task

dep

and Task

Indep

are successful executed and the

condition “leaving context” is verified.

Figure 4: Petri Nets of Task Model behaviour when

context change.

6 CONCLUSIONS AND FUTURE

WORK

This work will allow us to frame the issues for user

interface using context-awareness and open research

perspectives related to the adaptation of interfaces.

Using the history of the context to predict the task of

the user interface and adapt accordingly is a field to

explore. In fact, using this new paradigm of context-

sensitive, UI can be aware of any change in the

context of use. Thus, the user interface must to cover

a huge number of context configuration and

satisfying the user requirement in the adaptation

process.

Ontology provide a uniform way for specifying

the model’s concepts, subconcepts, relations,

properties and facts, altogether providing the means

for the sharing of contextual knowledge and

information reuse. The contextual knowledge is

interpreted and evaluated by use of ontology

reasoning (Rios, 2004). A reasonner can use

ontology to deduce conclusion about context in

order to make decision on content adaptation

Further work, we propose an approach that

interprets the context based on the type of user

(visual handicapped, deaf user, etc.), his location and

on physical environment’s properties like the degree

of luminosity (Low, Medium, High) and the noise

(Quiet or Noisy).

REFERENCES

Abascal, J., Aizpurua, A., Cearreta, I., Gamecho, B.,

Garay, N., & Miñón, R., 2011. A modular approach to

user interface adaptation for people with disabilities in

ubiquitous environments. Internal Technical Report

No. EHU-KAT-IK-01-11.

Akoumianakis, D., Savidis, A., and Stephanidis, C., 2000.

“Encapsulating Intelligent Interactive Behaviour in

Unified User Interface Artefacts”, Interacting With

Computers, Vol. 12, No. 4, pp. 383-408.

Akoumianakis, D., Savidis, A., and Stephanidis, C., 2000.

“Encapsulating Intelligent Interactive Behaviour in

Unified User Interface Artefacts”, Interacting With

Computers, Vol. 12, No. 4, pp. 383-408.

Ay, F., 2007. “Context Modeling and Reasoning using

Ontologies”, University of Technology Berlin.

Baldauf, M, 2007. “A survey on context-aware systems”,

Int. Journal Ad Hoc and Ubiquitous Computing, Vol.

2, No. 4, pp.263–277.

Bettini, C., Brdiczka, O., Indulska, J., Henricksen, K.,

Nicklas, D., Ranganathan, A., Riboni, D., 2010. “A

Survey of Context Modelling and Reasoning

Techniques”, In Journal Pervasive and Mobile

AutomaticGenerationofUIsforDisabledUsersusingContext-awareTechniquesandReasoning

417

Computing (2), Volume 6, pp 161-180.

Calvary, G, Coutaz, J, Thevenin, D, Limbourg, Q,

Souchon, N, Bouillon, L., Florins, M., and

Vanderdonckt, J., 2002. “Plasticity of User Interfaces:

A Revised Reference Framework”, Proceedings of the

first international workshop on Task Models and

Diagrams for User Interface Design, pp 127–134.

Calvary, G., Coutaz, J. and Thevenin, D., 2001.

“Supporting Context Changes For Plastic User

Interfaces: A Process And Mechanism”, Proceedings

of AFIHM-BCS Conference on Human-Computer

Interaction IHM-HCI.

Chen, H., 2003. “An Intelligent Broker Architecture for

Context-Aware Systems”, Phd Thesis, University of

Maryland Baltimore County.

Chen, H., Perich, F., W. Finin, T., Joshi, A., 2004.

“SOUPA: Standard Ontology for Ubiquitous and

Pervasive Applications”, in: 1

Annual International

Conference on Mobile and Ubiquitous Systems.

Elkoutbi, M., Keller, R. K., 2000. “User Interface

Prototyping Based on UML Scenarios and High-Level

Petri Nets”, In Application and Theory of Petri Nets

2000 (Proc. of 21 st Intl. Conf. on ATPN), Springer-

Verlag LNCS 1825, pp 166-186.

Gajos, K.Z., Weld, D.S., Wobbrock, J.O., 2011.

“Automatically generating personalized user interfaces

with Supple”, Artificial Intelligence Journal 174(12-

13), pp: 910–950.

Grigoris Antoniou, G., Frank, V.H., 2009. “Web Ontology

Language: OWL”, International Handbooks on

Information Systems, pp 91-110, 2009.

Henry,S.H, 2012. ‘Web Content Accessibility Guidelines

(WCAG) Overview’, http://www.w3.org/WAI.

Leonidis, A., Antona, M., Stephanidis, C., 2011. “Rapid

Prototyping of Adaptable User Interfaces”,

International Journal of Human-Computer

Interaction.

Limbourg, Q., Souchon, N., and Vanderdonckt, J., 2002.

“Task Modelling in Multiple Contexts of Use”,

Interactive Systems:Design, Specification, and

Verification, Volume 2545, pp 59-73.

Lopez, J.V., Montero, F., Fermandez, C. A., Lozano, M.,

2003. “Towards Adaptive User Interface Generation:

One Step Closer to People”, In Proceedings of ICEIS

(3), pp.97-103.

Manca, M., Paternò, F., Santoro, C., 2013. “Adaptive

Mobile Multimodal User Interfaces for Dynamic

Accessibility”, Workshop on Mobile Accessibility,

CHI.

Miñón, R., Abascal, J., Aizpurua, A., Cearreta, I.,

Gamecho, B., Garay, N., 2011. “Model-Based

Accessible User Interface Generation in Ubiquitous

Environments”, INTERACT (4), pp: 572-575.

Nathalie, S., Quentin, L., Vanderdonckt , J, 2002. “Task

Modeling In Multiple Contexts Of Use”, In

Proceedings of Design Specification and Verification

of Interactive Systems, Springer, pp 59-73.

Rios, D., Dockhorn, P., Guizzardi, G., Ferreira, L.,

Pereira, J., and Sinderen, M., 2004. “Using Ontologies

for Modeling Context-Aware Services Platforms”, In

Workshop on Ontologies to Complement Software

Architectures.

Stephanidis, C., and Savidis, A., 2001, “Universal Access

in the Information Society: Methods, Tools, and

InteractionTechnologies”, UAIS, 1: 40–55.

Stephanidis, C., Paramythis, A., Sfyrakis, M., Stergiou,

A., Maou, N., Leventis, A., Paparoulis, G.,

Karagiannidis, C., 1998. “Adaptable and Adaptive

User Interfaces for Disabled Users in the Avanti

Project”, 5th International Conference on Intelligence

in Services and Networks: Technology for Ubiquitous

Telecom Services, pp 153-166.

Strang, T., Linnhoff-Popien, C., 2004. “A Context

Modeling Survey”, Workshop on Advanced Context

Modelling, Reasoning and Management, The 6th

International Conference on Ubiquitous Computing.

Thevenin, D., 2001. “Adaptation En Interaction Homme-

Machine : Le Cas De La Plasticité”, Phd Thesis,

University of Joseph Fourier, Grenoble.

Thomas, S., Claudia, L.P., Korbinian, F., 2003. “CoOL: A

Context Ontology Language to Enable Contextual

Interoperability”, Proceedings of 4th IFIP WG 6.1

International Conference on Distributed Applications

and Interoperable Systems (DAIS2003), Volume 2893

of Lecture Notes in Computer Science (LNCS).

Wang, X. H., Gu, T., Zhang, D. Q., and Pung, H. K., 2007.

“Ontology Based Context Modeling and Reasoning

using OWL”.

Zhang, D., Gu, T., Wang, X., 2005. “Enabling Context-

aware Smart Home with Semantic Technology”,

International Journal of Human-friendly Welfare

Robotic Systems 6 (4), pp. 12–20.

KEOD2013-InternationalConferenceonKnowledgeEngineeringandOntologyDevelopment

418