RAPIDLY MODIFYING MULTIPLE USER INTERFACES OF ONE

APPLICATION

Leveraging Multi-level Dialogue Reﬁnement

Alexander Behring, Andreas Petter and Max M

uhlh

auser

FG Telecooperation, TU Darmstadt, 64283 Darmstadt, Germany

Keywords:

UI models, Reﬁnement, Engineering user interfaces, User interface description languages.

Abstract:

An increasing demand for supporting users in diverse contexts of use, e.g., depending on interaction device

and primary task, results in new challenges for User Interface (UI) development. Two key challenges are: how

to create these multiple UIs for one application (creation challenge), and how to consistently modify them

(modiﬁcation challenge). The creation challenge has been addressed in various works utilizing automatic UI

generation. We present our approach (Dialogue Reﬁnement) and its tool-support to address the modiﬁcation

challenge by allowing one modiﬁcation to be applied to multiple UIs at once.

1 INTRODUCTION

Multiple causes lead to an increase in the number of

User Interfaces (UIs) per application. Technological

advances allow us to interact with computers in situa-

tions that could not be supported by information tech-

nology before. We can observe that more and more

commercial services are already offered not only for

the traditional desktop computer, but for mobile de-

vices like the iPhone, Java-enabled cell phones or

alike. Examples beyond simple train, airplane or bus

schedule services are electronic boarding-passes, and

the possibility to buy electronic public transportation

tickets for and with mobile devices. Regarding these

UIs, key challenges are how to create UIs for differ-

ent contexts of use (creation challenge) and how to

apply modiﬁcations to them, once the UIs are created

(modiﬁcation challenge).

Our approach to address the modiﬁcation chal-

lenge is based on the key idea to allow one modiﬁ-

cation to change multiple UIs. In order to control the

propagation of a modiﬁcation, UIs are ordered in a

tree. The modiﬁcation is ”passed” down the tree along

associations between the UIs. Changing these associ-

ations allows the UI engineer to adapt the propagation

of modiﬁcations to suite the problem at hand.

Our main contributions in this paper are the reﬁne-

ment concept (sect. 2) and its editor-support (sect. 3).

A case study and preliminary user study are presented

subsequently (sect. 4). Finally, we discuss related ap-

proaches (sect. 5).

2 DIALOGUE REFINEMENT

After introducing objectives for addressing the mod-

iﬁcation challenge, our approach is introduced. it is

supported by a metamodel and a toolset, which are

presented afterwards.

2.1 Objectives

Objective 1: The UI engineer must be kept in the

loop.

Automatic approaches hold great potential to re-

duce the effort to provide multiple UIs for one ap-

plication. But their interfaces are prone to usabil-

ity problems and lack aesthetic quality (Myers et al.,

2000; Meskens et al., 2008) when compared to man-

ually designed UIs.

Objective 2: The artifact edited by the UI engineer

must resemble as much as possible the resulting UI.

Working on abstract artifacts isolates the UI en-

gineer from the concrete interface. This was a reason

why User Interface Management Systems (UIMS) did

not catch on, as Myers et al. discussed in (Myers

et al., 2000). They discovered that expressing graph-

ical concepts by graphical means was an important

success factor for interface builders. It lowered the

threshold of use and the modiﬁcations were not prone

to unpredictability. The term WYSIWYG – what you

344

Behring A., Petter A. and Mühlhäuser M. (2009).

RAPIDLY MODIFYING MULTIPLE USER INTERFACES OF ONE APPLICATION - Leveraging Multi-level Dialogue Reﬁnement.

In Proceedings of the 4th International Conference on Software and Data Technologies, pages 344-347

DOI: 10.5220/0002260103440347

 SciTePress

Root

Desktop

Regular Touch-Display

Mobile

Subnotebook Small-Screen

Nokia E70 iPhone Penbased

Concrete

Abstract

Figure 1: An exemplary Reﬁnement Tree of UIs reﬁned for different contexts of use. The topmost UI is called Root UI.

see is what you get – makes this even more clear:

there is no difference between the edited artifact and

the result.

Objective 3: The UI engineer must be able to

provide her information at the level of abstraction

suitable for the problem at hand.

UI changes can be situated at different levels of

abstraction, e.g., just for the concrete iPhone UI or

more abstract for all small-screen UIs. Depending

on domain and application, the number of abstraction

levels suitable may vary.

2.2 Concept

To create a UI for a new context of use, the UI engi-

neer reﬁnes an already existing one and adds, removes

or modiﬁes elements of it. Applying this step multiple

times results in a reﬁnement tree, as shown in ﬁgure 1.

The tree structure is captured through reﬁnement as-

sociations, created during the reﬁnement. These as-

sociations connect the reﬁned UI with the more ab-

stract

UI.

Our key concept to address the modiﬁcation chal-

lenge is to allow one modiﬁcation to be applied to

multiple UIs. When applying a modiﬁcation, the re-

ﬁnement associations are used to propagate the mod-

iﬁcation through the reﬁnement tree, thus applying it

to multiple UIs at the same time. Through changing

the reﬁnement associations, the UI engineer can in-

ﬂuence, to which UIs a modiﬁcation is propagated.

Consequently, a value is either reﬁned (i.e. the value

is inherited from the more abstract UI), or it is set lo-

cally.

We implemented the concept as a metamodel in

We use the word abstract in the context of reﬁnement

rather than generalization.

EMF (Eclipse Modeling Framework

). In this meta-

model, UIs are made up of InteractionObjects. The

UI engineer arranges them in UIBoxes to determine

the look and feel of a UI. A UIBox contains the UI

for a speciﬁc context of use and is a node in the re-

ﬁnement tree. Reﬁnement associations can connect

UIBoxes or InteractionObjects. The metamodel is ag-

nostic of the concrete UI toolkit used, they are inte-

grated through libraries.

3 WYSIWYG TOOL SUPPORT

Renderers (model interpreters), in our approach, work

directly on the UI model without intermediate code

artifacts. Thus, there are no synchronization prob-

lems, which allows for an easy extension of render-

ers with WYSIWYG editing capability (cf. ﬁgure 2).

Furthermore, editors must allow the UI engineer to

modify and identify the current reﬁnement state of a

value (reﬁned or set locally). The UI engineer must

be able to explore the reﬁnement associations for an

element: does it have reﬁning versions or an abstract

version?

We implemented renderers for HTML and Swing

in Java integrated into our UI research platform Ma-

pache. The Swing renderer was extended with edit-

ing capabilities and connected to Eclipse, as shown in

ﬁgure 2. The Eclipse property view allows to identify

and modify the reﬁnement state of element proper-

ties. Also in Eclipse, a Reﬁnement View showing the

reﬁnement tree (cf. ﬁgure 1) was implemented. Be-

sides showing the UI hierarchy, it can be used to ex-

plore which related elements (regarding reﬁnement)

exist for the currently selected element.

http://www.eclipse.org/emf

RAPIDLY MODIFYING MULTIPLE USER INTERFACES OF ONE APPLICATION - Leveraging Multi-level Dialogue

Refinement

345

Figure 2: The interactive model interpreter in edit-mode with its eclipse properties page. The property page is synchronized

with the interpreter and shows the currently selected item (Button ”Find an exhibit”). The value of the text-property is reﬁned

from a super-version, as depicted in the properties view.

4 CASE STUDY

We created a larger case study application. The use

case was a university guide (UniGuide), allowing

people to get information on objects in our computer

science building. It was targeted at three different

contexts of use: desktop, subnotebook and smart-

phone. Figure 3 shows the ”exhibit details” window

reﬁned for smartphones and desktop.

Figure 3: UIs of our case study – the ”exhibit details” win-

dow reﬁned for smartphone and desktop.

We used our editors presented in section 3 to build

the UIs for the case study. Attaching behavior can be

done at any reﬁnement level using an event-registry.

Domain elements are shared among different UI ver-

sions. Their properties can automatically be synced

with UI element properties (“Value Bindings”).

After implementing the case study, we conducted

a preliminary user study. Seven participants with a

great variance in GUI building skills had to complete

two UI modiﬁcation tasks. Task one consisted of re-

placing a text in all UIs, whereas task two was to

add tooltips to existing elements. After a short intro-

duction, we asked the participants to complete both

tasks with and without our tools. As a baseline, we

used Netbeans, because Eclipse does not provide an

industrial-grade interface builder. For both tasks to-

gether, 9 steps were necessary using our tools, 21

steps in Netbeans.

All participants, even those with no experience in

GUI creation, successfully completed both tasks us-

ing our approach and had no delays in applying the

concept with regard to the use of Netbeans. We can

conclude that the concept is easily comprehensible

and thus has a low threshold of use. The normal-

ized error rate (total errors per total number of steps)

hinted that participants were less likely to produce er-

rors when using our approach (4% ± 6%) versus Net-

beans (21%±20%). We attribute this to the repetitive

nature of the task in Netbeans. Such errors could pose

a great problem in larger projects with a bigger set of

supported contexts of use.

5 RELATED APPROACHES

Several works apply model-to-model transformations

to create or synchronize UI models. For example,

Sottet et al. (Sottet et al., 2006) use ATL, and

Limbourg applies a generic graph-transformation lan-

guage (Limbourg, 2004). Approaches that provide

generic solutions, as transformations do, can be ap-

plied to a great range of problem domains. But since

they are not focused on the UI, their syntax, and more

important their semantic, does not address UI speciﬁc

issues. Consequently, they are very abstract for the

UI engineer to use. In contrast to our approach, such

generic transformation approaches thus suffer from

unpredictability. Furthermore, a new (often complex)

language has to be learned, raising the threshold of

use. In addition, they are prone to usability problems

ICSOFT 2009 - 4th International Conference on Software and Data Technologies

346

and lack aesthetic quality.

Seminal Teresa (Mori et al., 2004) supports an ab-

stract and a concrete UI level. The semantics of ab-

stract and concrete levels are ﬁxed. In contrast, us-

ing our approach, the number of abstraction level can

be chosen with respect to the problem at hand. Also,

the UI engineer can choose the subject of abstraction

freely and is not tied to ﬁxed abstraction semantics.

Damask (Lin and Landay, 2008) is a tool mainly

targeted towards UI prototyping. The modiﬁcation of

existing UIs, or even UIs that are already used in run-

ning applications is not Damask’s focus. However,

the layer concept presented in Damask is similar to

the Reﬁnement Tree of our approach. But our work

goes beyond the two layers supported in Damask in

order to support a wide range of different contexts of

use.

Gummy (Meskens et al., 2008) is tool for creating

different UIs for different contexts of use in WYSI-

WYG fashion. UIs for new contexts of use can be

transformed from existing ones, but after creation the

connection to the source UI is lost: modiﬁcations can

only be applied to a single UI. In contrast, our ap-

proach keeps these connections and thus allows to ap-

ply one modiﬁcation to multiple UIs.

6 CONCLUSIONS

We presented the Dialogue Reﬁnement approach and

its tool-support to address the modiﬁcation challenge.

The approach allows the UI engineer to apply one

modiﬁcation to multiple UIs at once. A case study

and preliminary user study showed its applicability,

ease of use, and a lower error rate compared to an

industrial-grade GUI builder.

We currently research the integration of transfor-

mation approaches and the concepts presented in this

paper. Because hand-crafting user interfaces for mul-

tiple target platforms is a costly task, transformations

can be used for automatic generation of UIs. The aes-

thetic quality and usability of the UIs can be obtained

by easily allowing manual modiﬁcations to generated

UIs. Furthermore, we investigate the integration of a

voice-based toolkit (VoiceXML) into reﬁnement.

ACKNOWLEDGEMENTS

This work has been supported by the BMBF SoKNOS

project. We thank our colleagues for their valuable

feedback and Jannik Jochem for his support in imple-

menting the tools.

REFERENCES

Limbourg, Q. (2004). Multi-Path Development of User In-

terfaces. PhD thesis, Universit catholique de Louvain.

Lin, J. and Landay, J. A. (2008). Employing patterns and

layers for early-stage design and prototyping of cross-

device user interfaces. In CHI ’08, pages 1313–1322,

New York, NY, USA. ACM.

Meskens, J., Vermeulen, J., Luyten, K., and Coninx, K.

(2008). Gummy for multi-platform user interface de-

signs: shape me, multiply me, ﬁx me, use me. In AVI

’08, pages 233–240, New York, NY, USA. ACM.

Mori, G., Patern

o, F., and Santoro, C. (2004). Design and

development of multidevice user interfaces through

multiple logical descriptions. IEEE Trans. Softw.

Eng., 30(8):507–520.

Myers, B., Hudson, S. E., and Pausch, R. (2000). Past,

Present, and Future of User Interface Software Tools.

ACM Transactions on Computer-Human Interaction

(TOCHI), 7(1):3–28.

Sottet, J.-S., Calvary, G., Favre, J.-M., Coutaz, J., and De-

meure, A. (2006). Towards mapping and model trans-

formation for consistency of plastic user interfaces. In

Workshop “The Many Faces of Consistency in Cross-

platform Design”, CHI ’06. ACM.

RAPIDLY MODIFYING MULTIPLE USER INTERFACES OF ONE APPLICATION - Leveraging Multi-level Dialogue

Refinement

347