Assessing the Quality of User-interface Modeling Languages

Francisco Morais

and Alberto Rodrigues da Silva

1,2

Instituto Superior Técnico, Universidade de Lisboa, Lisbon, Portugal

INESC— ID, Lisbon, Portugal

Keywords:

Appropriateness, Domain-speciﬁc Languages, Evaluation, Quality, User-interface Modeling Languages.

Abstract:

Model-Driven Development (MDD) is an approach that considers model as ﬁrst citizen elements in the context

of software development. Since there are so many modeling languages, there is a need to compare them and

choose the best for each concrete situation. The selection of the most appropriate modeling language may

inﬂuence the output’s quality, whether it is only a set of models or software.

This paper introduces ARENA, a framework that allows to evaluate the quality and effectiveness of modeling

languages. Then we will apply ARENA to a speciﬁc subset of User-Interface Modeling Languages (namely

UMLi, UsiXML, XIS and XIS-Mobile), taking into account some of their characteristics and the inﬂuence

they have when models are generated.

1 INTRODUCTION

A modeling language might be classiﬁed as general-

purpose (GPML) or domain-speciﬁc modeling lan-

guage (DSML) (van Deursen et al., 2000; Luoma

et al., 2004; Mernik et al., 2005; Kosar et al., 2010).

A GPML is characterized by having a greater num-

ber of generic constructs, which encourages a wider

and widespread use in different ﬁelds of application.

UML or SysML are popular examples of GPMLs

by providing large sets of constructs and notations

used for specifying and documenting, respectively,

software systems according to the object-oriented

paradigm, or for system engineering. On the other

hand, DS(M)Ls tend to use few constructs or concepts

that are closer to its application domain.

Since a DS(M)L is expressed using domain con-

cepts, it is normally easier to read, understand, vali-

date and communicate with, facilitating cooperation

between developers and domain experts. Moreover,

some argue that DS(M)Ls can improve productiv-

ity, reliability, maintainability and portability (van

Deursen et al., 2000). However, the use of a DS(M)L

can raise some problems, such as the cost of learn-

ing, implementing and maintaining a new language,

as well as the support tools to develop with it (Mernik

et al., 2005).

Either for GPMLs as for DSMLs, nowadays there

is a great variety of modeling languages. For in-

stance, BPMN is speciﬁc to business process design

although UML’s activity diagram is also adequate for

this purpose (Object Management Group, 2013; Ob-

ject Management Group, 2011), DEMO is speciﬁc to

enterprise architecture (Dietz, 2001), XIS to Interac-

tive Applications, and Petri Nets to Distributed Sys-

tems (Desel and Juhás, 2001). Since there are more

languages and approaches than domains, this results

in overlapping effort for researchers and disorienta-

tion for modelers.

In order to help professionals selecting the most

suitable modeling language to work on so many spe-

ciﬁc and different contexts, evaluation frameworks

are needed. An evaluation framework is a set of prop-

erties, metrics, concepts and other parameters that

compare and assert the languages’ characteristics. For

this work, we have proposed ARENA Framework to

enlighten individuals that work with models on this

area. Our goal in this article is to help the user choos-

ing the most appropriate modeling language, in order

to assure a quality output, to feel more supported and

to work in a more rational way.

In this paper we propose a framework to evalu-

ate and compare the quality of User-Interface Model-

ing Languages (UIMLs), focused on either their gen-

eral and speciﬁc characteristics. The deﬁnition of a

modeling language involves multiple aspects or facets

which have to be taken into consideration, namely ab-

stract syntax, concrete syntax, and semantics.

UIMLs are DSMLs that are speciﬁcally used for

modeling the user interface of desktop, web or mo-

311

Morais F. and Rodrigues da Silva A..

Assessing the Quality of User-interface Modeling Languages.

DOI: 10.5220/0005369303110319

In Proceedings of the 17th International Conference on Enterprise Information Systems (ICEIS-2015), pages 311-319

ISBN: 978-989-758-097-0

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

bile applications, supporting the design and imple-

mentation phases (Achilleos et al., 2008). They de-

note conceptual abstractions that are present on user

interfaces (Frank, 1996). This paper analyses and

compares the main properties of four UIMLs, namely:

UMLi (Silva, 2002), UsiXML (Limbourg et al.,

2004), XIS (Silva et al., 2003) (Martins and Silva,

2007) (Silva et al., 2007) and XIS-Mobile (Ribeiro

and Silva, 2014b) (Ribeiro and Silva, 2014a). We

have selected them due to their support, complete

documentation, project visibility and availability of

their papers.

We are aware of other UIMLs such as Di-

aMODL (Trætteberg, 2008), IFML (Object Manage-

ment Group, 2014), MARIA XML (Paternó et al.,

2009) or WebML (Ceri et al., 2002). However, we

acknowledge that they don’t ﬁt the domain and our

criteria as well as the previous four. WebML had

strong inﬂuence in IFML, when Object Management

Group created this language and that IFML’s most re-

cent version (from 2014) is still in Beta. Therefore,

we consider that there were too much similarities be-

tween both and comparing them would not prove ef-

fective. Regarding DiaMODL and MARIA XML, the

ﬁrst one is too focused on Dialog Modeling, Dataﬂow

and State logic while the second is not quite appropri-

ate due to its models’ development being highly de-

pendable in Service-Oriented Architectures and Web

Services, which implies exploiting annotations at de-

sign time and the language itself at runtime to support

dynamic generation of user interfaces, thus limiting

usability and analysis.

This paper is organized in 6 sections: Section 1

gives a brief explanation and context of this work.

Section 2 introduces the background for this re-

search, that includes a deﬁnition of Quality and its

importance for this work, a brief overview of Mod-

eling Languages and an analysis of other Evalua-

tion Frameworks. Section 3 presents the proposed

ARENA Framework to the previously referred prob-

lem. Section 4 evaluates the quality of four UIMLs

with the ARENA’s properties and metrics. Section 5

discusses the evaluation made based on the ARENA

Framework. Finally, section 6 concludes this paper.

2 BACKGROUND

This section introduces the background of this re-

search, namely Quality, Modeling Languages and

Evaluation Frameworks.

2.1 Quality

Regarding a model that may generate software, its

quality can be divided into internal and external (In-

ternational Organization for Standardization and In-

ternational Electrotechnical Commission, 2001).

Internal quality consists on the characteristics of

the software product from an internal view. Inter-

nal quality requirements are used to specify properties

of interim products. They can include static and dy-

namic models, other documents and source code (In-

ternational Organization for Standardization and In-

ternational Electrotechnical Commission, 2001).

External quality is deﬁned by the characteristics of

the software product from an external view. It is the

quality of the executed software, which is typically

measured and evaluated while testing in a simulated

environment with simulated data using external met-

rics. These metrics should be aligned with the exter-

nal quality requirements, which goal is to specify the

required level of quality from the external view (In-

ternational Organization for Standardization and In-

ternational Electrotechnical Commission, 2001).

Quality is not yet a common priority when devel-

oping a Domain-Speciﬁc Language (DSL). The fo-

cus is currently on getting up a systematic develop-

ment of these Modeling Languages, a goal that has

not yet been reached because it has been doing more

study of the technical aspects of DSLs’ design and

implementation, such as: case studies and technical

reports on individual DSLs; design approaches and

techniques for implementing DSLs; and integrating

DSLs with other developmental approaches (Strem-

beck and Zdun, 2009).

2.2 Modeling Languages

A modeling language (ML) is a set of words and

symbols, supported by validation rules and semantics,

which make it possible to create models or diagrams.

It can be graphical/visual or textual, depending on its

scope and domain (He et al., 2007).

A modeling language is created from a metamodel

within a certain domain, therefore, it can be seen as

a model of a modeling language (Ma et al., 2004).

In order to give the modeling languages functional-

ity to use and evaluate the models, different types of

mechanisms are included in the metamodeling pro-

cess (algorithms, generic mechanisms, speciﬁc mech-

anisms and hybrid mechanisms) (Karagiannis and

Kühn, 2002). In other words, the metamodel is able to

highlight the properties of the model, derived from its

capacity of abstraction. There are several metamod-

eling approaches. The most commonly used is Meta

ICEIS2015-17thInternationalConferenceonEnterpriseInformationSystems

312

Object Facility (MOF) and it has been used, for in-

stance, in the development of UML and SysML.

In terms of representation type, MLs can be di-

vided in two groups: graphical and textual. On one

hand, graphical MLs use a diagram technique with

named symbols that represent concepts and lines that

connect the symbols and represent relationships and

various other notation to represent constraints. On the

other hand, textual MLs use standardized keywords

accompanied by parameters or natural language terms

and phrases to make computer-processable expres-

sions. Considering modeling approaches, in the ﬁeld

of computer science, more speciﬁc types of MLs

have recently emerged, due to new challenges on

very different areas, namely: Algebraic, Behavioural,

Discipline-Speciﬁc, Domain-Speciﬁc, Framework-

Speciﬁc, Object-Oriented or Virtual Reality.

An ML is very important because it helps to elu-

cidate their stakeholders that have modeling expertise

to understand what and how they can make a repre-

sentation of the system-of-interested. Also, its fea-

tures such as syntax, abstraction and compatibility

with programs can facilitate its use and its correspon-

dent metamodels may give a perception about the ad-

equacy of the language to the concrete situation.

2.3 Evaluation Frameworks

Hereby, we are exposing three frameworks that have

been developed to evaluate the quality of concep-

tual models and three business process modeling lan-

guages.

SEQUAL is a framework that was presented in

1994 with the goal of evaluating systematically the

quality of information systems and other conceptual

models through the notions of syntactic, semantic and

pragmatic applied to them (Krogstie et al., 1995). One

year later it was extended to more three features, in-

spired by FRISCO’s six semiotic layers of communi-

cation. It is considered the ﬁrst framework to have the

objective on evaluating models’ quality.

Essentially, the ﬁrst version of the framework only

had four concepts — Model, Domain, Language and

Audience interpretation — and six pair relations be-

tween them. Its extension, in 1995, has brought

the addition of a new entity (Participant knowledge)

and the creation of four relationship types. In 2012,

Krogstie makes a third instance of his framework

and adds "deontic" as a new quality criteria that fo-

cuses on what is right and needed for the organiza-

tion’s goals (Krogstie, 2012). SEQUAL’s formulas

are based on the audience, the language, the model,

the domain, the audience’s knowledge and the audi-

ence’s interpretation.

On 2005, a paper about focusing the standardiza-

tion of business process modeling was presented. It

intended to compare some Modeling Languages that

could be used for this context. The three selected

BPM languages were EEML, UML and BPMN,

and their framework comprised the following items:

Goals of modeling task, language extension, domain,

externalized model, knowledge of the stakeholders,

the social actors’ interpretation and the technical ac-

tors’ interpretation (Nysetvold and Krogstie, 2005).

On 1997, Teeuw and van der Berg published a pa-

per about their perspective on general quality crite-

ria for conceptual models and a framework that was

used to evaluate the redesign of business processes.

Like SEQUAL, they also defend that a good, qual-

ity model must have syntactic, semantic and prag-

matic qualities and, considering that the concepts

can be captured with a suitable language (assuring

the ﬁrst quality), they present as criteria for the sec-

ond and the third: completeness, inheritance, clarity,

consistency, orthogonality and generality (Teeuw and

van der Berg, 1997). Considering the Testbed evalu-

ation framework, it was applied to behaviour models

and it was designed with 4 dimensions: Functionality,

Ease of use, Business Process Redesign (BPR) trajec-

tory and General.

Although they use interesting criteria, the re-

lated work frameworks don’t satisfy the goal of this

work because they evaluate other matters instead of

UIMLs, respectively, the Quality of Conceptual Mod-

els, the Quality of Business Process and Business Pro-

cess redesign.

The selection of the appropriate Modeling Lan-

guage inﬂuences the quality of the ﬁnal output,

whether it is a model or a software application (In-

ternational Organization for Standardization and

International Electrotechnical Commission, 2008;

Krogstie et al., 1995; Rech and Bunse, 2009). Qual-

ity integration in software development processes that

use models, such as Model-Driven Software Develop-

ment, is the ultimate reason that led us to propose this

framework. ARENA displays a Framework that, after

analysing the MLs’ characteristics, shows the evalua-

tion of each one and helps the user to choose the most

appropriate ML, in order to assure a quality output.

It is the most appropriate evaluation framework for

modelling languages, because it is not only divided

into general and speciﬁc properties (which takes into

account the considered domain), but also because it is

an extensive comparison and assessment, resembling

the CMS Matrix.

AssessingtheQualityofUser-interfaceModelingLanguages

313

ML_Quality

Property

Metric

Dimension

ML_Info

General_Property Specific_Property

Range

«enumeration»

NotationKind

1 1

0..*

1..*

0..*

1..*

uses

1..*

Figure 1: ARENA Framework — Main Concepts.

3 THE ARENA FRAMEWORK

Figure 1 shows that a Modeling Language’s Quality

is the central class. It contains the ﬁnal quantitative

output. This class uses the Modeling Language’s

information as an input and has a Range.

This one is calculated using a formula that

receives as an input the values of the quality and

quantity evaluations (respectively represented as

Dimension and Metric classes) and multiplies each

by a previously deﬁned weight. The returned output

is a value that represents the language’s Quality

within a rating scale, as explained hereinafter on

section 4. All domains share the General Properties,

as opposite to Speciﬁc Properties, that differ from

each other (e.g. UIMLs, BPMLs etc). The Notation

Kind enumeration is developed as a droptext, so the

user can select one option from the displayed list.

The proposed framework intends to bridge that gap

and it is composed by general and speciﬁc properties.

3.1 General Properties

Interoperability. The language must be fully com-

patible with several tools, i.e. should allow to do

the same tasks and diagrams in different software

tools. Also, it shall be possible to combine with

another modeling and programming languages (PLs)

and tools, being supported by mechanisms that guar-

antee those possibilities. Dimensions and Metrics:

Number of Compatible Applications (International

Organization for Standardization and International

Electrotechnical Commission, 2008), Number of In-

tegration Mechanisms (Karagiannis and Kühn, 2002)

and Tool Supportability.

Notation. A Notation or concrete syntax is a set

of signs that enable to represent models. A model-

ing language may have two types of notation: graph-

ical/visual or textual. Dimensions: Representation

Type and Supporting Mechanisms.

Size. Completeness is one of the biggest chal-

lenges regarding the development of modeling lan-

guages and respective models (Baader et al., 2003;

Hoppenbrouwers et al., 2005; Teeuw and van der

Berg, 1997). This property states that the language’s

meta-model shall include the most important con-

cepts. Therefore, the meta-modeling approach is es-

sential for assuring that the modeling languages allow

producing concrete quality models. Metrics: Number

of Views, Number of Classiﬁers and Number of Re-

lationships.

3.2 Speciﬁc Properties

Application Actions. This property lists a series of

actions that the generated application can support, in

different contexts.

Other Features. It is a set of properties of

the UIMLs that provide additional information about

them. In this paper, they will be considered, but not

weighted to calculate the quality.

Pattern Usage. This property displays a list of

simple and reusable templates that help the user solv-

ing common problems that appear during the design

phase.

Tool Support - Model to Model (M2M) Trans-

formations. This property tells if the compatible

programs are capable of transforming the language’s

models format into another format or if the language

can produce more than one format.

Tool Support - Model to Text (M2T) Transfor-

mations. This property is a key aspect of model-

driven development. It refers the language’s ability

or inability to generate textual artefacts from models

and if so, which mechanisms or techniques make it

possible.

Tool Support - Validation. This property has

the purpose of listing which systems, applications or

mechanisms can analyse the models created by the

user and validate them.

User Interactions. This property intends to show

which ways users can interact with the application,

whether it is with mouse, keyboard, touch or other

means.

Widget Types. This property lists a set of graph-

ical user interface elements (either structural or be-

havioural) that the language makes available for the

designer.

ICEIS2015-17thInternationalConferenceonEnterpriseInformationSystems

314

Table 1: UIMLs comparison based on the ARENA Framework.

Property

Language

Property

Group

UMLi UsiXML XIS XIS-Mobile

Average

(UIMLs)

Number of Views Abstract Syntax 4 5 6 6 5

Number of Classiﬁers " 13 12 18 46 22

Number of Relationships " 11 20 12 16 15

Representation Type

Concrete

Syntax

Graphical Textual Graphical Graphical —

Supporting Mechanisms " Core meta-modelling

XML Metamodel and

Cameleon reference

framework

UML Proﬁle UML Proﬁle —

Tool Supportability " ARGOi

Eclipse, Visual Studio,

Notepad++, Kate,

W3schools.com and others

ProjectIT Studio Enterprise Architect —

Application Actions

Speciﬁc

Properties

OK, Cancel, Search, Back,

Next, Up, Down, Quit,

other customizable actions

(N/A)

CRUD, OK, Cancel,

Navigate, Select, Close,

Associate, Dissociate,

other customizable actions

CRUD, OK, Cancel,

Delete All, Open Browser,

Web Service, Navigate,

Select, other customizable

actions

User Interactions "

Click, Select, (Keyboard)

Type, Scroll

Move pointer, Click,

Double click, Depress,

Release, Drag over, Drag

drop, Focus, Select,

Choose, Toggle, View

Click, Select, Drag,

(Keyboard) Type

Tap, Double Tap, Long

Tap, Swipe, Pinch, Stretch,

(Touchscreen) Type

Pattern Usage "

Abstract Presentation

Pattern, Concrete

Interaction Object

Concepts & Task Model,

Abstract UI, Concrete UI,

Final UI, Inter-model

mapping, Context

translation

Composite, Single Choice,

Multiple Choice, List

Selection, Continuous

Filter, Menu Navigation,

Grid Layout, Tab Menu,

Tabular Set, Double List

Composite, Single Choice,

Multiple Choice, List

Selection, Single Text

Entry, Multiple Text Entry,

Springboard, List Menu,

Tab Menu, Option Menu

and 10 more

Tool Support - Validation "

ARGOi validates both

UML and UMLi models,

due to both grammars are

speciﬁed in terms of the

UML metamodel

(N/A)

Eclipse .NET, .NET

Framework and Project

IT’s three components:

Requirements, UML

Modeler and MDD Code

Generator

XIS-Mobile Framework,

namely EA’s Model

Validator, supports model

validation

—

Tool Support - M2M

Transformations

" No (N/A) No

Yes, using Enterprise

Architect’s MDG

technologies

—

Tool Support - M2T

Transformations

" No (N/A)

Yes, deﬁning architectures,

templates, and interface

generation processes that

are compatible with

Windows Forms.NET and

ASP.NET platforms

Yes, using XMI-validated

and generated models,

Acceleo and the OS’s

compatible programming

languages. It can generate

Java, C#, Objective-C,

XML and XAML

—

Number of Compatible

Applications

" 1 (ArgoUML) 1 (Eclipse) 1 (ProjectIT Studio) 1 (Enterprise Architect) 1

Number of Integration

Mechanisms

3 (ARGOi, OCL rules and

LOTOS rules)

4 (MDG Technologies,

XML Parser, UsiGesture

and UsiDistrib)

2 (Eclipse .NET and .NET

Framework)

2 (MDG Technologies,

XML Parser)

Widget Types "

Label, Text ﬁeld, Combo

box, Selectable list, Button

Push button, List box,

Check box, Window,

Panel, Table, Cell, Dialog

box, Embeded multimedia,

Menu, Spin button

Button, Text box, List,

Menu, Window, Link,

Search bar, Checkbox,

Radius button, Drop-down

list, Form, Dialog, Label

and 15 more

Button, Text box, List,

Menu, Window, Link,

Search bar, Checkbox,

Radius button, Drop-down

list, Label, Image, Date

Picker and 9 more

Other Features "

It is possible to model

Activity Diagrams in

UMLi, using the Use Case

Diagram and the

InitialInteraction construct.

It implements the µ7

concept, as it is Device-,

User-, Culturality-,

Organization-, Context-,

Modality- and

Platform-Independent.

Supports Windows

Forms.NET, ASP.NET and

JSP, using Model-to-Text

transformations.

Model validation uses a set

of rules deﬁned in C#,

implemented with EA’s

Automation Interface. It is

possible to generate

User-Interfaces View

models on EA.

—

4 APPLYING ARENA TO

EVALUATE USER-INTERFACE

LANGUAGES

Table 1 summarizes the main properties of four anal-

ysed UIMLs — UMLi (Silva, 2002), UsiXML (Lim-

bourg et al., 2004), XIS (Silva et al., 2003) (Mar-

tins and Silva, 2007) (Silva et al., 2007) and XIS-

Mobile (Ribeiro and Silva, 2014b) (Ribeiro and Silva,

2014a). The columns Number of Views, Number of

Classes and Number of Relationships represent the

metrics with the same name, from ARENA’s property

Size. The ARENA property Notation is represented

by two dimensions: Representation Type and Sup-

porting Mechanisms. The column Tool Supportability

refers to a dimension from ARENA’s property Inter-

operability, as well as metrics Number of Compatible

Applications and Number of Integration Mechanisms.

The remaining columns have exactly the same name

and meaning of the speciﬁc properties, as shown on

the previous section. The quality value of each lan-

guage is determined by the formula below, designated

as the ARENA General Equation of Quality.

AssessingtheQualityofUser-interfaceModelingLanguages

315

QualityLanguage

∑

i=1

∗ r

= w_absStxProps ∗ r_absStxProps + w_concStxProps ∗ r_concStxProps + w_specProps ∗ r_specProps =

= w_numViews ∗ r_numViews + w_numClassi f s ∗ r_numClassi f s +w_numRelats ∗ r_numRelats+

+w_repType ∗ r_repType + w_suppMechs ∗ r_suppMechs + w_toolSupp ∗ r_toolSupp+

+w_numAppActs ∗ r_numAppActs + w_numUserInts ∗ r_numUserInts + w_numPatterns ∗ r_numPatterns+

+w_Valid ∗ r_Valid + w_M2MTr f s ∗ r_M2MTr f s + w_M2T Tr f s ∗ r_M2T Tr f s+

+w_numCompApps∗ r_numCompApps +w_numIntMechs∗ r_numIntMechs+ w_numWidgets∗ r_numWidgets

The notation used on the formula above has

the following meaning: i — Framework’s met-

ric/dimension; j — Sum of the number of metrics

with the number of dimensions; n — Language’s

name; r — Metric’s/dimension’s rate; w — Met-

ric’s/dimension’s weight. The rating each property

can have is the following: 1 — Very Low; 2 — Low;

3 — Medium; 4 — High; 5 — Very High.

If quantiﬁable and with equal level of importance,

each property’s value is compared towards that

property’s average value in this work. Its rating is

given according to how far it is from the average,

upwards or downwards, according to the follow-

ing ranges: Value < AvgValue − 1/2 ∗ AvgValue ;

Value ∈ [AvgValue − 1/2 ∗ AvgValue,AvgValue −

1/4 ∗ AvgValue[ ; Value ∈ [AvgValue − 1/4 ∗

AvgValue,AvgValue + 1/4 ∗ AvgValue] ; Value ∈

]AvgValue + 1/4 ∗ AvgValue, AvgValue + 1/2 ∗

AvgValue] and Value > AvgValue + 1/2 ∗ AvgValue.

The exceptions (non-quantiﬁable properties) are:

Representation Type — The most popular value has

higher rating than the least; Supporting Mechanisms

— Since it’s very speciﬁc for each language, this

rating reﬂects easiness to understand and extend

the metamodel; Tool Supportability — It may be

unique for each language, so its rating is focused on

the programs’ general usability and functionality;

and Tool Support (Validation, Model to Model and

Model to Text) — Since the programs referred on

the Tool Supportability property generate models, we

assumed that Validation is correctly done, as well as

the transformations.

The languages in which we don’t know how these

characteristics are covered were assigned 1 (Very

Low) and the languages that don’t have these features

were assigned 2 (Low). Each rated property will be

multiplied by its weight, as referred on section 3, and

the sum of this products will return the language’s

quality, according to the ARENA General Equation

of Quality.

5 DISCUSSION

Having four UIMLs compared, it is possible to see

that, for the same domain, the languages do not differ

much in terms of number of views and number of rela-

tionships, but they do when it comes about supporting

mechanisms and tool supportability. This can be ex-

plained by their maturity and popularity. Also, there

are patterns concerning number of classiﬁers and no-

tation, both with a single exception.

If the Number of Views seems to be balanced for

these four languages (all have between 4 and 6), it

is not possible to conclude the same about the Num-

ber of Classiﬁers, because UsiXML, UMLi and XIS

have, respectively, 12, 13 and 18 and XIS-Mobile has

46, resulting in a difference of 34 items to the lowest

number. Less unbalanced is the Number of Relation-

ships — it varies from UMLi’s 11 to UsiXML’s 20.

For Representation Type, clearly there is a prefer-

ence for Graphical in disfavour of Textual, probably

due to easiness to see the models in a clearer way and

according to the WYSIWYG paradigm. About Sup-

porting Mechanisms, as expected due to their nature,

XIS and XIS-Mobile have the same value, oppos-

ing to UMLi that is an extension to UML and there-

fore uses a simpler approach and UsiXML that, be-

ing based on XML, results in a different architecture

that is complemented by Cameleon. Considering Tool

Supportability, all languages follow different tenden-

cies, with a clear advantage to UsiXML. This might

be due to XML’s universality (as it is used in RSS,

SOAP messages and XHTML), it is both human- and

machine-readable and it is capable of representing all

Unicode characters.

XIS is the top language concerning to Application

Actions, since it allows the software to do at least

9 different actions. Contrasting with that, UsiXML

is the better one in terms of User Interactions, since

it provides 12 gestures that can be done, while XIS

and UMLi only have 4 deﬁned. The language that in-

cludes more Patterns is XIS-Mobile, Composite being

ICEIS2015-17thInternationalConferenceonEnterpriseInformationSystems

316

Table 2: UIMLs evaluated with ARENA Framework.

Property

Language

Weight UMLi UsiXML XIS XIS-Mobile

Number of Views 0.033 3 3 3 3

Number of Classiﬁers 0.033 2 2 3 5

Number of Relationships 0.033 2 4 3 3

Representation Type 0.033 4 3 4 4

Supporting Mechanisms 0.033 3 3 3 3

Tool Supportability 0.033 3 4 2 4

Application Actions 0.1 3 1 3 3

User Interactions 0.1 2 5 2 3

Pattern Usage 0.1 1 2 3 5

Tool Support - Validation 0.1 3 1 3 3

Tool Support - M2M

Transformations

0.1 2 1 2 4

Tool Support - M2T

Transformations

0.1 2 1 4 4

Number of Compatible

Applications

0.05 3 3 3 3

Number of Integration

Mechanisms

0.05 3 3 3 3

Widget Types 0.1 1 2 5 4

Other Features — — — — —

Total Quality 1 2 2 3 4

the only one of Structural Design type and the remain-

ing 19 of UI Design, that is 10 more than XIS. Still

about this property, UsiXML has 4 which are based

on Cameleon reference framework and UMLi has 2

patterns, each for a UML package that was extended.

About Tool Support - Validation, each language

has its own method, always inﬂuenced by the compat-

ible applications and the ones shown in Tool Support-

ability column, which in many cases, are the same.

We don’t know how it’s done for UsiXML, but for

XIS it is the most extensive process. Now referring

to Tool Support - Model to Model Transformations,

only XIS-Mobile has this feature implemented, tak-

ing advantage of EA’s capabilities, while regarding

Tool Support - Model to Text Transformations, XIS

and XIS-Mobile provide this MDD-core feature, as

opposed to UMLi, with a great focus on Web tech-

nologies. Again, we know nothing about these two

properties on the UsiXML case.

Now regarding the Number of Compatible Ap-

plications, all languages have the same value, there-

fore we considered than none stood out more than the

other. It seems to be a bit contradictory to the Tool

Supportability property. The difference is that the lat-

ter doesn’t take into account the language’s suitable

program but the produced models’ compatibility with

other platforms. For the Number of Integration Mech-

anisms, despite some are easier to work with (XML

Parser) than others (LOTOS rules), all languages have

received the same rating, due to the number is close

to the average of this property, in this work.

The attractive and useful Widget Types inform

that XIS and XIS-Mobile have a higher number of

items (28 and 22, accordingly), although not exactly

the same. UsiXML stands with 11 and UMLi has only

5, which is expected from an extension that claims to

"be a conservative extension of UML" and "should

introduce as few new models and constructs into the

UML as possible" (Silva, 2002).

According to Table 2, for this evaluation, XIS-

Mobile has the highest quality value — 4 — among

these four UIMLs, in the considered domain.

6 CONCLUSION

This paper presents an overview of academic research

related to the problematic of choosing the best from

several modeling languages and the quality assess-

ment frameworks as a response to that problem. We

have evaluated four UIMLs using ARENA, a frame-

work oriented to that solution. In this case, ARENA’s

most adequate dimensions and metrics were used,

along with User-Interface speciﬁc properties, deﬁned

for this purpose and context. With this analysis, it

is possible to list the features in which each UIML

stands out from the others, either because they are bet-

ter or they are unique. Clearly there is a tendency to

bet on graphical notations. This can be due to guide-

lines that compose the well known Model-Driven En-

AssessingtheQualityofUser-interfaceModelingLanguages

317

gineering. Also an advantage, is to use these graph-

ical models to implement the alternative approach to

produce software: Model-Driven Software Develop-

ment.

The metamodel types chosen vary very much from

modeling language to modeling language, since some

use MOF, others BNF or UML Proﬁles, and even

MVC is still used. Another possible choice is to ex-

tend other ML’s metamodel. Tool support is very im-

portant, in terms of usability and functionality. They

are also responsible to render and validate the pro-

duced models, so this aspect can be the quality bot-

tleneck. In terms of look-and-feel, it is very impor-

tant for the designer to have available design patterns

and builder tools, so he can produce an attractive and

easy interface. He must also be able to choose be-

tween several actions and widgets, so the mobile, web

or desktop software application can have an attractive

layout and great impact.

In the future, ARENA will also be used to com-

pare and evaluate General-Purpose Modeling Lan-

guages, such as UML and SysML or Business Process

Modeling Languages, namely BPMN and DEMO, al-

ways taking into account the general and the domain

speciﬁc characteristics.

ACKNOWLEDGEMENTS

This work was partially supported by the ARENA

2012 IBM Country Project, and by national funds

through Fundação para a Ciência e a Tecnologia

(FCT) with references UID/CEC/50021/2013 and

EXCL/EEI- ESS/0257/2012 (DataStorm).

REFERENCES

Achilleos, A., Yang, K., Georgalas, N., and Azmoodech,

M. (2008). Pervasive service creation using a model

driven petri net based approach. In Wireless Commu-

nications and Mobile Computing Conference, 2008.

IWCMC ’08. International, pages 309–314.

Baader, F., Calvanese, D., McGuiness, D., Nardi, D., and

Patel-Schneider, P. (2003). The Description Logic

Handbook: Theory, Implementation and Applications.

Cambridge University Press.

Ceri, S., Fraternali, P., Bongio, A., Brambilla, M., Comai,

S., and Matera, M. (2002). Designing Data-Intensive

Web Applications. Morgan Kaufmann Publishers Inc.,

San Francisco, CA, USA.

Desel, J. and Juhás, G. (2001). What Is a Petri Net?, volume

2128 of LNCS. Springer-Verlag.

Dietz, J. (2001). DEMO: Towards a discipline of organi-

sation engineering. European Journal of Operational

Research, 128:351–363.

Frank, M. R. (1996). Model-based User Interface Design by

Demonstration and by Interview. College of Comput-

ing, Georgia Institute of Technology 1996. Directed

by James Foley.

He, X., Ma, Z., Shao, W., and Li, G. (2007). A Metamodel

for the Notation of Graphical Modeling Languages.

In Proceedings of the 31st Annual International Com-

puter Software and Applications Conference - Volume

01, volume 1 of COMPSAC ’07, pages 219–224. IEEE

Computer Society.

Hoppenbrouwers, S., Proper, E., and van der Weide, T. P.

(2005). A Fundamental View on the Process of Con-

ceptual Modeling. In Conceptual Modeling - ER 2005,

volume 3716, pages 128–143. Springer-Verlag.

International Organization for Standardization and Interna-

tional Electrotechnical Commission (2001). ISO/IEC

9126-1:2001(E) Quality Model.

International Organization for Standardization and Interna-

tional Electrotechnical Commission (2008). ISO/IEC

CD 25010.2 Software and Quality in use models.

Karagiannis, D. and Kühn, H. (2002). Metamodelling Plat-

forms. In Proceedings of the Third International Con-

ference EC-Web 2002 - Dexa 2002, volume 2455,

pages 182–195. Springer-Verlag.

Kosar, T., Oliveira, N., Mernik, M., Pereira, M. J. V.,

Cre-

pinšek, M., da Cruz, D., and Henriques, P. R. (2010).

Comparing general-purpose and domain-speciﬁc lan-

guages: An empirical study. In Computer Science and

Information Systems, volume 7. University of Novi

Sad, Serbia.

Krogstie, J. (2012). Model-Based Development and Evo-

lution of Information Systems: A Quality Approach,

chapter 5, pages 249–280. Springer-Verlag London.

Krogstie, J., Lindland, O. I., and Sindre, G. (1995). Deﬁn-

ing quality aspects for conceptual models. Faculty of

Electrical Engineering and Computer Science -, The

Norwegian Institute of Technology.

Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon,

L., Florins, M., and Trevisan, D. (2004). USIXML:

A User Interface Description Language for Context-

Sensitive User Interfaces.

Luoma, J., Kelly, S., and Tolvanen, J.-P. (2004). Deﬁn-

ing domain-speciﬁc modeling languages: Collected

experiences. In OOPSLA 4th Workshop on Domain-

Speciﬁc Modeling. ACM.

Ma, H., Shao, W., Zhang, L., Ma, Z., and Jiang, Y. (2004).

Applying OO Metrics to Assess UML Meta-models.

In UML 2004, LNCS, volume 3273, pages 12–26.

Springer-Verlag.

Martins, C. and Silva, A. R. (2007). Modeling user inter-

faces with the xis uml proﬁle. In Proceedings of the

ICEIS 2007.

Mernik, M., Heering, J., and Sloane, A. M. (2005). When

and how to develop domain-speciﬁc languages. ACM

Computing Surveys, 37:316–344.

Nysetvold, A. G. and Krogstie, J. (2005). Assessing busi-

ness processing modeling languages using a generic

quality framework. Norwegian University of Science

and Technology.

ICEIS2015-17thInternationalConferenceonEnterpriseInformationSystems

318

Object Management Group (2011). OMG Uniﬁed Mod-

elling Language (OMG UML), Infrastructure - Ver-

sion 2.4.1. Object Management Group.

Object Management Group (2013). Business Process

Model and Notation (BPMN) - Version 2.0.2. Object

Management Group.

Object Management Group (2014). Interaction Flow Mod-

elling Language (IFML) - FTF - Beta 2 - Revision 21.

Object Management Group.

Paternó, F., Santoro, C., and Spano, L. D. (2009). MARIA:

A Universal, Declarative, Multiple, Abstraction-

Level Language for Service-Oriented Applications

in Ubiquitous Environments. ACM Transactions on

Computer-Human Interaction, 16:1–30.

Rech, J. and Bunse, C. (2009). Model-Driven Software De-

velopment: Integrating Quality Assurance. Informa-

tion Science Reference.

Ribeiro, A. and Silva, A. R. (2014a). Evaluation of xis-

mobile, a domain speciﬁc language for mobile appli-

cation development. In Journal of Software Engineer-

ing and Applications, number 7 in 11. Scientiﬁc Re-

search Publishing.

Ribeiro, A. and Silva, A. R. (2014b). Xis-mobile: A dsl

for mobile applications. In Proceedings of ACM SAC

2014 Conference. ACM.

Silva, A. R., de Sousa Saraiva, J., Silva, R., and Martins,

C. (2007). XIS - UML Proﬁle for eXtreme Modelling

Interactive Systems. In Proceedings of the MOMPES

2007. IEEE Computer Society.

Silva, A. R., Lemos, G., Matias, T., and Costa, M. (2003).

The XIS Generative Programming Techniques. In

Proceedings of the 27th COMPSAC Conference. IEEE

Computer Society.

Silva, P. P. (2002). Object Modelling of Interactive Sys-

tems: The UMLi Approach. PhD thesis, University of

Manchester.

Strembeck, M. and Zdun, U. (2009). An approach for

the systematic development of domain-speciﬁc lan-

guages. Software Practice and Experience, 39:1253–

1292.

Teeuw, W. B. and van der Berg, H. (1997). On the Quality of

Conceptual Models. In 16th International Conference

on Conceptual Modeling - ER’97, volume 1331, pages

1–18. Springer-Verlag.

Trætteberg, H. (2008). Integrating Dialog Modeling and

Domain Modeling - the Case of Diamodl and the

Eclipse Modeling Framework. In Journal of Univer-

sal Computer Science, volume 14, pages 3265–3278.

J.UCS.

van Deursen, A., Klint, P., and Visser, J. (2000). Domain-

speciﬁc languages: An annotated bibliography. ACM

SIGPLAN Notices, 35(6):26–36.

AssessingtheQualityofUser-interfaceModelingLanguages

319