A Usability Evaluation of Graphical Modelling Languages
for Authoring Adaptive 3D Virtual Learning Environments
Ahmed Ewais
1,2
and Olga De Troyer
1
1
WISE Research Group, Department of Computer Science, Vrije Universiteit Brussel, Brussel, Belgium
2
Department of Computer Science, Arab American University, Jenin, West Bank, Palestine
Keywords: Graphical Modelling Language, Domain Specific Modelling Language, Adaptivity, Virtual Reality,
3D Virtual Learning Environments.
Abstract: Adaptive three-dimensional (3D) Virtual Learning Environments (VLEs) offer many advantages for
learning, but developing them is still far from easy and is usually only done by specialized people.
However, involving teachers in the development of learning material is essential. One way to support
teachers in authoring adaptive 3D VLEs is the use of domain specific modelling languages as such
languages provide a high level of abstraction. In addition, graphical languages are recommended for non-
technical users. Although such an approach, i.e. graphical domain specific modelling languages, seems to be
promising there is a need for evaluating this in practice. Usability and acceptance could become a problem
because the authoring process could become relatively complex. This paper reports on a pilot evaluation
performed to evaluate the use of graphical modelling languages for designing (i.e. authoring) adaptive 3D
VLEs.
1 INTRODUCTION
To come to effective and challenging adaptive
Virtual Learning Environments (VLEs), it is
essential to involve educators and experts in the
subject domain, in the development of the VLE.
However, developing a 3D VLE is still quite a
technical issue, and adding adaptivity to such an
environment does not make it easier. Supporting or
involving educators in the development of adaptive
3D VLEs is still in its infancy. Therefore, this is a
priority for our research.
In the context of educational games, it is already
noted that involving educators in the development
of educational 3D games can be achieved by
providing user friendly, effective and efficient
authoring tools (Overmars, 2004; Marchiori et al.,
2011). This is also what we want to achieve for 3D
VLEs. Therefore, we proposed a set of easy to use
graphical Domain Specific Modelling Languages
(DSMLs) for authoring adaptive 3D VLEs (Ewais
and De Troyer, 2013).
The rationale behind using graphical languages
is that graphical specifications are, in general, easier
for the communication with non-technical people.
They make it easy to convey information, as many
people can think and remember things in term of
pictures (Boshernitsan and Downes, 2004).
Furthermore, they can provide appropriate
abstractions that make the specifications easier
(Moody, 2009). However, they should be defined
with care to be usable and effective.
In general, DSMLs are languages that use a
specific vocabulary dedicated to the modelling and
designing a specific class of problems (Deursen et
al., 2000). They are particular well suited for
domain experts as they use the vocabulary of the
domain rather than some general modelling
language vocabulary. Mostly, DSMLs are graphical
languages.
Giving due consideration to the usability of
software is essential. Good usability provides
different advantages: improved user satisfaction,
increased usefulness and effectiveness, improved
ease of learning and use, reduced training and
support costs. Therefore, we conducted a pilot
evaluation of the graphical DSMLs proposed for
authoring adaptive 3D VLEs. The primary aim of
the conducted evaluation was to reveal if we had
chosen the right direction with these DSMLs and if
they were usable for the task of specifying an
adaptive storyline-based 3D VLEs.
459
Ewais A. and De Troyer O..
A Usability Evaluation of Graphical Modelling Languages for Authoring Adaptive 3D Virtual Learning Environments.
DOI: 10.5220/0004947204590466
In Proceedings of the 6th International Conference on Computer Supported Education (CSEDU-2014), pages 459-466
ISBN: 978-989-758-020-8
Copyright
c
2014 SCITEPRESS (Science and Technology Publications, Lda.)
Section 2 briefly describes the graphical
languages. Section 4, 5 and 6 present respectively
the evaluation and its results. Section 7 concludes
the paper.
2 AUTHORING ADAPTIVE 3D
VLE
Based on our previous work done in the context of
the EU-project GRAPPLE (De Bra et al., 2010) and
insight obtained from a literature review, we
proposed a new approach for authoring adaptive 3D
VLEs (Ewais and De Troyer, 2013). The kernel of
the approach is a set of three DSMLs: one for
expressing the pedagogical structure of the
underlying learning domain (the Pedagogical Model
Language), one to define the (adaptive) learning
path (i.e. storyline) for the course (the Adaptive
Storyline Language), and one for specifying the 3D
adaptivity inside the different topics of the course
(the Adaptive Topic Language). We described each
language briefly.
The Pedagogical Model Language (PML)
To define the pedagogical structure of the adaptive
3D VLE, the authors can connect learning concepts
(defined in the Domain Model
1
) via Pedagogical
Relationships Types (PRTs) (see Figure 1).
A typical example of a PRT is the prerequisite.
The goal of this PRT is to define when a learning
concept is a prerequisite for another learning
concept meaning that the learner needs to study the
first concept before he can start learning second
concept. Other possible PRTs are Defines,
Illustrates, Interest, Propagates_knowledge, and
Update_knowledge.
Figure 1: PML elements: A) learning concept B)
Pedagogical Relationship Types (PRT).
The PRTs are associated with Pedagogical Update
Rules (PURs), which are condition-action rules.
This rule mechanism is used to define how the
knowledge of the learner (kept in the User Model
2
)
1
The Domain Model defines the learning concepts. It is outside
the scope of this paper.
2
The User Model is a typical model used in adaptive systems. It
captures all information about the user, like preferences and his
knowledge about the learning concepts.
should be updated, i.e. a rule defines how and which
User Model attributes should be updated. When the
learner follows the course, the PUR of a PRT is
triggered on accessing the source learning concepts
of the PRT.
The general format for the PURs is as follows:
IF <user_model_condition>
THEN <user_model_update_actions>
Different PRTs are predefined. In general, the
predefined PRTs and their associated update rules
(PURs) are sufficient to accommodate common
pedagogical relationships between learning
concepts in different domains. However, authors
can define new PRTs or change default PURs.
An example Pedagogical Model is given in
Figure 2. The learning concept Sun is a prerequisite
for the learning concepts Mercury, Venus, Mars,
and Earth. The PUR associated with this
prerequisite-for PRT is given in the callout symbol.
Furthermore, when the learner learns about
Mercury, his knowledge about Venus will also
increase. This is specified in the PUR associated
with the update-knowledge-of PRT between
Mercury and Venus. This PRT is also applied to
other concepts: Venus, Mars, and Earth.
Figure 2: Pedagogical Model for a number of learning
concepts related to a Solar System Course.
Adaptive StoryLine Language (ASLL)
This language allows defining a learning path inside
a 3D VLE (i.e. storyline) by enables the authors to
define a set of topics and connecting them. Next,
the author can also indicate how this storyline
should adapt to the individual learner.
Decomposing the storyline into different topics
is used to reduce complexity. Topics can be
compared to chapters in regular courses. To express
the adaptivity of the storyline, each topic is
connected with a next topic via a so-called Storyline
Adaptation Rule. An example is given in Figure 4.
Figure 3 shows the graphical notations of ASLL.
Note that a storyline has a start and end. Start and
end symbols (Figure 3 (A) and (F)) can be used to
CSEDU2014-6thInternationalConferenceonComputerSupportedEducation
460
specify the textual and/or audio/video messages to
be presented to the learner at this point. Such
messages can e.g., be used to instruct the learner
what to do or what he achieved. Furthermore, for a
Storyline Adaptation Rule (Figure 3 (C)) the arrow
points from the source topic to the target topic.
Rules should be given a meaningful name in order
to increase the readability of the model.
Figure 3: ASLL elements: A) Start of a storyline; B)
Topic in a storyline; C) Topic Adaptation Rule; D)
condition; E) Adaptation state; F) End of a storyline.
A Storyline Adaptation Rule is a condition-action
rule. The condition (Figure 3 (D)) specifies when
the learner can proceed from the source topic to the
target topic, and is, in general, based on the
learner’s knowledge level about the source topic
and the suitability of the target topic. The action
part (Figure 3 (E)) is used to specify what kind of
adaptation should be applied to the learning
concepts involved in the target topic. Example
adaptations are marking learning concepts with
bounding boxes or annotations, hiding learning
concepts, or providing a guided tour to the concepts
related to the topic. Possible adaptation are
predefined, see (De Troyer et al., 2010).
Figure 4 shows an example of an adaptive
storyline for a course about the solar system. The
storyline is composed of the topics: Learning About
Stars, Inner Solar System, Moons, Outer Solar
System, and Advanced Topics. The learner will start
with a guided tour for the topic Learning About
Stars. After acquiring the required knowledge for
this topic, the learner will be directed to a new
topic, either to the Outer Solar System or to the
Inner Solar System, depending on the truth-values
of the conditions associated with the two storyline
adaptation rules. For instance, the storyline
adaptation rule between Learn About Stars and
Inner Solar System, is as follows:
IF ‘Learn About Stars’.knowledge greater than 90
AND ‘Inner Solar System’.suitability is TRUE
THEN APPLY ‘markobject’ TO ‘Inner Solar System’
The rule states that if the learner’s knowledge
about topic Learn About Stars is above the specified
value and the suitability of topic Inner Solar System
is true, then the markObjects adaptation should be
applied to the learning concepts of the Inner Solar
System topic.
Figure 4: Adaptive Storyline for 3D VLE Solar System
Course.
Adaptive Topic Language (ATL)
This language allows describing how the content
related to each topic should be adapted to the
individual learner, i.e. it allows specifying the
adaptivity within a single topic. This is done by
means of adaptation rules between learning
concepts. The rules are event-condition-action rules.
They are triggered by activities performed in the 3D
VLE. Figure 5 shows the symbols used in ATL.
Figure 5: ATL elements: A) Learning concept, B)
Adaptation rule, C) VR event, D) Condition, E) and F)
Adaptation type to be applied to a 3D learning Concept,
G) Notification Message.
A topic is composed of a set of learning concepts
(Figure 5 (A)). Learning concepts are connected
through so-called Topic Adaptation Rules. A topic
Adaptation Rule (Figure 5 (B)) has a source
(learning concept) and a target (learning concept).
The event part of the rule (Figure 5 (C)) specifies
the event that will trigger the rule. This event has to
occur with the source. The rule will only be
executed when the condition in the condition part
(Figure 5 (D)) is true. The action part (Figure 5 (E)
or (F)) specifies the adaptation to be applied on the
target learning concept. Version (E) is used when
AUsabilityEvaluationofGraphicalModellingLanguagesforAuthoringAdaptive3DVirtualLearningEnvironments
461
source and target are different; version (F) when
source and target are equal. Also a notification or
feedback message to the learner can be specified
(Figure 5 (G)). This message will be shown when
an adaptation rule is applied. A set of predefined
adaptation types is available (De Troyer et al.,
2009). Figure 6 shows an example Topic
Adaptation Rule. This rule will be fired once the
learner “comes close to” (VR event) Sun. Next, the
condition of the rule will be evaluated. Here, a test
on how often the learner already interacted with
Sun. If this is higher that the specified value, the
action part is executed, i.e. SemiDisplay adaptation
type will be applied to Earth to display Earth is a
semi way.
Figure 6: Example topic adaptation rule.
VR events are used to indicate when the adaptation
rules should be evaluated. The VR events are events
related to the learner’s activities inside the 3D VLE,
e.g., interaction with a 3D object or navigating to a
3D object. The condition must be satisfied in order
to actually perform the action-part of the rule. The
condition will in general deal with the learner’s
preferences, his learning background, and progress,
but may also consider previous activities performed
by the learner in the 3D VLE (captured by so-called
3D VLE activity history attributes). By including
conditions on previous activities performed by the
learner in the 3D VLE, the author is able to control
the learner’s behaviour in the 3D VLE, e.g., to
avoid that the learner wastes too much time by
playing around.
Two examples adaptation rules are given in
Figure 7. The first adaptation rule (Figure 7(A)) is
between two different learning concepts (Sun and
Earth). The VR event close to will trigger the rule.
When the rule is applied, the adaptation type display
will be applied to the target (Earth) to display the
C
Sun Earth
CloseTo
Display
C
Navigatetoward
Earth
Earth
DisableInteraction
Touch
Interactionhas
beendisabled
A) B)
Interacttoo
manytimes
KnowledgeSunisgood
enough
Figure 7: Examples of adaptation rules: A) adaptation rule
between two learning concepts; Sun and Earth, B)
adaptation rule on a single learning concept.
VR object that represents earth. The second
adaptation rule (Figure 7 (B)) is on a single learning
concept (Earth). The rule uses a touch VR event
and the disable_interaction adaptation type to
disable user interaction with the Earth 3D object
once the learner has already interacted with it too
many times. In both examples, a notification
message is provided.
3 PILOT EVALUATION
The goal of the conducted evaluation was to
perform a first evaluation of the modelling
languages from a usability and acceptability point of
view, in order to evaluate whether the approach
taken was appropriate and to gather feedback to
improve the languages before starting to implement
tool support.
As we were looking for critical feedback from
the viewpoint of usability and user satisfaction, we
asked PhD students from our universities to
participate in the evaluation. Four PhD candidates
and researchers and ten instructors were involved in
the evaluation. All of them were from the Computer
Science department, but they were rather novice in
3D or VR.
The evaluation was divided in three steps. The
first step introduced the participants to the different
notations of the languages; example models created
using the languages; a list of available Pedagogical
Relationship Types and their default associated
update rules; and a selection of possible adaptation
types.
In the second step, the participants had to do an
authoring task, i.e. designing an adaptive 3D VLE
about the Solar System using the three languages.
This authoring task was done using regular paper
and pen. Because it was not our purpose to evaluate
an authoring tool, but rather the level of
expressiveness of the visual notations of the
proposed languages and the effort needed to create
an adaptive 3D VLE using the graphical languages,
the use of pen and paper is acceptable. This
approach also avoided that we already spent a lot of
resources on the development of a software tool
before receiving any feedback on the proposed
languages. However, we also admit that using
paper-pencil rather than an authoring tool also has
some limitations. For instance, a software tool could
guide the correct use of the languages; this cannot
be achieved with pen and paper. To solve such
issues, an instructor was responsible for guiding and
helping the participants with syntactical issues.
CSEDU2014-6thInternationalConferenceonComputerSupportedEducation
462
In the third step, the participants filled in an
online questionnaire. The questionnaire was
carefully constructed to reduce possible bias. For
instance, there were positively as well as negatively
formulated questions, and questions were
formulated carefully to avoid that participants might
be encouraged to give more favourable answers
(Lazar, Feng, and Hochheiser, 2010, p.196).
Furthermore, the questions’ order was in such a way
that the answer to one question did not influence the
response to another question. Furthermore, each
participant did the evaluation individually, the
results were treated anonymously, and the
participants were informed that there were no right
or wrong answers and that it is was not an
evaluation of the participants themselves.
The questionnaire was composed of questions
from six categories: Demographic Information,
Authoring Adaptive 3D VLE (A3DVLE),
ISONORM 9241/110-S Evaluation Questionnaire
(ISONORM) (Prumper, 1999), Subjective
Impression Questionnaire (SIQ) (Davis et al., 1989),
Qualitative Feedback (QF), and Workload
Perception (WP) (Hart and Staveland, 1988).
All questions were mandatory. As already
indicated, there were positively formulated
questions and negative formulated questions and all
closed question had a Likert scale from 1 (Strongly
Disagree) to 5 (Strongly Agree). For each individual
evaluation feedback on a positive question, a score
of 3 or higher was considered as “good”, as well as
a score of 3 or lower on a negative formulated
question.
4 EVALUATION RESULTS
All participants were from the domain of Computer
Science, but the demographic data indicated that
only few participants were familiar with VR/3D (5
out of 14 participants). However, all participants
were using the computer on a daily basis. The
average age was 32 (youngest was 26, eldest 36).
The majority of the participants were males (11 out
of 14). Only 2 (out of 14)participants reported to be
inexperienced with authoring standard courses. But
most of the participants (11 out of 14) had only
limited experience in authoring 3D VEs or
videogames in the context of e-learning. All
participants were familiar with graphical modelling
languages like UML.
In average, participants spent about 40 minutes on
the tasks (best time was 25 min.; worst was 90
min.).
Usability:
The usability of the graphical languages was
evaluated as ‘Medium/Neutral’ to ‘Good to Perfect’.
The bars chart in Figure 8 presents the results
concerning both ISONORM and A3DVLE
questionnaires. 8 positive formulated questions
related to the A3DVLE questionnaire that were
evaluated as ‘Good to Perfect’, while 3 questions
(positive formulated) were evaluated as ‘Neutral’.
Concerning the negatively formulated questions in
the A3DVLE questionnaire, 4 questions were rated
as ‘Good to Perfect’, 2 questions as ‘Neutral’, and 1
as ‘Poor’. 5 questions related to ISONORM
9241/110-S questionnaire were rated as ‘Good to
Perfect’ and 3 questions as ‘Neutral’. We now
provide more details.
Figure 8: ISONORM and General Authoring 3D VLE
Questionnaires Results.
In general, all questions related to the suitability for
the task, conformity with user expectations, self-
descriptiveness, usefulness and easy-to-use, were
rated good. The visual notations of the modelling
language allowed the participants to do the
authoring task without being 3D/VR experts. The
overall positive feedback on the usability questions
indicates that the modelling languages are
appropriate for the task. In addition, most of the
participants considered the modelling languages
rather intuitive. However, we have to note that the
participants had a good knowledge of modelling.
However, also a number of questions received a
neutral score. In particular, the questions related to
the suitability for learning were rated as
‘Medium/Neutral’. Furthermore, questions related
to understanding the goal of pedagogical model
were towards ‘Medium/Neutral’. Although many
participants agreed that there were no unnecessary
input or effort, some of them spent quite some time
in understanding the adaptation types provided and
defining the course structure before they could start
with the actual design of the adaptive 3D VLE. The
question “The defined Pedagogical Relationship
Types are difficult to understand” was rated ‘Poor’.
AUsabilityEvaluationofGraphicalModellingLanguagesforAuthoringAdaptive3DVirtualLearningEnvironments
463
Acceptability:
The different acceptability aspects scored in average
good (see Figure 9). 4 questions related to perceived
ease of use were rated as ‘Good’, while the other
questions (3 questions) were rated as “Neutral’.
Concerning the two questions related to attitude, 1
question was rated as ‘Good’ and the other was
rated as ‘Neutral’. Finally, the perceived usefulness
questions (2 questions) were rated as ‘Good’.
Given the fact that 11 participants (out of 14)
had only limited experience in authoring 3D VLE’s
or videogames, we rather expected a neutral rate to
the perceived ease of use and attitude. But the
scores were in average good. However, some
participants gave a neutral rate to the aspect
perceived usefulness.
Figure 9: Subjective Impression Questionnaire Feedback.
Qualitative Feedback:
The open questions were about three aspects:
appreciation, depreciation, and recommendations.
Appreciation
Most of the participants (11 out of 14) noted that
they liked the fact that there are three modelling
languages for creating the whole adaptive 3D VLE.
Further on, ease of use and consistency was
mentioned (7 times). Others (5) considered the
availability of the predefined adaptation strategies,
which could be applied to all 3D objects related to a
topic, as very useful.
Concerning the question “Was the Adaptive
Storyline Language expressive enough to specify the
overall storyline of the adaptive 3D course? (Please
describe why”), answers revealed that connecting
topics with adaptation rules helped to define the
overall flow of the storyline. In particular, 8
participants liked the adaptation rules between
topics and the fact that they could choose which
adaptation strategies to apply to the 3D objects
inside the topic.
Concerning the question “Was the Adaptive
Topic Language expressive enough to specify the
details of each topic? (Please describe why)”, 4
participants highly appreciated that they could
specify which adaptation types to applied to
different 3D objects. Furthermore, being able to
define when the adaptation type should be triggered
by means of a VR event helped them to obtain a
general overview of when the adaptations would
take place. In addition, 7 participants gave a credit
to the possibility of being able to give messages to
the learner.
Depreciation
In responding to the question “What did you like
least about the authoring approach in general and
its languages?” answers revealed some limitations
and flaws summarized into the following categories:
A need for software tool (3 times)
Using different adaptation types to the same
object was confusing (4 times).
The need to specify learning concepts for each
topic was confusing (1 time).
Distinction between the adaptive storyline and
adaptive topic (6 times).
The answers on the question “Was the Pedagogical
Model language expressive enough to specify the
pedagogical aspects for the adaptive 3D courses?
(Please describe why)” provided some explanation
why the question “The defined Pedagogical
Relationship Types are difficult to understand”
received the score ‘Poor’. For instance, 6
participants needed quite some time to understand
the meaning and the use of the Pedagogical
Relationship Types (PRTs). Others (7) found the
use of different colours for different PRTs
confusing.
Recommendations
In responding to the question “What should be
improved and how?” most of the answers were
related to the need for a supporting tool. Indeed, the
use of the modelling languages within an authoring
tool could support the authors with different help
mechanisms like tool tips and tutorials.
Furthermore, it would be easier to detect errors in
the models. Another issue related to a supporting
tool is the fact that a tool could ease the
specification/modification, e.g., by providing
menu’s.
Workload Perception:
Participants were requested to give feedback on the
mental demand, the effort required to accomplish
the task, and their frustration. Most of the answers
were neutral, but some frustrations were reported.
For instance, some participants were wondering
whether it is the author’s role to make sure that the
order of learning concepts in the Storyline Model
CSEDU2014-6thInternationalConferenceonComputerSupportedEducation
464
should be consistent with the Pedagogical Model.
This is indeed a fair question. However, providing
an authoring tool that checks the consistency of
both models can easily solve this. This is feasible as
authoring tools in the context of adaptive
hypermedia such as AHA! (De Bra, Smits and
Stash, 2006) and GRAPPLE (Hendrix et al., 2008)
already check adaptation rules.
5 DISCUSSION
Overall, the evaluation results of the graphical
languages were quite positive. However, it is
necessary to recall that all participants were
computer scientists; this could have influenced the
results. However, most of them did not have true
experience with developing 3D/VR application,
which corresponds with one of the main
characteristics of our target users. Furthermore,
conducting an empirical evaluation with a relative
small number of users (14 participants in our case)
may also affect the validity of the result of the
evaluation. However, this evaluation was a pilot
evaluation and performed in order to obtain a first
feedback.
Usability and acceptance results were good
despite the fact that most of the participants lacked
experience in authoring adaptive 3D VLEs and
there was no true learning period, while it is
obvious that some time is required to get acquainted
with the visual notations.
The effectiveness of our authoring approach
turned out to be good in this evaluation since all
participants were able to define the adaptive 3D
VLE in the right way. They could specify an
adaptive storyline and managed to specify
adaptation for the topics. Furthermore, the
decomposed specification of a topic adaptation rule,
into a VR event to trigger the rule, a condition that
needs to be satisfied, and the resulting action (the
adaptation type), made it easy for the participants to
keep an overview on the adaptations.
In addition, the qualitative feedback provided
useful information for further work. As expected,
tool support is essential. But also some specific
requirements related to tool support were given,
such as pull down menu’s to select the User Model
and 3D VLE activity history attributes when
(re)defining the update rules in the pedagogical
model, as well as when defining the adaptation rules
in the adaptive topic model. Interesting to note it
that in the evaluation, the 3D VLE activity history
attributes and the User Model attributes were given
as one list, although conceptually there are
separated in our approach. We thought one list
would be simpler for the author, as both categories
of attributes may be needed in the context of
defining the adaptive topic model. However
feedback indicated that it would be better to keep
this conceptual difference and to present them as
two separate lists. Furthermore, the use of different
colours for different Pedagogical Relationship
Types surprisingly turned out to be confusing and it
was advised to remove this or leave it up to the
author to define when different colours should be
used.
6 CONCLUSIONS
We presented and discussed a usability evaluation
of graphical modelling languages developed to
support 3D-novice educators in the process of
specifying (i.e. authoring) adaptive 3D VLEs.
The evaluation was done with 14 people from
the domain of Computer Science. After an
introduction to the approach, they performed an
authoring task. Next, they filled in a questionnaire
consisting of closed, as well as open questions. The
results indicate that the modelling languages
proposed are intuitive and can be used by people
without deep knowledge of 3D/VR to perform the
authoring process within a fair period of time.
Moreover, the participants found the visual
notations easy to use. Not surprisingly, the
evaluation revealed the need for software support.
We acknowledge that the evaluation has some
limitations, the most important ones being: the fact
that the participants were computer scientists and
the limited amount of participants. Also the fact that
the authoring exercise was done with pen and paper
can be a limitation. On the other hand, it avoided
that the tool was evaluated rather than the
languages. In order to fully evaluate our approach,
additional evaluations should be conducted when a
(functional prototype of an) authoring tool has been
developed with a larger number of people including
people with different backgrounds, like experts in
VR for validating the advanced features as well as
non-technical people, people with and without
modelling experience, and people with different
teaching experience. It may also be important to
measure the required time for completing the tasks
by the different categories of users. If the time
required to author a course is too long, people may
not be prepared to use it in practise.
AUsabilityEvaluationofGraphicalModellingLanguagesforAuthoringAdaptive3DVirtualLearningEnvironments
465
REFERENCES
Boshernitsan, M., & Downes, M. (2004). Visual
programming languages: A survey. Control.
Computer Science Division, Univ. of California.
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989).
User Acceptance of Computer Technology: A
Comparison of Two Theoretical Models.
Management Science, 35(8), 982–1003.
De Bra, P., Smits, D., Stash, N., (2006). Creating and
Delivering Adaptive Courses with AHA!,
Proceedings of the first European Conference on
Technology Enhanced Learning, EC-TEL 2006,
Springer LNCS 4227, pp. 21-33.
De Bra, P., Smits, D., Van Der Sluijs, K., Cristea, A. &
Hendrix, M. (2010). GRAPPLE: Personalization and
Adaptation in Learning Management Systems. In:
Proc. of World Conference on Educational
Multimedia, Hypermedia and Telecommunications.
Association for the Advancement of Computing in
Education (AACE), pp. 3029–3038.
De Troyer, O., Kleinermann, F., Pellens, B. & Ewais, A.
(2009). Supporting virtual reality in an adaptive web-
based learning environment. In: Proc. of the 4th
European Conf. on Technology Enhanced Learning
Learning in the Synergy of Multiple Disciplines.
Nice, France: Springer-Verlag, pp. 627–632.
De Troyer, O., Kleinermann, F. & Ewais, A. (2010).
Enhancing Virtual Reality Learning Environments
with Adaptivity : Lessons Learned. In: The 6th int'l
conf. on HCI in work and learning, life and leisure:
workgroup human-computer interaction and usability
engineering. Klagenfurt, Austria: Springer-Verlag, pp.
244–265.
Deursen, A., Klint, P., & Visser, J. (2000). Domain-
specific languages: An annotated bibliography. ACM
Sigplan Notices, 35(6), 26–36.
Ewais, A. & De Troyer, O. (2013). Authoring Storyline-
based Adaptive 3D Virtual Learning Environments.
In: Proc. of the 5th Int'l Conf. on Computer Supported
Education (CSEDU 2013). Aachen, Germany.
Hart, S., & Staveland, L. (1988). Development of NASA-
TLX (Task Load Index): Results of empirical and
theoretical research. Human mental workload, 1–46.
Hendrix, M., De Bra, P., Pechenizkiy, M., Smits, D. &
Cristea, A. (2008). Defining Adaptation in a Generic
Multi Layer Model: CAM: The GRAPPLE
Conceptual Adaptation Model P. Dillenbourg & M.
Specht (eds.). Times of Convergence Technologies
Across Learning Contexts. pp. 132–143.
Lazar, J., Feng, J. H., & Hochheiser, H. (2010). Research
Methods in Human-Computer Interaction. Wiley
Publishing, Inc.
Marchiori, E. J., Del Blanco, Á., Torrente, J., Martinez-
Ortiz, I., & Fernández-Manjón, B. (2011). A visual
language for the creation of narrative educational
games. Journal of Visual Languages Computing
Computing, 22(6), 443–452.
Moody, D. (2009). The “Physics” of Notations: Toward a
Scientific Basis for Constructing Visual Notations in
Software Engineering. IEEE Transactions on
Software Engineering, 35(6), 756–779.
Overmars, M. (2004). Teaching Computer Science
through Game Design. Computer, 37(4), 81–83.
Prumper, J. (1999). Test It: ISONORM 9241/10. In
Human-Computer Interaction? Communication,
Cooperation, and Application Design (pp. 1028–
1032). Mahwah, NJ: Lawrence Erlbaum Associates.
CSEDU2014-6thInternationalConferenceonComputerSupportedEducation
466
