![](bg6.png)
crawl a mirror of the Wiki to create the topic map.
We also used the Administrator wizards to put in
hierarchy diagrams, double cell diagrams and
comparison matrices to add some domain-dependent
relations and keywords that the Crawler did not pick
up in the pages.
Table 1: The experiment tasks.
Give an example of how credit card service can use the
Strategy pattern.
e
a
s
y
Consider a chat room application. What is the subject and
what are the observers?
What is the class of design pattern that Decorator belongs
to? What is the class of design pattern that Memento
belongs to?
List two patterns that promote decoupling by using a central
object for communication.
Consider a pager service. Why is the façade pattern
inadequate for this scenario? Which pattern would be more
suitable?
d
i
f
f
i
c
u
l
t
You would like to make a deposit to your personal accounts
at the bank. The bank has a group of tellers; you are called
to the teller that can handle deposit to personal accounts,
who will process your request. What pattern does this
resemble?
After giving them a short tutorial on Wiki and
ENWiC, the subjects answered a total of six
questions: three using ENWiC and three using the
Wiki. Each set of three contained one easy question
that involves only one design pattern and two
questions that required subjects to combine
information from different patterns. The subjects
have a maximum of 15 minutes to answer each
question.
During the experiment, we observed the subjects
and we recorded the time to completion, the number
of unique nodes/pages visited, the number of mouse
clicks on ENWiC nodes and the number of link
traversals and back-button clicks on Wiki. After they
completed their tasks, we asked our subjects to fill
out a questionnaire about the usability of the two
applications, and their overall impression and
experience with the two applications.
Finally, we evaluated the correctness and the
completeness of their answers.
5.2 Results and Observations
We found that time-to-completion was not an
adequate measurement since it depended more on
the subject than the condition. Generally, completion
times were shorter with ENWiC, but some subjects
took more time than others regardless of the
conditions. Also, subjects took less time to answer
later questions since they remembered information
from the first few questions and had a better idea of
where to look. On average, completion time for a
task in ENWiC is 3:17 minutes, while in Wiki it is
3:31 minutes. Since ENWiC’s loading time per page
is quite longer than Wiki, the completion times for
ENWiC could potentially be even better.
For a better measure, we examined the number
of unique pages/nodes visited and the number of
clicks. As shown in Table 2, we noticed our subjects
used significantly less number of clicks to complete
the hard questions when using ENWiC. This was not
significant with the easy questions where the
answers were on a single page, but on the hard
questions, subjects were able to move to desired
pages just by using the graph, as opposed to having
to click on the Back button many times while using
a traditional browser.
Table 2: The number of clicks and unique pages/nodes
visited for each subject (A-F) for each question (1-6).
1 2 3 4 5 6
A
5,E:3,2 2,W:5,5 1,W:8,6 4,E:1,1 6,E:7,3 3,W:43,26
C
2,W:4,2 5,E:3,3 4,E:5,3 1,W:40,14 3,W:33,17 6,E:3,2
E
2,W:2,2 5,E:3,3 4,E:4,4 1,W:24,7 3,W:9,4 6,E:2,2
B
5,W:4,3 2,E:1,1 1,E:8,7 4,W:19,16 6,W:29,18 3,E:6,6
D
2,E:3,3 5,W:2,2 4,W:10,6 1,E:3,3 3,E:3,3 6,W:9,7
F
2,E:4,3 5,W:2,2
4,W:2,2
1,E:13,8 3,E:4,4 6,W:2,2
Each cell (formatted as O,W/E:C,P) reports the order
in which the question was asked (O), the tool in
which it was answered (E for ENWiC and W for
Wiki), the number of clicks involved (C) and the
number of unique nodes/pages (P) visited. The first
three subjects in the table used Wiki first and then
switched to ENWiC, while the other three subjects
used ENWiC first before using Wiki. The crossed-
out values indicated that the subject was timed-out
before he could finish the task.
The numbers for the easy questions (1 & 2) were
similar for both tools, but for the harder questions, it
can be seen that the number of pages/nodes the
subjects used in Wiki is more than ENWiC except in
one case (outlined in bold), where that particular
subject knew which pages to look. In terms of the
number of clicks, the margin between ENWiC and
Wiki is even higher, indicating that the subjects went
through significantly less irrelevant information and
pages when using ENWiC.
According to our observations, reading times
were also reduced in some cases, especially when
the subjects had to seek out certain relationships, as
it was faster to look at the links on the graph than to
read the text to find the information. Also, subjects
using Wiki may overlook links to potentially better
pages; for example, when answering question (1)
from Table 1, the Wiki page that described Strategy
pattern contains a link to another page that has an
example of Strategy pattern. On ENWiC, that link
has a section node “Example” associated with it, and
thus ENWiC users easily recognized that link as a
ICEIS 2006 - HUMAN-COMPUTER INTERACTION
40