TOOL-SUPPORTED ASSESSMENT OF WIKI-BASED

ASSIGNMENTS

Zuzana Kubincov

, Martin Homola

1,2

and Roman Janajev

Faculty of Mathematics, Physics and Informatics, Comenius University, Mlynsk

a dolina, 842 48 Bratislava, Slovakia

FBK-IRST, Via Sommarive 18, 38050 Povo, Trento, Italy

Keywords:

Wiki, Education, Assessment, Evaluation.

Abstract:

Integrating wiki-based activities into education encourages development of students’ competencies that are

important for their future professional life. As wikis support social learning and interaction, when working

with them students learn to cooperate, plan and organize collaborative tasks, create concepts, express ideas,

etc. Many teachers recognize undeniable beneﬁts of a wiki in learning and try to involve it in their teaching.

Since wikis were developed for collaborative work but not for use in education, they typically do not include

proper tools for tracking and assessing the students’ activities, which makes the evaluation difﬁcult for the

teacher. In this paper we present a tracking and assessment tool, which we have proposed and developed

and report on our experience with evaluation of wiki-based assignments using this tool as well. Since the

evaluation of wiki-based assignments is a nontrivial problem not only from the technical point of view but

also from the point of methodology, the assessment methodology is also discussed here.

1 INTRODUCTION

One of the most important competencies which our

students will need in their future professions is the

ability to collaborate during problem solving. Un-

fortunately there is only a limited number of courses

offered at our faculty where teamwork is exercised

and students work on projects in groups. The major-

ity of courses relies strictly on individual assignments

and hence the students are not trained in collaboration

well. In order to improve this status we have reached

toward wikis, and integrated wiki-based assignments

into multiple courses.

Wikis were developed as a tool for collaborative

creation and sharing of Web documents. This tech-

nology provides users with the possibility of publish-

ing documents (called wiki pages) on the Web, which

can then be accessed by other Web users who are able

to read but also revise these documents. New pages

can also be added and existing pages can be removed

if considered useful by any of the users. Hence wiki

pages are typically an outcome of multi-author col-

laboration. In extreme cases, such as some popular

Wikipedia pages, there can be hundreds of coauthors

contributing to a single wiki page.

Apart from being inherently collaborative, wikis

are easy to use and excessive knowledge of Web tech-

nologies is not required for users to participate. In

order to manage the collaborative activity, they of-

fer special tools such as revision history tracking and

comparison that greatly support the collaborative ef-

fort.

Applications of wikis in the educational process

help to develop basic skills such as technical and cre-

ative writing, working with external sources, and of

course language skills (Neumann and Hood, 2009;

Rick and Guzdial, 2006; de Paiva Franco, 2008). In

addition to these basic skills, the collaborative na-

ture of wikis reinforces social learning (i.e., orga-

nization of knowledge in the form that is suitable

and understandable by other members of the com-

munity), and helps to develop some more advanced

skills such as cooperation, organization and planning

of the collaborative tasks, but also communication

skills, critical and analytical thinking, and the ability

to express one’s ideas clearly (Ruth and Houghton,

2009; Popescu, 2010). As mentioned in several stud-

ies (Rick and Guzdial, 2006; de Paiva Franco, 2008;

Ruth and Houghton, 2009) using this tool stimulates

community building, students’ active participation in

learning and responsibility of students for their own

learning as well as for one another’s learning.

While there is a number of apparent advantages

that clearly motivate applications of wikis in educa-

Kubincová Z., Homola M. and Janajev R..

TOOL-SUPPORTED ASSESSMENT OF WIKI-BASED ASSIGNMENTS.

DOI: 10.5220/0003921900580067

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 58-67

ISBN: 978-989-8565-07-5

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

tion, and there is a body of literature that supports

this approach, it is not completely clear from this lit-

erature how exactly a wiki should be applied and how

students’ interaction with it should be assessed. In

this respect, the most powerful and useful aspect of

wikis – their collaborative nature – quickly becomes

the most problematic one. It shows that students

are not accustomed to collaborative tasks very well,

they need excessive motivation and guidance. Stu-

dents should be continually involved with the wiki,

which requires periodical involvement of the teacher

as well. Finally, even if most wikis provide basic

activity tracking tools, the collaborative activity car-

ried out by a group of students over longer periods of

time breaks down into an increased number of rather

heterogeneous revisions which are difﬁcult to track

and evaluate from the didactic point of view. Hence

suitable assessment methodologies remain largely an

open issue.

In our approximately four years experience with

wikis we run into many of these problems and were

forced to tackle them. In order to get a better overview

of students’ activity we have designed and developed

a special activity tracking tool that enables to organize

students but also wiki pages related to the course into

groups and allows the teacher to ﬁlter out the group

activity within any given time frame. To keep track on

students’ fulﬁllment of the task, their revisions can be

graded with evaluation points of multiple types from

which overall evaluation can be computed. Statistical

summaries and visualization of the activity and results

are available as well. Our tool is freely available for

download and reuse.

In addition our experience forced us to seek an-

swers also for many of the methodological issues.

Even if our conclusion in this respect may not be un-

equivocal and universal, the lack of research reports

on this problem encourages us to share them with the

community. We believe that they can provide useful

insight for other seeking to integrate this useful tool

into their curricula.

In the following section we describe wikis and

their uses in education. Then our attention turns to-

ward evaluation of students’ activity, the basic activ-

ity tracking tools available in wikis and the function-

ality that is missing. In Section 3 we concentrate on

methodological issues. Then in Section 4 we describe

the tool that we have developed and its use. Discus-

sion on related work and ﬁnal conclusions follow.

2 WIKIS IN EDUCATION

There are many ways how wiki can be used in the ed-

ucational process as documented in numerous stud-

ies (Schwartz et al., 2003; Gobbo and Lanzarone,

2006; Juan Ramon Prez Prez, 2006; Neumann and

Hood, 2009; IT User Services, 2008; Popescu, 2010).

Among the most interesting we ﬁnd:

Collaborative Study Materials. In case of newly

created courses or courses in quickly develop-

ing areas and topics, a wiki based collaborative

text book with active collaboration from the side

of students may be useful. Collaborative lecture

notes or annotations written by students may be

used even for well established courses. The infor-

mation in the wiki textbook or lecture notes is up

to date and the students’ output provides valuable

feedback to the lecturer.

Collaborative Report. In the assignments which re-

quire a report on the results (e.g., experiments,

data analysis, etc.) students can communicate the

results of their previous work and collaboratively

write the report using wiki.

Creating a Library. In several courses it may be

useful to create a reference library of topics and

resources related to the course (e.g., a collection

of algorithmic problems and their various solu-

tions). A slightly modiﬁed perspective is to use

wiki as host-environment to collect and share re-

sources of various kinds such as papers, images,

audio and video ﬁles, external links, etc., which

can be uploaded, shared, annotated and discussed.

Project Diary. A group of students working on a

team project assignment may use wiki as a com-

mon space where they record the project goals,

their progress, schedule, and other project related

issues.

Reciprocal Correcting of Essays. In the assign-

ments requiring to write an essay, the essays

can be uploaded into a wiki and then students

may be asked to review the essays written by

their classmates, check and correct the grammar,

typos, sentence structures, etc.

Besides the beneﬁts mentioned in the previous

section, integrating wiki into teaching brings many

other advantages. As it follows from recent compar-

ative studies (Rick and Guzdial, 2006) student per-

formance can improve measurably after adapting cur-

riculum to this new tool, however the success is prob-

ably related to the culture and discipline in the class-

room as well as the subject of study. Another stud-

ies showed higher students’ involvement (Neumann

and Hood, 2009) and acquiring signiﬁcant skills in

self directed learning (Ruth and Houghton, 2009)

when using wiki in higher education. According to

(Mikeor, 2011) using wiki in education can help stu-

dents to build greater relations between new and old

TOOL-SUPPORTEDASSESSMENTOFWIKI-BASEDASSIGNMENTS

knowledge by allowing student-created structure for

the information and ideas. It also stimulates discus-

sion and meta-cognition, different conceptualization

of the same content, develops creativity and writing

skills in students, etc. Monitoring of wiki discus-

sions by teacher can help to determine the areas or

topics of the course that are problematic for students.

Also the development of interpersonal and communi-

cation skills, especially consensus-building and com-

promise, as well as teamwork skills we consider to be

very important beneﬁt of employing wiki into educa-

tion.

As we already outlined in the introduction, one of

the main reasons that hinders more widespread use of

wikis in education is the problem of evaluation of stu-

dents activity with a wiki. This evaluation is needed

and should be continual or at least repetitive in several

stages during the course in order to increase students

engagement with the tool.

When a teacher is to evaluate a contribution of

a particular student, she is confronted with a large

number of revisions – the actions that have been per-

formed – the students have carried out on a set of doc-

uments that are part of the assignment. These revi-

sions are typically presented as a long list and have to

be reviewed in detail because of their heterogeneous

nature (e.g., some added new content, some edited ex-

isting content, some were improvements, some were

corrections of typos and grammar, etc.).

A typical and probably the most popular example

of wiki software is MediaWiki

. It offers two basic

tools that can be very helpful to the teachers:

Revision History. Edits are stored in the database

and can be listed and reviewed. The user may

retrieve the history per each document, per par-

ticular user, or for all documents contained in the

wiki. Timestamps are available, so the teacher is

able to ﬁgure out the frequency and regularity of

edits. Each version can be reviewed and any ver-

sion can be restored if needed.

Version Comparison. This tool is integrated into re-

vision history. It shows two selected versions side

by side and visualizes the difference. This tool is

essential for evaluation which would otherwise be

nearly impossible.

The intended use of these tools is in tracking the

basic activity of a large number of often anonymous

contributors, identiﬁcation of abuse and vandalism,

etc. This is not to say that the tools are not useful

MediaWiki is a free software provided by the WikiMe-

dia foundation. It is used by the foundation’s projects such

as Wikipedia and many others. It can be obtained from:

http://www.mediawiki.org

to the teacher, they indeed are. However, their func-

tionality is only very basic in order to support the

task of student assessment which becomes difﬁcult

especially with larger groups of students or with la-

bor intensive assignments. For such tasks, more com-

prehensive and speciﬁcally directed tools are needed.

From our experience we were able to identify the fol-

lowing functionality that is needed to support the as-

sessment task and it is currently lacking or not satis-

factory in most of the readily available wiki softwares:

User Groups: wiki is rarely deployed for one single

course. Typically a wiki is reused for multiple

courses and also for multiple runs of same courses

in consecutive years. Very often also students of

one course are divided into several groups each

working on a separate assignment or its version

as collaboration in excessive groups becomes in-

creasingly difﬁcult. It is handy if the teacher is

able to associate documents of interest with each

group.

Filters & Search: teachers need to be able to ﬁlter

out revisions of a particular set of documents,

user, or user group. Sometimes it is handy to

add or remove students/documents from the ﬁl-

ter. It is also necessary to be able to ﬁlter out

revisions in any given time frame. For instance,

MediaWiki offers only most recent revisions with

different windows the largest being last 500 revi-

sions in last 30 days. As typical course runs in a

semester spanning multiple months this is clearly

unsatisfactory.

Evaluation Records: while wiki may offer some ba-

sic revision metrics such as number of edits and

number of bytes changed in a revision, for a thor-

ough assessment of student’s activity one needs to

assign some kind of evaluation score to the revi-

sions. Evaluating a revision, teacher needs to take

to account not only the amount of text edited or

removed, but must be able to assess also the qual-

ity of the revision: is it addition a new content,

modiﬁcation of existing content, major error cor-

rection, or just grammar and typos correction? It

may be desirable to assign a different weight or

score to multiple quality categories for each revi-

sion.

Statistics & Visualization: a comprehensive assess-

ment supporting tools should provide statistics

and charts for analysis and visualization of both:

the revision metrics provided by the wiki software

and the evaluation points assigned by revisions by

the teacher. It should be able to spot the most ac-

tive students, the most frequent types of edits, the

relation between the time period and activity of

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

students, etc. It would be also very desirable to be

able to compare students and student groups.

In Section 5 we review a number of various tools

which extend the basic functionality of MediaWiki

with the aim of enhanced activity tracking. As we did

not ﬁnd any of these tools satisfactory for our needs

we decided to design and develop our own activity

tracking tool, which we describe in Section 4.

Another important problem is that even with dif-

ferent supporting tools, to our best knowledge, there

is no methodology for wiki-based assignments assess-

ment. We elaborate on this in the next section.

3 ASSESSMENT

METHODOLOGY

Evaluation of wiki-based assignments is a non-trivial

task even for an experienced teacher. It is difﬁcult

in general to evaluate the products of students’ col-

laborative work and in addition a wiki offers to stu-

dents many diverse ways how to contribute and hence

typically a combination of multiple evaluation criteria

need to be applied.

Before we get to the details, let us take a slightly

broader perspective on wiki-based assignments. In

the ﬁrst place, attention needs to be given to formulat-

ing the assignments properly. Vaguely or too gener-

ally formulated assignments, (e.g., “contribute to the

wiki with an article related to the topic of the course”

and providing a list of topics) should be avoided. Such

an assignment would most probably work ﬁne with

individual tasks, such as essay or blog article writing.

In case of wiki-based assignments most probably one

of the aims is also to exercise collaboration – this may

not be obvious to the students however. Therefore stu-

dents need further instructions and explanations how

much of collaboration is expected, how should it be

executed and coordinated.

It can be very handy to provide such instructions

in the form of separate howtos and guides, that can

reside in the very same wiki that will be used by the

students as separate wiki pages. This way they are ac-

cessible to the students and can be shared by multiple

assignments and even courses if applicable (in most

cases the organization of the work will be very sim-

ilar for different assignments). To some extent these

instruction should outline also some details on how

the assignment will be evaluated. For instance, we

typically include a “How to get points” section which

explains that certain amount of collaborative activi-

ties is needed to gain the full score, examples of such

activities, and a warning that just adding new content

will not be satisfactory.

One advantage of wikis is that students’ contribu-

tions, even partial, become immediately visible to the

teacher revision by revision. The teacher may thus su-

pervise and provide guidance if necessary. In collab-

orative assignments an increased amount of guidance

is needed. Teachers thus become actively involved

with their students, they continually provide feedback

and discussion. Wikis are very well suited for this

purpose. In MediaWiki, for instance, each page and

each user as well have a dedicated discussion page.

The teacher can immediately use these pages to de-

liver feedback, and students themselves should be en-

couraged to use discussion pages to cooperate their

contribution. In addition, in large wiki projects such

as Wikipedia, so called message box templates have

become a standard to indicate if there is some prob-

lem on a wiki page. These templates can be easily

adopted for didactic goals, and we encourage their

use especially by the teachers to deliver more targeted

feedback to the students. An example of such a mes-

sage box that we employ is shown in Figure 1.

Figure 1: Example of a message box.

Most typical didactic message boxes that we em-

ploy ask for correcting errors, adding clariﬁcations

and further explanations, reorganizing text, etc. Such

message boxes, once created, can be reused between

assignments and courses. While in most cases the

teacher adds the message box describing some prob-

lem when needed, and consecutively some of the stu-

dents would resolve the problem and remove the mes-

sage box (which the message box itself suggests), also

the students should be able add message boxes and

suggest some improvements to their colleagues if they

ﬁnd it useful.

Collaborative projects with number of participants

typically only yields meaningful results if executed

over a reasonable period of time. For most applica-

tions in higher education this can ideally match with

the overall length of the course. Minimal time span

for such assignments depends on the difﬁculty and

size of expected results and the size of the team. On

the other hand, our experience suggests that at least

4–5 weeks will be typically required in order to grasp

the assignment and wiki basics, to establish effective

collaboration, and to produce ﬁrst meaningful results.

This kind of assignments also requires that the

students participate continually and coordinate their

contribution with the others. Students must work as

TOOL-SUPPORTEDASSESSMENTOFWIKI-BASEDASSIGNMENTS

a team and their results will depend on other team

members. Our experience shows that for this rea-

son these assignments cannot be completely volun-

tary, and should somehow be part of the course as-

sessment. For instance, in order to get high grading

students need to collect a certain amount of evalu-

ation points through the wiki-based assignment. In

order to encourage continual involvement and team

work we suggest that the following aspects of their

work should be evaluated:

• quantity of contribution,

• quality of contribution,

• periodicity of contribution,

• amount of coordination with colleagues.

As students require feedback, the evaluation

should be provided periodically. Our experience

shows that one method to achieve a reasonable trade

off between quantity and periodicity is to split the as-

signment time into multiple intervals which are eval-

uated separately. Approximate amount of content

added or revised during one period is described to

the students and the maximum number of evalua-

tion points per period is set. Students are hence re-

quired to split their contribution between the periods

and immediately receive feedback. The periods them-

selves may be weekly, biweekly, monthly, or so. We

have achieved the best results with weekly evaluation,

however it certainly puts increased workload on the

teacher. Monthly periods seem already to be too long,

as students then tended to leave all work to the last

couple of days and decrease in cooperation activities

was observable.

The evaluation points awarded for each revision

will depend on the revision quantity (amount of added

or revised content), the quality of added content or

of the improvement, and collaboration activity. How-

ever a revision on a wiki page may be contributive in

many different ways, hence we suggest to distinguish

between multiple types of evaluation points respec-

tive to different evaluation criteria. These criteria are

probably best deﬁned by each teacher, however we

suggest criteria as follows:

New Content (n): respective to the amount of new

content was added.

Modifying Content (m): for signiﬁcant improve-

ment in the content.

Grammar/Editing (g): improving grammar, lan-

guage and structure of the existing content.

Topicality (t): a coefﬁcient (0-1) indicates whether

the content is on topic.

Cooperation (c): use of descriptive comments, dis-

cussion with other users, planning of new content,

etc.

The overall score (s) for each revision is then com-

puted by a suitable equation, for instance:

s := t × (n + m) +g + c , (1)

but certainly other equations may be applied, depend-

ing on the needs of particular teacher and particular

assignment. For instance, the topicality coefﬁcient t

may be applied to grammar and cooperation values as

well, or even more complex equations with weighted

sums of the score types may be handy.

Total score is then calculated as sum of totals for

each revision. Structuring the score into different

types is not only helpful in order to evaluate each re-

vision properly but may as well provide interesting

statistical summary of student’s activities over longer

periods of time. From the sums in the score types

(average in case of the coefﬁcient t) the teacher can

obtain interesting insights on overall students’ inter-

action with the wiki and their dominant types of ac-

tivities.

Another important aspect is the size of students’

groups that should participate on one wiki-based as-

signment. Clearly, certain minimal amount of stu-

dents is required to achieve effective team work. On

the other hand, if the group is too large, some students

may complain that there is too little space for them to

contribute, as everything what has to be done is done

already. Also unnecessary competition may arise be-

tween the students as everyone wants to get the best

(or easiest) share of evaluation points which degrades

team work as well. Therefore large groups of students

should be divided and each group should be given an

independent assignment. It is hard to estimate an ex-

act maximal size of a group, it largely depends on the

size of the assignment, especially on how broad is the

topic. Our experience shows that groups of twenty

and more students are already quite difﬁcult to man-

age.

Finally we would like to consider also the issues

related to copyright and intellectual property law. Al-

though our university implements strict sanctions for

ence shows that a number of undergraduate students

but not uncommonly master students as well are quite

ignorant in the copyright issues. We share the opin-

ion that students should be educated in these issues

in the ﬁrst place and this education must come dur-

ing the courses as well. Creative assignments such as

collaboration on wiki documents are well suited for

this purpose. We have achieved interesting results by

implementing small penalization into the course eval-

uation system (e.g., evaluation points are not awarded

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

Figure 2: List of revisions for speciﬁed group of users.

for the given evaluation period if part of the content

was reused without permission of the author, or even

a small amount of points may be subtracted). Such

rules of course need to be formally implemented in

the evaluation methodology and the students need to

be acknowledged about them at the beginning. The

main goal here is to rise awareness of the copyright is-

sues, as in most cases the problem is that students are

not aware of how information from external sources

should be processed and credited properly. Provid-

ing students with extensive guides and howtos on this

topic is extremely helpful.

4 ASSESSMENT TOOL FOR

MEDIAWIKI

In Section 5 we review a number of various WikiMe-

dia plugins and other tools that were created or can be

used with the aim of improving the activity tracking

task in the wiki. Unfortunately, we did not ﬁnd any

existing tool that would support the assessment task

appropriately as we explained in the previous sec-

tions. Therefore we designed and developed our own

tool (Janajev, 2011). It is called TrackingBundle and

it is freely available

as a MediaWiki extension.

The TrackingBundle extension for MediaWiki

is available from the WikiMedia extension matrix:

Once the extension is installed into a run-

ning MediaWiki it creates three new special

pages: TrackingBundle, AssignUserGroups, and

ActivityWatch. The ﬁrst page provides global con-

ﬁguration options, the second page serves for admin-

istration of user groups, and the third one contains the

tracking interface itself.

Let us have a closer look on the functionality sup-

ported by the extension. On the AssignUserGroups

page the teacher is able to create groups for tracking

and evaluation. Users can be added to each group, one

user may possibly belong to multiple groups. Users in

the group are split into ordinary members (students)

and leaders (teachers). This is handy when there are

more than one teacher responsible for a course. Group

leaders can add or remove members, and assign or

cancel the leader role to any member. Apart from

users the leaders can also assign speciﬁc pages of in-

terest to their group. Hence the group serves as a col-

lection of users and documents associated with a par-

ticular course, assignment, team of students, etc.

Once user groups are conﬁgured the activity is

tracked on the ActivityWatch page. This page is

split into two tabs: Ratings and Statistics. An

important part of this page is the ﬁltering interface

common to both of the tabs, with help of which the

teacher can easily ﬁnd relevant revisions in between

http://www.mediawiki.org/wiki/Extension:TrackingBundle.

TOOL-SUPPORTEDASSESSMENTOFWIKI-BASEDASSIGNMENTS

Figure 3: Chronological graph with the score overview table.

the large number of revisions done by many users of

the wiki (Fig. 2). The ﬁltering is enabled along three

different axes: ﬁrst, revisions carried out by a selected

set of users which can be loaded from a user group but

users can be also dynamically added or removed from

the ﬁlter; revisions done on a set of documents asso-

ciated with a group or a wiki category; and third, revi-

sions carried out within a speciﬁc window of time. All

ﬁlters can be combined according to teacher’s needs.

The Ratings tab lists all revisions that meet the

ﬁltering criteria in the chronological order (Fig. 2).

For each revision its author, revision time, revision

summary, and link to the revision itself (showing the

difference w.r.t. the previous version) is displayed.

Additionally, each revision can be rated in terms of

evaluation points that can be assigned on this page as

well. As we explained above, there are different as-

pects to be rated for each revision, hence the teacher

may assign evaluation points of multiple types. These

types and their relation to total score can be conﬁg-

ured on the TrackingBundle page; the three prede-

ﬁned types are new content, modified content,

and grammar/editing. A comment explaining the

evaluation may be added as well, so the student is able

to ﬁnd out the reason why she was assessed that way.

The Statistics tab provides a score overview

of all students showing the totals for each score type

and the grand total as well as total number of edits in

a summary table (Fig. 3). These results are also re-

spective the current settings, and so the teacher may

review and compare students’ activity and results dur-

ing selected periods of time or for selected documents

only.

The total score (s) can be computed in multiple

ways, which is also conﬁgured by the teacher on the

TrackingBundle page. The preconﬁgured options

include simple sum of points from different score

types, weighted sum of points, and a special prede-

ﬁned contribution score formula that was originally

introduced in the ContributionScores plug-in as

described in Section 5:

s := upe + 2 ×

(e − upe) (2)

In this formula, the basis of the score is formed

by the number of unique pages edited (upe). This is

increased by bonus score gained from total number of

revisions (e) decreased by the number of unique pages

edited. This way if someone did only one change per

page, she gets no bonus score. In the case of more

revisions per page a bonus score is assigned. In order

not to overweight the base score value with the bonus

score, the difference (e − upe) is square rooted and

multiplied by a constant.

The teacher may also deﬁne her own evaluation

formula consisting of regular mathematical expres-

sions and variables corresponding to the respective

score types.

Finally, the Statistics tab offers also three dif-

ferent ways of visualization of students’ activity and

score:

Chronological Graph: a line graph visualizing the

activity of the students within the time frame set

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

(a) (b)

Figure 4: Ratios of evaluation points (a) earned by different students; (b) for particular score types.

by the current ﬁlter. The value which is visualized

can be selected by clicking on the column in the

score overview table (Fig. 3);

Shares of People: a pie chart displaying ratio of

points earned by different students (Fig. 4 (a)).

Also for this graph the score type to be visualized

can be selected from the score overview table;

Shares of Points: a pie chart visualizing the ratio of

evaluation points for particular score types in all

watched revisions (Fig. 4 (b)).

As we can see one of the main design goals of the

tool was ﬂexibility. Partly due to the fact that there is

no uniﬁed methodology that would allow to evaluate

wiki-based assignments in one particular way. Also,

great ﬂexibility of ﬁltering is needed due to the large

number of revisions that students can possibly gen-

erate during one course. Powerful visualization tools

may also be very helpful to tackle this task.

In Section 6 we share some of our experience with

wiki-based assignments and with their evaluation us-

ing this tool.

5 RELATED WORK

Several plug-ins supporting evaluation in wiki were

developed and are available as extensions stored

on ofﬁcial MediaWiki extension matrix (Mediawiki,

2011). Some of them are purely cosmetic extensions,

e.g., TransformChanges, which transforms the de-

fault Recent Changes extension from a list of re-

visions into a table. But there are also extensions

trying to visualize data from wiki databaze in other

than standard way with the aim to acquire informa-

tion about revisions more effectively.

Most of them offer the list of users who con-

tributed to the wiki and sort them by the amount

of their revisions. The difference among the par-

ticular plug-ins is that some of them (e.g., the

plug-in Contributors) list the contributors of a

speciﬁc wiki page, while others (e.g., the plug-in

SpecialUserScore) gather statistics about revisions

from all the pages on the whole wiki or revisions re-

lated to a particular user (e.g., CountEdits).

One of the similar but more complex tools –

ContributionScores – not only lists the contribu-

tions from the last 30 days according to some ﬁlter

but offers also the possibility to calculate the evalua-

tion score using the formula (2).

There are also plug-ins in MediaWiki focused on

revision activities of a speciﬁed user. For example

the extension EditCount lists all pages edited and for

every page it shows the number and the percentage

share of revisions done by a selected user.

Since these extensions use the number of revisions

as the main measure of the user’s contribution to the

wiki, we do not ﬁnd them satisfactory for our pur-

poses. The number of revisions does not reﬂect the

substantial factors of contribution, such as the amount

of the text added and, more importantly, its quality, so

this information could be misleading. Also the statis-

tics information could give only the evidence of who

is active with the wiki in general but it is not helpful

in evaluating of the contributions related to a speciﬁc

course. All the above mentioned plug-ins are insufﬁ-

cient also for other reasons. They mostly do not offer

the opportunity to set the time period, create groups

of students or groups of pages to be monitored, they

do not allow to set the roles (teacher/student) or deﬁne

teacher’s own criteria and evaluation rules.

As one of the most inspiring MediaWiki exten-

sions we consider Annotation. This tool tries to

TOOL-SUPPORTEDASSESSMENTOFWIKI-BASEDASSIGNMENTS

interactively visualize which parts of a single page

have been edited by which user. Different colors are

used to highlight parts of the page edited by differ-

ent users. When hovering over a highlighted area,

detailed changes respective to this revision are vi-

sualized. Unfortunately this helpful and interesting

project was never ﬁnished.

Assessment extensions of DokuWiki (other pop-

ular Wiki type) were also reported. One of them,

EdDokuWiki (Popescu and Manafu, 2011) tracks stu-

dents activities, enables teacher and also peer evalua-

tion and allows to use various assessment criteria.

Independent from the MediaWiki and DokuWiki

projects, the ClassroomWiki system developed at the

University of Nebraska-Lincoln (Khandaker and Soh,

2010), offers an environment in which students work

collaboratively on wiki-based assignments. Based

on the set of pedagogy theories explaining collab-

orative learning, the authors implemented the tool

with tracking, modelling and group formation capa-

bilities. The system offers a mechanism for dynamic

creation of heterogeneous student groups, which has

been showed to improve the collaborative learning

outcomes (Roberts and McInnerney, 2007). This tool

offers the roles of teachers and students with differ-

ent rights. Teachers are provided with an interface

to create assignments for speciﬁc student groups and

afterwards to review the revisions made by students

and assess them. On the other hand, the dedicated

students interface is similar to an ordinary wiki, en-

riched by forums, where they can discuss their contri-

butions. The ClassroomWiki seems to be a powerful

tool considerably improving the collaborative learn-

ing skills of students. We found many of its features

useful and inspiring also for the development of our

own tool and methodology.

6 CONCLUSIONS

Wiki as one of the educational tools encouraging

the constructivist approach to knowledge acquisi-

tion (Heafner and Friedman, 2008; Parker and Chao,

2007) brings many beneﬁts which we ﬁnd important

in modern teaching and learning. Moreover, it helps

students to develop skills that are substantial not only

for their learning but also for their future professional

life, e.g., teamwork, conceptualization, communica-

tion skills etc.

For that reason we started to use this tool in our

courses more than three years ago. Various types

of wiki-based assignments were gradually integrated

into several courses in different study programs.

One of the ﬁrst courses where wiki was employed

was the master course of Computational Logic. Since

this course is a new element of the study programs,

there is a lack of study materials for it. Therefore the

main wiki-based assignment for students was collab-

orative writing of lecture notes.

Other courses (e.g., Algorithms and data struc-

tures for the bachelor students of Applied informatics

or Principles of databases in the master program for

future teachers of informatics) use wiki for creating

additional materials related to various course topics.

As the aims of these two approaches of integrat-

ing wiki into education are different, the ways how to

include wiki-based assignments into course and also

how to evaluate them differ as well. In the case of

preparing lecture notes the assignment was compul-

sory, students’ contributions were evaluated weekly,

so that the students received feedback almost imme-

diately. This method brings strong motivation for the

students to work continuously and periodically.

Creating of additional materials is an optional as-

signment and it is awarded some extra points. We

ﬁrst tried to employ this type of assignments in the

database course and the evaluation was carried out

at the end of the course, so the students had all the

semester to work on them. This approach did not

work properly, because the students were not moti-

vated enough to contribute regularly. Most of them

started to publish their contributions in the last few

days of the semester and that way one of the main

aims of these assignments – collaborative work – was

impossible to accomplish. For this reason, when we

consecutively applied this optional type of assign-

ments in the course on algorithms, the evaluation

was adjusted: the wiki contributions were evaluated

monthly and per every month a maximal amount of

evaluating points which could be assigned to individ-

ual student was speciﬁed. This helped partially but

we were not fully satisﬁed as only about 13 % of the

total number of students contributed to the wiki.

However, the evaluation of students’ contributions

was difﬁcult task in all these cases. The problems

related to the students’ activity tracking and assess-

ment served us as a motivation to develop the tool

introduced in one of the previous sections. Subse-

quently, this tool was used in assessing some runs of

the courses mentioned above.

Considering the small number of students con-

tributing to the optional assignments and rather low

level of their cooperation we used this tool with three

predeﬁned types of evaluation criteria: new content,

modified content, and grammar/editing. The to-

tal score was counted as a weighted sum of evalua-

tion points, as described in Section 4. Although in

this case when wiki pages created by the small group

of students were evaluated, the assessment tool was a

useful aid because the number of revisions evaluated

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

at once was quite large.

As regards the compulsory assignments where

the number of contributing students was relatively

large and students were forced to cooperate, we

added two new evaluating criteria (topicality and

cooperation) to the three criteria mentioned above.

The overall score was calculated according to the for-

mula (1). While evaluating these contributions we

ran up against the problem of large number of re-

visions with not too signiﬁcant changes, e.g., minor

spelling corrections or format changes, that needed to

be checked one by one.

This problem and also our ﬁrst experience with

the tool inspired us to the proposal of several minor

or major improvements. One of the most needed im-

provements is grouping multiple revisions of the same

user as one revision. This could avoid checking long

lists of small revisions without signiﬁcant changes

one by one. In addition, an improved visualisation

of changes made by individual users on a single wiki

page in a fashion similar to the Annotation plug-in

described in Section 5 would be very helpful. Another

interesting feature would be some kind of originality

check that would enable marking of suspected parts of

students’ contributions and searching for excessively