AN OPEN-ENDED ASSESSMENT PLATFORM

IN A HOSTED E-LEARNING PORTAL

Patrick Stalljohann

Center for Innovative Learning Technologies (CiL), RWTH Aachen University, Aachen, Germany

Keywords:

e-Assessment, Assignment submission, Open assessment, LMS, Portal technology, SharePoint, Hosting.

Abstract:

Assessment activities play a central role in university courses. Assessment for learning during a semester can

support students’ performance and motivation, but needs a lot of time and effort for organizational issues.

This paper presents a system which tries to combine generic applicability across different domains with the

support of highly specialized processes and with the knowledge of speciﬁc domains. Portal Technologies are

used to provide a university-wide hosting of assessment tools within a single architecture. The connection

to cloud-services facilitates the integration of open assessment activities into formal assessment processes. A

prototypical implementation demonstrates possible points of extension.

1 INTRODUCTION

Technology enhanced assessment is a topic for many

years. Several systems have been developed to sup-

port either a speciﬁc scenario or common tasks gen-

erally. None of these systems seems to be well suited

as a hosted platform for a university-wide service.

With the aim of providing such a platform, investi-

gations about adaptability for special scenarios, ex-

tensibility for future forms of assessment, and the in-

tegration of external services are needed. The paper

at hand describes a concept of modular service man-

agers, which allows evolutionary enhancement of in-

tegrated assessment platforms. A prototypical imple-

mentation demonstrates the modular integration of a

cloud service to an assignment submission tool.

The paper proceeds as follows: Fundamentals of

assessment and arising problems of current assess-

ment systems are discussed in section 2. In section 3,

an overall concept with loosely coupled components,

event-based binding and integration of cloud services

is presented. Section 4 describes a prototypical im-

plementation based on a portal technology within a

university-wide hosted e-learning platform. In section

5, an overview of related work concerning extensible

assessment platforms is given.

2 MOTIVATION

Assessment takes a central place in higher education

teaching. On the one hand, ﬁnal exams are used

to evaluate students’ performances after a period of

learning. This is called ”assessment of learning”. On

the other hand, ”Assessment for learning” improves

students’ performance and their motivation (Bostock,

2004) during the learning process.

Mainly, three dimensions for the implementation

of assessment for learning can be distinguished: type

of activity, assessor, and formation of participants.

Examples for possible types of activities are assign-

ments, tests, or presentations. Beside a teacher, who

is the most traditional assessor, activities can be as-

sessed by peers (Peer Assessment), electronic sys-

tems (Automatic Assessment), or the learners them-

selves (Self-Assessment) (Race, 2001). The third

dimension contrasts the assessment of an individual

performance (Individual Assessment) with the assess-

ment of collaboratively performed activities within a

group of students (Group Assessment). However, col-

laborative assessment brings along some difﬁculties.

One difﬁculty to assess such group work is to evalu-

ate not only the product as a whole, but to take each

students’ individual participation or the collaboration

itself into account (Elliott, 2007).

392

Stalljohann P..

AN OPEN-ENDED ASSESSMENT PLATFORM IN A HOSTED E-LEARNING PORTAL.

DOI: 10.5220/0003474503920401

In Proceedings of the 3rd International Conference on Computer Supported Education (ATTeL-2011), pages 392-401

ISBN: 978-989-8425-50-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

2.1 Current Assessment Systems

A wide range of different systems have been devel-

oped to reduce the required time and effort which

comes along especially with Assessment for Learn-

ing. Furthermore, the enhancement of assessment

processes by technology can facilitate new forms of

assessment. Figure 1 shows a classiﬁcation of those

systems which can be divided into the categories

Electronic Test Systems, Organizational Assignment

Systems, Domain-Speciﬁc Systems, and Learning

Management Systems.

Figure 1: Classiﬁcation of classic systems with assessment

support.

Electronic Test Systems like Hot Potatoes

provide

computer-based support for creation, management,

execution and statistical evaluation of closed ques-

tions. Beside the simple electronic form of single- and

multiple-choice tests, additional forms of interaction

(e.g. drag and drop) and the integration of multime-

dia content (e.g. videos) are possible. The interop-

erability of standard tests and questions is supported

by the IMS QTI

speciﬁcation. The main advantages

of such kinds of assessment are automatic output of

marks and feedback as well as the capability for do-

main independent usage (e.g. for driving licence tests

or exams in medical studies). Disadvantages are the

limitation to closed questions and static feedback.

Organizational Assignment Systems are more fo-

cused to support speciﬁc organizational steps of as-

signment submission. Nowadays web-based systems

http://hotpot.uvic.ca/ (01.02.2011)

http://www.imsproject.org/question/ (01.02.2011)

are developed to handle the whole submission process

(Jones et al., 2005). These systems emphasize only a

small selection of speciﬁc functionality. Example are

Arop”a (Hamer et al., 2007) and PeerPigeon (Millard

et al., 2008) which have been constructed to facilitate

peer review processes. Other systems are focused on

group assessment. The main advantage of those sys-

tems is precise assistance for particular scenarios. In

many cases, they are restricted to manual correction

and feedback generation.

Although open questions cannot be corrected au-

tomatically in general, many Domain-Speciﬁc Sys-

tems try to use domain-speciﬁc knowledge to (pre-

)correct open-ended assignments and provide in-

telligent feedback. Hollingsworth, for example.

used automatic graders for programming classes

(Hollingsworth, 1960). Later systems are often web-

based and provide different static and dynamic test

procedures for a speciﬁc programming language, e.g.

CourseMaster (Higgins et al., 2003), TRAKLA2

(Malmi and Korhonen, 2004), PASS (Choy et al.,

2007) and DUESIE (Hoffmann et al., 2008). Systems

specialized to other domains, like musical dictations

(Tremblay and Champagne, 2002), essays (Burstein

et al., 2004), or mathematical proofs (Gruttmann

et al., 2008), are available as well. The main advan-

tage of these systems is domain-speciﬁc knowledge

which increases automation possibilities for open

questions.

Learning Management Systems (LMS) are de-

signed to support a wide range of learning and teach-

ing processes. Systems like Moodle

or BlackBoard

combine tools for course planning, communication

(e.g. chats and discussion forums), and collaboration

(e.g. wiki pages). Modules for the deﬁnition and ex-

ecution of closed question like the above mentioned

Electronic Test Systems are often included. Basic

support of a standard assignment submission process

is often available as well. The main advantages of

LMS are the integration with other tools which are

not directly related to assessment and the generic ap-

plicability for almost any course.

Elliott calls the use of technology to imitate only

traditional assessment processes Assessment 1.5 (El-

liott, 2007). He recommends to extend and renew as-

sessment strategies along with the innovative possibil-

ities of Web 2.0 to reach Assessment 2.0, also called

Open Assessment. When students are active as Web

2.0 authors (Gray et al., 2010), they might proﬁt from

the more collaborative manner of user-generated con-

tent, the power of the crowd, openness, and network

effects of Web 2.0. Thus, new forms of assessments

http://moodle.org/ (01.02.2011)

http://www.blackboard.com/ (01.02.2011)

AN OPEN-ENDED ASSESSMENT PLATFORM IN A HOSTED E-LEARNING PORTAL

393

have to be developed, like the assessment of activity

in online discussions (Vonderwell et al., 2007), wiki

pages (Cubric, 2007; Lie et al., 2010), or blogs (Lee

and Allen, 2006). The main advantage of Open As-

sessment is a personalised and more authentic assess-

ment scenario utilizing online tools of students’ ev-

eryday lives.

2.2 Problems and Objectives

Systems of each above mentioned category conduce

a special need and have a right to exist. But the par-

allel usage of several systems to support the overall

assessment process of a course is problematic. It is

therefore desirable to have a central platform for the

support of various settings and the integration of ex-

ternal systems, which could be provided a consistent

tool for several courses. One single system could not

cover all current and upcoming new forms of assess-

ment, what caused the multiplicity of currently avail-

able systems. Therefore, extensibility for various pur-

poses is a critical factor. Especially, integration of

external services to allow for more open assessment

scenarios is an objective of the work presented. An-

other objective is a more comprehensive usage, rather

than the application for only some different scenarios

in a single course or some related ones. Thus, such a

platform has to be ready for a hosting scenario, e.g.

for a whole university or perhaps as a service for other

institutions.

Most LMS and some test systems provide an ex-

tensible infrastructure to include new modules such as

additional types of tests, and especially new forms of

interaction for closed questions. A more open system

should be prepared for extensions at all levels of the

assessment process.

One point of extension could be the integration of

available cloud services for content creation and pub-

lishing on the Internet, to facilitate Open Assessment

scenarios with common Web 2.0 tools. These ser-

vices are more suitable within the overall assessment

process than used separately. Furthermore, these ser-

vices could be enhanced with additional functionality

(e.g. peer review, grouping or marking) within a cen-

tral platform.

To increase the usability, security, and reliabil-

ity as well as ease the operation and maintenance of

such a platform, it should be constructed to run in

a Hosting Environment. Thus, it can be provided as

part of a university-wide platform. The difﬁculty is

to provide the best-ﬁtting service for most courses

and domains. For this purpose, the system should be

extremely adaptable without overstraining the users.

For instance, the extension for automatic correction

of source code could be useful in the programming

course but probably not in the course for English

poem writing.

An approach to reach these objectives for an open-

ended assessment platform will be presented in the

following sections.

3 CONCEPTUAL APPROACH

For the deﬁnition of an architecture for a new assess-

ment platform, related models or reference architec-

tures have to be taken into account. Evaluations of

some models show that they are either applicable to

a speciﬁc subset of assessment processes only or they

are too general to cover the speciﬁc requirements of

extensibility.

3.1 Reference Models

The Four Process Model (Almond et al., 2002) di-

vides assessment into the four processes Activity Se-

lection, Presentation, Response Processing, and Sum-

mary Scoring. It allows a ﬂexible combination of dif-

ferent tools for each of the four processes. This model

is focused on individual test scenarios. The modular-

ity is limited to the four processes and reuse or exten-

sion of given components are not part of the model.

The Educational Model (Joosten-ten Brinke et al.,

2007) describes a static model of the assessment

model in six parts: Assessment Design, Item Con-

struction, Assessment Construction, Assessment Run,

Response Rating and Decision Making. The static na-

ture of the model conﬂicts with idea of a highly ex-

tensible platform.

The e-Framework Reference Model for Assess-

ment (FREMA) (Wills et al., 2009) tries to describe

the whole assessment domain. An overview about

current dimensions of assessment is given using topic

maps. While the suggested service oriented archi-

tecture allows an easy replacement of different ser-

vices, the modularity is limited to the currently known

domains of assessment. The integration of emer-

gent technologies or ﬁne-grained extensions to exist-

ing functionalities are not covered by this reference

model.

Even though the above mentioned models are not

directly applicable, several of their process descrip-

tions can be adopted.

3.2 Evolutionary Extension Modules

Evolution of technology and services leads to the need

of evolutionarily extensional systems. Those systems

CSEDU 2011 - 3rd International Conference on Computer Supported Education

394

Figure 2: Open Ended Assessment Platform.

have to allow ﬂexible points of extension to avoid the

development of completely new systems with a single

advancement and several reimplementations of avail-

able functionality.

Figure 2 presents an architecture with modularity

of mainly three dimensions: assessment tools, orga-

nization modules, and basic components. The most

general forms of reuse and extensibility are achieved

by the integration of assessment tools as a whole, ei-

ther internal or external ones. For instance, an ex-

isting test tool can be integrated into the platform as

well as a new tool for the assessment of collaborative

activity in wiki pages.

Organization modules provide reusable function-

ality for a special purpose of a typical step in an

assessment process. For example, a grouping mod-

ule allows the arrangement of students, which is

part of the assessment design for group assessments.

Additional content editors (e.g. for mathematical

equations or music notation) can be used as part

of item construction, response submission, or feed-

back. The creation of marking rubrics is part of

the assessment design, whereas the marking with

rubrics is part of response rating and decision mak-

ing. (Semi-)Automatic feedback and peer review as

special kinds of correction could be integrated with

correctionﬂows, which are modular deﬁnitions of cor-

rection processes like within the eAixessor-system

(Altenbernd-Giani et al., 2008) with the use of a

workﬂow engine. These modules for organization

can reuse basic components like user management

or document management. Many available platforms

provide different kinds of those components, which

should be extensible as well. The integration of ex-

ternal cloud services is considered on all levels of the

platform, e.g. with use of a whole external system or

integrated with an organization module.

An architecture is needed to organize and integrate

functionality among modules (Baldwin and Clark,

2000), which are weakly connected by deﬁnition.

However, it is difﬁcult not to limit the extensibility

within the architecture. Therefore an approach for

the modular integration of services, which themselves

can handle modules in their speciﬁc area of function-

ality, has been developed.

In the generic approach, a service manager works

as a kind of broker between one or more service-

providers and several service-consumers (see ﬁgure

3). The main differences to service-orientation in gen-

eral are that the consumer does not call the service-

provider directly and that events can be forwarded to

a consumer. A service manager is used for the reg-

istration of providers and consumers and serves as a

facade for speciﬁc functionality a provider can of-

fer. This event-based approach aims to reduce the

dependencies between modules (here consumer and

provider). Providers can be added, removed, or ex-

changed without any change to the service manager

or a consumer.

Figure 4 shows an example of a video service

AN OPEN-ENDED ASSESSMENT PLATFORM IN A HOSTED E-LEARNING PORTAL

395

Figure 3: Event-based services for encapsulation of func-

tionality.

manager, which can be used by a module for the

extension of an assignment submission tool with

videos from different services. Providers for this ser-

vice could be an internal video storage as well as

cloud connectors for the integration of YouTube

MyVideo

, for instance. Other examples could be ser-

vices for grouping or assessed activities in general.

Figure 4: An example service for video integration from the

cloud.

An example for the use of services, basic com-

ponents and modules for the evolutionary develop-

ment of an assignment submission tool is illustrated

in ﬁgure 5. It is assumed that a basic assignment sub-

mission tool has been developed to handle the pub-

lication of assignments and the acceptance of indi-

vidually submitted responses. This tool serves as a

provider of an assignment service which is linked to

a more generic assessment service. Functions for get-

ting a list of assignments are provided by this service.

Events for reaching a publication date or a deadline

for submission can be forwarded to consumers. The

main idea is to enhance the basic tool with additional

functionality without modifying its basic source code.

Several modules can be created to build an extended

tool, which now works as a provider for the assign-

ment service as a whole. Advantages of this modular

approach are that several services, modules, and com-

ponents can be used with less dependency. For exam-

ple, the basic assessment tool and the grouping tool

http://www.youtube.com/ (01.02.2011)

http://www.myvideo.de/ (01.02.2011)

are not directly linked to each other, but the knowl-

edge of groups is needed for group assessment as well

as for peer assessment. Thus, corresponding exten-

sions are consumers of a group service, which can

be provided by the grouping tool. This tool can be

replaced by another tool without having to alter the

extensions. Another new requirement for the assign-

ment submission tool could be that students are orga-

nized in tutorials, in which only an associated tutor

is allowed to correct their submissions. Therefore, a

tutorial service, which is linked to the group service,

is used to provide an extension for tutorial manage-

ment with the partition of students into tutorials. Be-

cause the basic tool is built on the basic components

of a platform (compare ﬁgure 2), a Java editor ex-

tension can be integrated into the basic content man-

agement component what leads to integration to the

assignment submission tool as well. The integration

of external services and content from the cloud works

the same way. As shown in ﬁgure 4, an open video as-

sessment extension can be used to enable students to

include their online published videos in their assign-

ment submissions.

4 IMPLEMENTATION

Since 2007 the tailor-made, university-wide e-

learning portal L

P supports presence teaching and

is available for all students and teachers at RWTH

Aachen University as a central service

(Schroeder

et al., 2008). A so called virtual course room can be

set up to enhance a lecture with information, commu-

nication and collaboration functionality. Started with

270 virtual course rooms in summer term 2007, there

are about 2.200 lectures from eight different faculties

in winter term 2010/10 which are supported by a vir-

tual course room. Overall, the portal reached a num-

ber of about 11.000 virtual course rooms during the

last eight terms. Approximately 25.000 enrolments

are registered per term, what means that every user is

enrolled in six course rooms on average. Currently the

system is used by about 15.000 users per day. About

2.5 million pages and 750 GB of data are transferred

per week.

P has been developed based on Microsoft

SharePoint Server 2007, a portal technology which

contains several standard functionalities such as user

management, document management, content man-

agement, workﬂow management and search. Using

such a well established platform facilitates the usage

as a hosting service and increases security, reliabil-

http://www.elearning.rwth-aachen.de/ (01.02.2011)

CSEDU 2011 - 3rd International Conference on Computer Supported Education

396

Figure 5: Modular architecture for assignment submission with exemplary extensions and cloud-service integration.

ity, and maintainability of the system. Additionally,

it provides multiple deployment mechanisms to add

user deﬁned functionalities, which allow the develop-

ment of custom integrated applications. Among oth-

ers, these possibilities has been used to integrate spe-

ciﬁc assessment tools. A custom assignment submis-

sion system (Dyckhoff et al., 2008) has been devel-

oped, which has been improved stepwise with several

functionalities like the opportunity to provide direct

feedback (Stalljohann et al., 2009). The management

of students’ marking results and criteria for passing a

lecture is handled by a custom integrated grade book

(Stalljohann and Schroeder, 2010), which currently

runs in several pilot installations.

4.1 Portal Deployment Mechanisms

SharePoint employs a hierarchical architecture start-

ing with a whole farm, which contains several web

applications. Within these web applications there are

multiple sites, which build independent workspaces.

In L

P , a separate site is used for each virtual course

room. These sites are structured itself by a tree of

so called webs or rather sub webs. They can be used

to deﬁne different areas, e.g. for information, learn-

ing materials, literature management, or assessment.

The content and document management is handled

by lists, which provide an abstracted layer of SQL

tables, including mechanisms for versioning, check-

in/out, controls for the user interface, and others. Lists

can handle meta data and ﬁles as their items. Fields,

corresponding to columns in a SQL table, represent

options of an items property, like the name, data type,

rendering and others. To handle different types of

items within a list, they can have different content

types, which deﬁne the properties (ﬁelds) of an item.

The content types are organized with help of inheri-

tance. A list for literature management for example,

contains ﬁelds for title, author, pages, publisher, jour-

nal, and ISBN. A content type for an abstract publica-

tion contains the ﬁrst four ﬁelds. A content type for

an article inherits this ﬁelds and adds the journal ﬁeld.

Another content type for a book inherits from the ab-

stract content type as well and includes the ISBN ﬁeld

additionally. The authorization of users can be han-

dled in different ways, like with an active directory,

forms authentication with a custom database, or cus-

tom providers. The authorization is managed with the

user of groups, role deﬁnitions, which are assigned

to a whole site, a web, a list, or a single list item.

The combined presentation of information, actions,

and settings from several sources is realized by web-

parts

. They encapsulate speciﬁc data, logic, as well

as its form of presentation. A weather webpart for

example can be placed on every page within the sys-

tem beside other webparts. A connection to webparts

which handles a list of cities could facilitate a kind of

ﬁlter.

There are several points for extension already

available within SharePoint which allow for the devel-

opment of evolutionary extensions, which is intended

Names for webparts in other systems are portlets, gad-

gets, or widgets.

AN OPEN-ENDED ASSESSMENT PLATFORM IN A HOSTED E-LEARNING PORTAL

397

related to the concept in section 3. Custom ﬁeld types

allow the deﬁnition of new data types, their valida-

tion and presentation for usage as a ﬁeld in a list.

Fields, content types, and templates for lists can be

deﬁned declaratively. The deployment of these ele-

ments is managed with so called features, which can

be installed to a SharePoint farm and activated in a

speciﬁc scope. Possible scopes are the entire farm, a

web application, a site or a single web. Others el-

ements that can be deployed with a feature are for

example webparts, event receivers, workﬂow deﬁni-

tions, custom actions, or external content types. Web-

parts contain code for the presentation of information

of data and the execution of actions. Event receivers

are classes which can be bound to a speciﬁc compo-

nent, e.g. a list item or a web, to execute custom

logic, when an event was triggered. Workﬂows and

related workﬂow activities can be developed to spec-

ify a modular process, which can be executed for an

item. Another important aspect is the possibility to

attach custom actions to the user interface, by deﬁn-

ing the type (e.g. button or link), its style (image,

caption, ...), and the corresponding action (execution

of server code, client script or a link). External lists

or rather External content types are a mechanism to

integrate external systems by consuming a web ser-

vice, a database, or a custom connector to arbitrary

sources. This integration allows the presentation and

handling of external data just like data in an ordinary

list. All of these components and several more ones

allow the creation of modular extensions for the plat-

form, which can packaged as a solution for reuse in

different farms.

4.2 Developing Modular Tools

As mentioned above, those deployment techniques

have been used to build different custom applications

for assessment purposes on top of SharePoint. By us-

ing these mechanisms, the whole application can be

deployed to different farms. The use of features fa-

cilitates the (de-)activation of speciﬁc functionalities

within the application separately. Thus, the modules

are adaptable for different scenarios. The main prob-

lems with the ﬁrst implementations of the systems are

the reusability of functionalities for other tools and

the extensibility with new functionalities, which are

the result of frequently added requirements. For ex-

ample, the tool for assignment submission contains

a grouping functionality, which allows the dynamic

building of groups for cooperative assignment sub-

mission. Adding other strategies for grouping as well

as using these groups for other tools is not straight

forward.

A restructuring of these applications with the use

of service managers (see ﬁgure 3) provides a way to

solve these problems. Some aspects for the devel-

opment of a service manager architecture are already

available. The paradigm is similar to event receivers

in SharePoint, but these are limited to a ﬁxed set of

elements and events that are triggered (e.g. an item

was added to a list). The same strategy is now used

to extend additional services with related semantic

events, like an assignment service which triggers an

event when the deadline was reached or a student’s

submission was handed in. A new service manager

is build as a class which is connected to a feature.

With this, the functionality of the service manager

can be (de-)activated at a speciﬁc scope. A provider

as well as a consumer reference the service manager

project and register themselves with their fully quali-

ﬁed names. If the service of a provider is consumed

or an event is triggered, the service manager creates

an instance of the corresponding classes by reﬂection.

This way, new services, providers and consumers can

be deployed, installed, and activated in a SharePoint

farm modularly.

The shift to this new paradigm has been demon-

strated by a reimplementation of the assignment sub-

mission system on base of SharePoint 2010. Figure

6 illustrates the workspace for a submission from the

perspective of a teacher or tutor. The whole tool is

integrated as a separate sub web and can be reached

from the left navigation bar (entry assessment). The

main area is a combination of several webparts. By

default, there are webparts for metadata (e.g. the

student, the related assignment, and the submission

state), documents in the solution, correction docu-

ments created by a tutor, a list of textual feedback,

and the given marks. Additionally, an extension for

the integration of videos from YouTube places an ex-

tra webpart related to an external list for the student’s

videos on the page. New videos for this submission

can be directly added, deleted, and modiﬁed within

the platform. Using the same service on the YouTube

webpage is also possible, by tagging the videos as a

submission. Other extensions, such as providing a

list for sample solutions for each assignment or al-

lowing the building of groups, have been separated

into different modules as well. The main point here

is that these extensions are able to consume the as-

signment service. The grouping extension could for

example synchronize the composition of groups from

the grouping service. After the event that the dead-

line has been reached is triggered, the groups for this

assignment are ﬁxed within the submission tool.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

398

Figure 6: Using YouTube Videos in Assignment Submissions.

5 RELATED WORK

The Flexible Assignment System (FAST) tries to pro-

vide a ﬂexible support for different assessment sce-

narios by the use of collaboration scripts (Topcuoglu,

2006). Students can be assessed individually or in

groups and they can use an integrated workspace to

create their solutions collaboratively. The process

how their collaboration is managed and how they are

assessed, by a tutor or by peers, can be deﬁned with

the collaboration script. But this ﬂexibility is lim-

ited to this process description, such that additional

functionality like the integration of external services

or self-regulated grouping has to be added within the

basic system.

A more ﬂexible approach concerning domain-

speciﬁc assessment support is provided by EASy

(Gruttmann, 2010). Started as a system for the deﬁ-

nition, submission and (semi-)automatic correction of

mathematical proofs, it was extended to a platform to

host modules for different domains (see 7). The as-

sessment process is divided into item construction,

Figure 7: Platform and modules of EASy (based on

(Gruttmann, 2010)).

response construction, and correction and marking.

A new module has to provide an editor for each

of these steps. Modules for programming assign-

ments, mathematical proofs, veriﬁcation proofs, and

multiple-choice tests are already available. Disadvan-

tages of this system are the limitation to individual

assessment by tutors. Grouping, peer assessment, or

AN OPEN-ENDED ASSESSMENT PLATFORM IN A HOSTED E-LEARNING PORTAL

399

other forms of assessment are neither available nor

modularly integrable. The integration of collabora-

tive activities or open assessment is not covered ei-

ther. Furthermore, the two systems FAST and EASy

are standalone systems, which implement their own

user management. Hence, they are not directly inte-

grated into an e-learning system to support other pur-

poses than assessment as well.

More integrated approaches, which leverage the

functionality of content management systems (CMS),

are for example the systems iPal (Pinkwart et al.,

2005) and EduComponents (R

osner et al., 2007). iPal

extends the standard functionalities of the PostNuke

CMS with modules for building tutorials, as well

as handling assignments and lecture notes. These

provide general support for typical teaching scenar-

ios, but do not offer an additional extension model

above the standards of the CMS. The aim of EduCom-

ponents was to enhance the CMS Plone

with ad-

ditional components to support different learning

and assessment activities. It is a collection of the

tools ECLecture (general course information), EC-

Quiz (multiple-choice tests), ECAssignmentBox (as-

signment submission with tutor marking), and ECAu-

toAssignmentBox (assignment submission with auto-

matic marking). The latter allows extension by dif-

ferent so called back-ends, which encapsulate the

domain-speciﬁc logic for the correction of assignment

submissions, e.g. one back-end for Java source code

and another one for UML diagrams. The reuse of an

existing back-end for building a new one on top of it

is not intended.

6 CONCLUSIONS AND FUTURE

WORK

Assessment takes a big part in formal learning, espe-

cially in higher education. Several tools have been

developed for technology enhancement of special as-

sessment processes. These systems, standalone as

well as integrated to general e-learning platforms,

support only a subset of possible assessment pro-

cesses and it is obvious that they lag behind new

forms of assessment. While platforms like CMS and

portals use well established functionalities and de-

ployment techniques to be highly extensible, exten-

sions on base of these platforms do not or limitedly

offer modular architectures themselves.

The paper at hand describes a concept of modu-

lar service managers, which allow an evolutionary en-

http://www.pn-cms.de/ (01.02.2011)

http://plone.org/

hancement of an integrated assessment platform. First

parts of this concept have been proven by an imple-

mentation based on Microsoft SharePoint portal tech-

nology. The example of a stepwise enhancement of a

basic assignment submission tool shows how open as-

sessment processes, which integrates YouTube videos

with students’ assignment submissions, can be inte-

grated by an extension.

Future work includes pilot installations using the

new version of the system in different lectures and re-

lated evaluation. The development and evaluation of

other assessment tools, e.g. for assessment of collab-

orative learning in wiki pages or more informal as-

sessment settings connected to cloud services and so-

cial networks, are objectives as well. Furthermore,

the usability of the contained sub-tools and exten-

sions for general usage as well as with mobile devices

needs to be evaluated. While the platform is currently

hosted for multiple institutions of a single university,

multi tenancy support could allow a hosting service

for different universities. The approach of evolution-

ary module extension could perhaps be used for each

tenant differently.

REFERENCES

Almond, R., Steinberg, L., and Mislevy, R. (2002). Enhanc-

ing the design and delivery of assessment systems:

A four-process architecture. Journal of Technology,

Learning, and Assessment, 1(5):164.

Altenbernd-Giani, E., Schroeder, U., and Stalljohann, P.

(2008). eAixessor - A Modular Framework for Auto-

matic Assessment of Weekly Assignments in Higher

Education. In Uskov, V., editor, Proceedings of the

7th IASTED International Conference for Web Based-

Education (WBE 2008), pages 70–75. ACTA Press.

Baldwin, C. Y. and Clark, K. B. (2000). Design Rules: The

power of modularity. MIT Press, Cambridge, MA.

Bostock, S. (2004). Motivation and electronic assessment.

Effective learning and teaching in computing, pages

86–99.

Burstein, J., Chodorow, M., and Leacock, C. (2004). Auto-

mated essay evaluation: The Criterion online writing

service. AI Magazine, 25(3):27.

Choy, M., Lam, S., Poon, C. K., Poon, C., Wang, F. L.,

Wang, F., Yu, Y., Yu, Y. T., and Yuen, L. (2007). To-

wards Blended Learning of Computer Programming

Supported by an Automated System. In Fong, J. and

Wang, F. L., editors, Proceedings of Workshop on

Blended Learning’ 2007, pages 9–18. Pearson.

Cubric, M. (2007). Using wikis for summative and forma-

tive assessment. In Re-Engineering Assessment Prac-

tices (REAP) International Online Conference.

Dyckhoff, A., Rohde, P., and Stalljohann, P. (2008). An

integrated web-based exercise module. In Uskov, V.,

CSEDU 2011 - 3rd International Conference on Computer Supported Education

400

editor, Proceedings of the 11th IASTED International

Conference on Computers and Advanced Technology

in Education (CATE 2008).

Elliott, B. (2007). Modernising Assessment: the use of

Web 2.0 for formative and summative assessment.

In Khandia, F., editor, 11th CAA International Com-

puter Assisted Assessment Conference, pages 179–

188, Loughborough.

Gray, K., Thompson, C., Sheard, J., Clerehan, R., and

Hamilton, M. (2010). Students as Web 2 . 0 au-

thors : Implications for assessment design and con-

duct. Australasian Journal of Educational Technol-

ogy, 26(1):105–122.

Gruttmann, S. J. (2010). Formatives E-Assessment

in der Hochschullehre Computerunterst

utzte Lern-

fortschrittskontrollen im Informatikstudium. Disser-

tation, Westf

alische Wilhelms-Universit

at M

unster.

Gruttmann, S. J., Kuchen, H., and B

ohm, D. (2008). An

E-Assessment System for Mathematical Proofs. In

Proceedings of the 11th IASTED International Con-

ference on Computers and Advanced Technology in

Education (CATE 2008), volume 614, pages 090–120.

Hamer, J., Kell, C., and Spence, F. (2007). Peer Assessment

Using Arop

a. In Proceedings of the ninth Australasian

conference on Computing education, page 54. Aus-

tralian Computer Society, Inc.

Higgins, C., Hegazy, T., Symeonidis, P., and Tsintsifas, A.

(2003). The coursemarker cba system: Improvements

over ceilidh. Education and Information Technolo-

gies, 8(3):287–304.

Hoffmann, A., Quast, A., and Wiesm

uller, R. (2008).

Online-

Ubungssystem f

ur die Programmierausbil-

dung zur Einf

uhrung in die Informatik. In Seehusen,

S., Lucke, U., and Fischer, S., editors, DeLFI 2008:

Tagungsband der 6. e-Learning Fachtagung Infor-

matik, pages 173–184, Bonn. Gesellschaft f

ur Infor-

matik.

Hollingsworth, J. (1960). Automatic graders for program-

ming classes. Commun. ACM, 3:528–529.

Jones, D., Cranston, M., Behrens, S., and Jamieson, K.

(2005). What makes ICT implementation success-

ful: A case study of online assignment submission.

Open Learning and Distance Learning Association of

Australasia (ODLAA), University of South Australia,

pages 1–11.

Joosten-ten Brinke, D., Van Bruggen, J., Hermans, H.,

Burgers, J., Giesbers, B., Koper, R., and Latour, I.

(2007). Modeling assessment for re-use of traditional

and new types of assessment. Computers in Human

Behavior, 23(6):2721–2741.

Lee, J. and Allen, K. (2006). Edublogs as an online as-

sessment tool. In Proceedings of the 4th International

Conference on Multimedia and Information and Com-

munication Technologies in Education, pages 391–

395.

Lie, C., Dyckhoff, A. L., and Stalljohann, P. (2010). Sup-

porting quality assessment of collaborative learning

activities with wiki pages. In Auer, M. E. and

Schreurs, J., editors, Proceedings of the 13th Inter-

national Conference on Interactive Computer Aided

Learning (ICL 2010), pages 1102–1109, Hasselt. In-

ternational Association of Online Engineering.

Malmi, L. and Korhonen, A. (2004). Automatic feedback

and resubmissions as learning aid. In IEEE Interna-

tional Conference on Advanced Learning Technolo-

gies, pages 186–190. IEEE.

Millard, D., Sinclair, P., and Newman, D. (2008). Peer-

Pigeon: A Web Application to Support Generalised

Peer Review. In Proceedings of World Conference

on E-Learning in Corporate, Government, Health-

care, and Higher Education 2008, pages 3824–3836.

Chesapeake.

Pinkwart, N., Harrer, A., Lohmann, S., and Vetter, S.

(2005). Integrating portal based support tools to foster

learning communities in university courses. In Uskov,

V., editor, Proceedings of the 4th International Con-

ference on Web-Based Education, volume 461, pages

201–206. ACTA Press.

Race, P. (2001). A brieﬁng on self, peer and group assess-

ment. Learning and Teaching Support Network.

osner, D., Piotrowski, M., and Amelung, M. (2007). A

sustainable learning environment based on an open

source content management. In Proceedings of the

German e-Science Conference.

Schroeder, U., Rohde, P., and Gebhardt, M. (2008). eS-

trategy der RWTH Aachen: nahtlos durchg

angige

eLearning-Prozesse. Waxmann.

Stalljohann, P., Altenbernd-Giani, E., Dyckhoff, A., Rohde,

P., and Schroeder, U. (2009). Feedback mit einem

webbasierten

Ubungsbetrieb. In Schwill, A. and

Apostolopoulos, N., editors, DeLFI 2009: Tagungs-

band der 7. e-Learning Fachtagung Informatik, pages

283–294, Bonn. Gesellschaft f

ur Informatik.

Stalljohann, P. and Schroeder, U. (2010). A portal-based

gradebook. In Proceeding of the 10th IEEE Interna-

tional Conference on Advanced Learning Technolo-

gies (ICALT 2010). In Press.

Topcuoglu, H. (2006). FAST-Flexible Assignment System.

In Proceedings of E-Learn.

Tremblay, G. and Champagne, F. (2002). Automatic Mark-

ing of Musical Dictations By Applying the Edit Dis-

tance Algorithm On a Symbolic Music Representa-

tion. In Musical Application Using XML (MAX2002),

pages 11–17, Milan, Italy. IEEE Computer Society

Technical Commitee on Computer Generated Music.

Vonderwell, S., Liang, X., and Alderman, K. (2007). Asyn-

chronous discussions and assessment in online learn-

ing. Journal of Research on Technology in Education,

39(3):309.

Wills, G., Bailey, C., Davis, H., Gilbert, L., Howard, Y.,

Jeyes, S., Millard, D., Price, J., Sclater, N., Sherratt,

R., Tulloch, I., and Young, R. (2009). An E-Learning

Framework for Assessment (FREMA). Assessment &

Evaluation in Higher Education, 34(3):273–292.

AN OPEN-ENDED ASSESSMENT PLATFORM IN A HOSTED E-LEARNING PORTAL

401