Using the Open Source Collaborative Model for Digital Educational

Content

azvan Nit

1 a

, Alexandru Apostolescu

, Liza Babu

, Mihai B

arut

Adrian R

azvan Deaconescu

1 b

, Teodor-S

tefan Dut

, S

tefan-Dorin Jum

area

Viorel-Gabriel Mocanu

, Vlad-Iulius N

astase

, Adrian S

endroiu

, Sergiu Weisz

Anna Helga J

onsd

ottir

2 c

, Eggert Karl Hafsteinsson

and Gunnar Stefansson

2 d

National University of Science and Technology Politehnica Bucharest, Romania

University of Iceland, Iceland

Keywords:

E-Learning, Open Educational Resources, Version Control, Collaborative Content Creation, Learning

Environment.

Abstract:

Computer-aided learning has recently seen signiﬁcant adoption among learners. Automated tools that offer

online courses, artiﬁcial intelligence assisted tutoring, practice items, etc. have been developed and are avail-

able for use. On the other side of the spectrum, educators are offered little improvement for the process of

curating the educational materials that are to be used in the classroom. Worse, even though in most cases the

curriculum for one ﬁeld is similar, different educators create different hand-out materials, homework, practice

items, and tests. The preparation for the above is signiﬁcant and the educational content produced is created,

used, and reviewed by a small number of people. In this paper we propose a novel methodology for using

the open-source collaborative model to create, use, and deliver high-quality educational content. Educational

content developed following our methodology guidelines is easy to use, modify, and remix. Educators who

want to use the content may easily select the topics of interest from a proposed set and all of the educational

materials, such as reading materials, assignments, practice items, and even exam items, will be generated.

Some of the content is made available to learners via a generated website, whereas other parts of the content

(such as exam items or assignment solutions) may be hidden. Additionally, all of this content is public and

usable by anyone, including learners of all types (self-educated or following a formal programme). There-

fore, the content may be reviewed and updated by multiple educators and learners alike. We use the proposed

methodology to create educational materials for multiple university courses and present the impact of using

such materials from both educator and learner perspectives.

1 INTRODUCTION

Educational computer programs have been used with

great success to stimulate and deepen the learning

process. Examples of such tools are: online courses

for most of the disciplines of study (from beginner

to expert levels), websites that offer exercise banks

that are automatically evaluated, tutoring programs

that leverage the artiﬁcial intelligence capabilities to

incrementally provide explanations and exercises for

learners(Benigno and Trentin, 2000)(Tallent-Runnels

et al., 2006).

https://orcid.org/0000-0001-9713-2760

https://orcid.org/0000-0001-8287-1712

https://orcid.org/0000-0002-0614-0309

https://orcid.org/0000-0002-2693-9142

Although these tools may be used to some extent

individually by learners, ideally, they would be inte-

grated by educators in their methods. However, given

the vast amount of tools and resources available on

the Internet, educators often use exorbitant amounts

of time to search, select, adapt, and provide educa-

tional materials in various forms: slides, practical

items, exams, homework, projects, etc.(Judith Harris

and Koehler, 2009).

To make matters worse, each educator does their

own due diligence for a speciﬁc class, thus duplicat-

ing the effort of other educators from the same ﬁeld.

Although presentation materials such as slides may be

adapted for one’s needs, other content such as practi-

cal items and exam questions may be shared among

educators to facilitate reuse and collaboration.

Nitu, R., Apostolescu, A., Babu, L., B

arut, M., Deaconescu, A., Du¸tu, T., Jum

area, ¸S., Mocanu, V., N

astase, V., ¸Sendroiu, A., Weisz, S., Jónsdóttir, A., Hafsteinsson, E. and Stefansson, G.

Using the Open Source Collaborative Model for Digital Educational Content.

DOI: 10.5220/0012536200003693

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 16th International Conference on Computer Supported Education (CSEDU 2024) - Volume 1, pages 239-249

ISBN: 978-989-758-697-2; ISSN: 2184-5026

239

Ideally, the learning materials for each ﬁeld of

study would be stored in a public location where any

educator or learner can access it. Additionally, the

content should be easy to understand, browse, select,

adapt and be published to learners. In this case, any

educator can access the content and use it according

to one’s needs, thus minimising the effort of produc-

ing class materials. As an added beneﬁt, the content

would be used by more educators and therefore could

be improved collaboratively. Content development,

use and curation are a challenge for both learners and

educators(Kebritchi et al., 2017).

To that end, in this paper we propose a novel

methodology for creating, using and curating high

quality educational content by using the open-source

collaborative model. Additionally, we provide a set

of tools that may be used to either create educational

content or to directly customise and use it. The con-

tent produced using our methodology is available to

the general public and anyone with an Internet con-

nection may access it. All of the educational materials

are easily modiﬁable since we provide all of the con-

tent in an editable format that may be used to generate

immutable content (such as .pdf or .png ﬁles). Educa-

tors may easily select a subset of the topics presented

and the tools we provide will manage the deployment

of the content that contains all of the required mate-

rials: slides, reading materials, exam items, practice

items, etc. Additionally, we strive to automate edu-

cational tasks that require a lot of effort such as: the

evaluation of practice items, assignments, and exams

as much as possible.

Besides making it easier for educators to discover

and use educational content, the added beneﬁt of our

approach is represented by the fact that the content

may be shared among multiple educators. Having

multiple entities using the same content will create

a community around it which is incentivised to main-

tain a high standard for it.

We have applied our methodology to the devel-

opment of educational materials for multiple univer-

sity courses from a computer science programme. We

show that using the collaborative model for educa-

tional content is beneﬁcial not only for educators but

also for learners, considering the different types of in-

teraction(Moore, 1989). In our particular case, stu-

dents have beneﬁted from having automatic evalua-

tion of most of their tasks and by actively contributing

to the educational content.

In short, the main contributions of this paper are:

• We propose a novel methodology for creating, us-

ing and curating high quality educational content

by using the open-source collaborative model.

• We propose a range of open source tools that may

be used to create and use high quality educational

content.

• We use our methodology to create educational

content for multiple university courses.

• We present the impact on both students and teach-

ing assistants on using the proposed methodology.

The remained of this paper is organised as follows:

Section 2 presents an overview of the methodology.

Section 3 presents the current state of using technol-

ogy in the process of learning. Section 4 details the

actual methodology, as sections dealing with content

development, use and curation. Section 5 is an eval-

uation of the methodology. Section 6 and Section 7

conclude the paper.

2 OVERVIEW

The proposed methodology aims to provide answers

to common questions related to educational content,

such as:

• How do I develop / create a digital educational

content repository?

• How do I use (as an educator) a digital educa-

tional content repository in an actual implemen-

tation (course, training)?

• How do I contribute to a digital educational con-

tent repository?

• How do I curate / manage a digital educational

content repository?

By providing answers to these questions, the

methodology offers uniform guidelines that are fol-

lowed by different content types. New content repos-

itories that follow the same guidelines will present a

comfortable and familiar view to content developers,

users and maintainers, improving the effectiveness of

teaching ﬂows.

To this end, the methodology revolves around

three key concepts:

• content aspects: These are classes of actions re-

lated to digital content in the educational process.

• roles: These are roles of stakeholders in the edu-

cational process. There is a one to one mapping

between roles and content aspects.

• infrastructure: These are digital components

that are used throughout the educational process.

Figure 1 presents the connection between the three

concepts. The digital infrastructure serves as a com-

mon point for three roles and three teaching aspects.

Each of the three teaching aspects, and so each of the

CSEDU 2024 - 16th International Conference on Computer Supported Education

240

Ideas,

Knowledge,

Best Practices

Content Creator

Infrastructure

Content

Repository

Develop &

Organize

Content

Repository

Educator

Configure,

Deploy &

Deliver

Content

Repository

Content Curator

Maintain &

Manage

Digital

Course

Content

Repository

& Community

Input

Role

Aspect

Output

Figure 1: Methodology Concepts.

three roles, relies on the infrastructure and generates a

type of output from a type of input. These three ﬂows

are detailed below:

Content creators develop and organise high

quality learning materials that respect the 5 Rs of open

educational resources(Hyl

en, 2006): Retain, Reuse,

Revise, Remix, Redistribute. They use ideas, knowl-

edge and good practices as input to produce content

repositories as output. The infrastructure employed

consists of open source tools for storing, reviewing

and validating digital content. For content creators,

the methodology provides the guidelines for creating,

organising and storing educational resources.

Educators conﬁgure, deploy and deliver these

materials to learners. They use the content reposi-

tories as input to produce actual digital courses as

output. The infrastructure employed consists of en-

gines to format the content to learners and to automate

teaching workﬂows. For educators, the methodology

provides guidelines on how to use reuse, modify and

remix the existing educational content provided by

content creators.

Content curators maintain and manage the ex-

isting content. They use the content repositories and

the digital courses as input to improve the content

repositories and to create communities around that

content. The infrastructure employed consists of the

same tools for storing, reviewing and validating dig-

ital content, together with communication and col-

laboration tools to include educators using the con-

tent. For content curators, the methodology provides

guidelines on overseeing the use of the content, incen-

tivising contributors, managing modiﬁcation requests

and organising discussions around the existing con-

tent.

Figure 2 provides another view of the ﬂows, fo-

Content

Creator

Content

Curator

Educator

Content

Repository

Digital

Course

Ideas,

Knowledge,

Best Practices

Community

Discussion

& Collaboration

Channels

Version

Control

Systems

Content

Review

Services

Content

Deployment

Engines

Automation

Engines

employs

produces

uses

Figure 2: Methodology Flows.

cused on infrastructure components and connection

between inputs and outputs.

The methodology consists of detailed practical

guidelines for the three ﬂows. Section 4.3 details the

ﬂow related to content creators and the developing

and organising of content. Section 4.4 details the ﬂow

related to educators and the deployment and deliver-

ing of content in actual courses. Note that we use the

term course as a placeholder for any instance of deliv-

ering content to an audience. Section 4.5 details the

ﬂow related to content curators and the maintenance

and management of existing content.

The methodology itself is organised inside a con-

tent repository that follows its own guidelines. This

makes the methodology the target of its own ﬂows, in-

cluding its continuous improvement. The repository

provides actionable items in the form of guides, ex-

amples, references to existing content and pointers to

infrastructure components. These are to be followed

by the person acting as a content creator, educator, or

curator to provide a collection of high quality, contin-

uously improved digital educational content reposito-

ries.

Note that one person could be ﬁlling multiple

roles. For example, someone can choose to ﬁll only

the educator role and use and deliver existing content

as part of their own digital course. In the absence of

existing content, someone can initially ﬁll the role of

content creator to develop required content, then the

role of educator to deliver it, and ﬁnally the role of cu-

rator to maintain the community around the project.

Overall, the methodology, as detailed in Section

4, structures the common ﬂows related to educational

content. It answers questions of the form “How do I

. . . ?”, thus providing the focus for effective content-

related aspects: development, delivery, and mainte-

nance.

Using the Open Source Collaborative Model for Digital Educational Content

241

3 BACKGROUND

Web-based learning environments have grown signif-

icantly in usage and popularity over the past couple of

decades (Chirikov et al., 2020), (Mathias Decuypere

and Landri, 2021), (Chen et al., 2020). Some of these

are open and free to use, others commercial prod-

ucts. These digital platforms, ranging from compre-

hensive Learning Management Systems (LMS) like

Moodle and Canvas to Massive Open Online Courses

(MOOCs) platforms such as Coursera and edX, offer

diverse and customizable learning experiences. For

those focusing on technical skills, interactive coding

platforms like Codecademy provide hands-on cod-

ing exercises, while a platform like DataCamp offers

courses in data analysis. Also, there are specialized

platforms such as Duolingo for language learning and

the tutor-web, an open-source learning environment

designed to teach mathematics and statistics (Jonsdot-

tir and Stefansson, 2014), (Jonsdottir et al., 2017).

Many of these platforms offer features beyond simple

content delivery. For instance, there are features for

interactive assessments, forums for discussion, tools

for peer review, and mechanisms for instructors to

provide feedback, a key element in student learning.

As stated in (Black and Wiliam, 1998), several studies

have shown ﬁrm evidence that innovations designed

to strengthen the frequent feedback that students re-

ceive about their learning yield substantial learning

gains. Providing students with frequent quality feed-

back can be time consuming for educators but by us-

ing these available tools students can get feedback on

their work that does not require marking by teachers.

There is no doubt that web-based learning envi-

ronments have increased accessibility to learning, al-

lowing students from diverse backgrounds and geo-

graphic locations to access high-quality educational

resources, and breaking down traditional barriers (Zi-

Yu Liu and Korobeynikova, 2020), (Puggioni et al.,

2021), (Eleftheria et al., 2013), (He et al., 2022).

From the teacher’s standpoint, however, there is of-

ten a limitation to these platforms; they are not easily

contributable, meaning that an educator does not have

the possibility of selecting subsets of topics or organ-

ising entire courses tailored to their learners’ needs.

To address these issues, attempts have been made

to use version control systems (Angulo and Ak-

tunc, 2019), such as Github

, to store educational

resources and share them with students. GitHub is

a good choice for storing open educational resources,

promoting collaboration and version control, it makes

it easier for educators to work together on course ma-

terials, track changes, and offer resources that can be

www.github.com

accessed and improved by everyone, even by the stu-

dents (Glassey, 2019), (Bhasin et al., 2021). It offers

free hosting using GitHub Pages, making it easier to

create a website.

Multiple course repositories exist, however, each

implements its own structure and deployment options.

Since the resources don’t have a standard structure

it’s hard to reuse that content, making the contribu-

tion from other communities less likely. In fact, there

are databases of courses on GitHub where each course

follows a different structure. This makes it difﬁcult

for educators to discover and navigate the content that

they need. Another problem with the already present

courses on GitHub is that the content is stored in PDF

format. While PDFs are widely used for document

sharing, they have several limitations in the context

of open education resources. PDF format is typically

non-editable, making it difﬁcult for educators to mod-

ify the content for their speciﬁc needs or to contribute

improvements.

Our methodology addresses this limitation, by

creating a generic repository structure that is easy

to navigate, contribute to and reuse even by non-

technical educators. To the best of our knowledge,

there are no other proposed methodologies on organ-

ising and using educational resources for the open

source environment.

4 METHODOLOGY

The methodology that we propose offers guidelines

for all levels of interaction with a course repository.

In this section, we describe the main aspects of how

users are going to interact with a potential repository

depending on their needs.

Versioning systems fundamentally group informa-

tion into repositories(Santos-Hermosa Gema, 2017).

As such we consider that all of the educational re-

sources pertaining to a particular course or ﬁeld of

study are going to be organised into a repository.

4.1 Content Types

Educational resources come in various forms and

sizes. Examples of such resources are represented by

books, wikis, videos, computer learning games, and

multiple choice questions - just to name a few. To

be able to easily navigate the content from an educa-

tional repository it is imperative that a wide range of

content types be supported.

We group educational resources into 6 broad cate-

gories that are representative of most (if not all) types

CSEDU 2024 - 16th International Conference on Computer Supported Education

242

Table 1: Types of content.

Content Type Examples

Reading books, wikis, articles

Media images, recordings, videos, games

Slides .pptx ﬁles, .ppt ﬁles

Guides tutorials, demonstrations

Drills quizzes, problems, essays

Projects assignments

of content. Table 1 highlights these categories which

are detailed below.

The reading content type is the most common

form of storing educational materials (or information,

in general). As the name suggests the content is de-

veloped to be read, therefore it comes in the form of

text. The reading content type is typically used stan-

dalone or in conjunction with other types of content

to explain topics.

The media content type refers to audio, visual, or

audiovisual educational materials. The media content

type represents an easily digestible format for present-

ing information. As such, media is typically embed-

ded within the text that is destined for reading, slides,

guides, drills, and projects.

Slides are the main support material for live activ-

ities, such as lectures. They can also be used during

practical sessions or talks (such as conferences). The

main role of slides is to provide visual support mate-

rial for presenting information and engaging partici-

pants in discussions.

Guides are pieces of content consisting of detailed

steps that are commonly used in practice. Guides are

to be used by learners as tutorials or by educators as

demonstrations.

Drills are educational activities designed to rein-

force learning through repetitive practice. Drills come

in many forms such as multiple choice questions, es-

says, practice items, etc.

Projects are practice items that are larger in scope

and require extensive time, effort, and resources to

complete. When compared to drills, projects aim to

present a comprehensive narrative that can be divided

into smaller, coherent steps.

Each repository for educational content is going

to contain a mix, if not all, of the above content cat-

egories. To make it easy to understand and reuse the

educational content, we propose a speciﬁc structure

for the repository.

4.2 Repository Structure

The structure of the repository needs to be easy to

navigate so that educators intuitively ﬁnd the re-

Repository Name

ChapterName1

TopicName1

Drills

Guides

Media

Projects

Reading

Slides

TopicName2

TopicName3

ChapterName2

ChapterName3

ChapterName4

Figure 3: Repository Structure Sample - A repository con-

tains any number of chapters. Each chapter contains any

number of discussed topics. Each topic contains all of the

content types deﬁned for it.

sources that they search for, while also making it easy

to select content at whatever granularity it is needed.

To that end we have devised the repository struc-

ture presented in Figure 3. The resources of a course

or ﬁeld of study are grouped into a numbers of chap-

ters. Each chapter is represented by a number of top-

ics. For each topic, a number of content types (ideally,

all) exist.

By using this form of organisation, educators can

easily browse through the available resources and se-

lect whatever topics or chapters of interest. Once a

topic is selected to be a part of the course, all of the

resources pertaining to that speciﬁc topic will be in-

cluded in the ﬁnal presentation material. Essentially,

all of the content that is needed for a lecture or a prac-

tical session is just one click away.

If some content needs to be excluded at a smaller

granular level (some speciﬁc drills, or some speciﬁc

assignments) that is also possible, but generally, top-

ics should be treated as indivisible units.

Using the Open Source Collaborative Model for Digital Educational Content

243

Listing 1: Drill File Example. The drill information is

stored as a list of key-value pairs of the form Title: ”Sam-

ple Question Name”, Difﬁculty: ”easy”, Tags: ”topic-1”,

”topic-2” etc.

# Sample Q ues t i on Name

## Di f fi c u lt y

easy

## Tags

- topic -1

- topic -2

## Qu e s tio n T e x t

Is t h i s a samp l e qu es t i on ?

## Qu e s tio n An swe r s

+ Yes

- No

- M a y b e

## Fe e d bac k

Th e q u est i o n is self - r e f le c ti n g

For content curators and content creators this form

of organisation is advantageous because it lifts the

burden of thinking about dependencies between top-

ics. The topic is created and if prior knowledge is

necessary, a simple link to a different topic (that the

current one is depending on) or chapter may be in-

serted. It is the burden of the educator to establish the

order in which different concepts are taught.

4.3 Developing Content

For content developers, the methodology provides

guidelines on how to create interactive and attrac-

tive content that is easy to reuse, modify and pub-

lish. Therefore, content creators are given recommen-

dations in two directions: (1) how to develop content

that is valuable for learners and (2) how to create con-

tent that is easily usable by educators.

For creating attractive educational resources, the

methodology provides recommendations regarding

interactivity, using as much as possible media items

instead of reading materials, how to create the layout

of an educational activity etc.

For creating content that is easily usable by edu-

Educator

Publishing

and

Deployment

Infrastructure

Deployment

Configuration

Drill

Interface

Published

Content

Learner

Content

configuration

Content

Repository

Figure 4: Flow for using the educational content: the educa-

tor browses the content and selects the topics of interest by

providing a conﬁguration ﬁle; additionally, a deployment

conﬁguration is provided that describes various deployment

options referring to the publishing format. The publisher

tool that we have developed then renders the course materi-

als and updates the websites that host the drill interface.

cators, the methodology offers guidance on what for-

mats are to be used for the educational materials so

that they are easily editable and the tools that may be

used to achieve this.

We focus on the latter aspect, since the former is

beyond the scope of this paper.

The easiest form in which information may be

edited is text. As such, we strive to store all of the ed-

ucational resources as text, in the form of Markdown

ﬁles. We have chosen Markdown because it offers

an easy way to format text by using speciﬁc annota-

tions. Additionally, Markdown is used to nicely ren-

der text ﬁles in most versioning systems. Excepting

media ﬁles, we use Markdown for all other content

types.

Reading, projects, guides content types are eas-

ily stored in Markdown since these are essentially

text-based resources.

Slides are also stored in Markdown and are ren-

dered either in the browser or as a .pdf ﬁle by using

reveal-md. reveal-md is sufﬁciently powerful to han-

dle most of the situations that a graphical presentation

tool such as PowerPoint can, however, it is text-based.

In our experience, creating slides using Markdown

and reveal-md is easier and has the added beneﬁt of

having the source easily editable.

Drills are also stored using Markdown, however,

depending on the drill type and the publishing format,

additional transformations may be used. Additionally,

we want to also store the correct answer in the drill

ﬁle, so that (1) educators understand what is the ex-

pected answer, (2) learners who solve the drill as part

of a self-study session can check their answers and

CSEDU 2024 - 16th International Conference on Computer Supported Education

244

(3) if the drill solution is automatically veriﬁable, a

correct answer is needed.

As a consequence, we have developed a standard

for representing drills in Markdown ﬁles. Listing 1

highlights a sample of such a drill ﬁle. Such ﬁles are

parsed by a script we provide which creates a dic-

tionary where each ## preceded line represents the

key and the following lines up to the next # repre-

sent the value. Some keys are mandatory, namely

”Title”, ”Question Text”, ”Question Answers” and

”Feedback” but the others are optional. The format

is ﬂexible enough to support any kind of question that

targets any kind of publisher. All that needs to be done

is that the drills are annotated with the proper meta-

data so that the publisher correctly supports them.

Additionally, the format is simple enough that even

a non-technical person may understand it and modify

the drill content.

Media ﬁles come in various forms and each type

has different properties. For example, images may be

stored as .svg ﬁles which are easily modiﬁable and

translatable to a different format such as .png, .jpg or

.pdf using draw.io. However, videos, audio ﬁles or

recordings are not easily editable. As a consequence,

in these situations, we store the non-editable version

since, typically, when this sort of content type man-

dates modiﬁcations, a new version is created from

scratch. In the particular case of videos that are cre-

ated out of static images, the images could be stored in

.svg format alongside the script that creates the video.

4.4 Using Content

Using the content is designed to be as easy as possi-

ble. Figure 4 highlights the main aspects of using ed-

ucational content that is developed using our method-

ology. The educator browses the existing content and

selects the topics or chapters of interest. The selection

is made via a graphical interface which then is trans-

lated into a content conﬁguration ﬁle. Additionally,

the educator may select deployment options such as:

the format in which the materials will be published

(https for rendering on a website, .pdf for presenta-

tions or other custom targets), and the target platform

for drill hosting (so that appropriate pre-processing is

done). The educator need not bother to create the de-

ployment settings as the default option is to render

and host everything on a public website. If the ed-

ucator has a preference for a different ﬂavor of end

publishers, that may also be customised, however, in

this case, the educator must have the knowledge to

properly conﬁgure the deployment parameters. Addi-

tionally, the publishing and deployment infrastructure

may easily be extended to support any kind of pub-

lishing format, as long as translators from Markdown

to the targeted formats are provided. Once the deploy-

ment and content conﬁguration ﬁles are extracted and

handed over to the publishing and deployment infras-

tructure, all of the educational materials are translated

to the appropriate formats and deployed on the pro-

vided targets. Once deployed, students may access

and consume the educational materials.

By providing a simple selection interface, the ed-

ucator may obtain educational materials for an entire

course just by doing a few clicks. Additionally, both

the educator and the student may propose changes to

the materials.

Note that the educational materials developed fol-

lowing our methodology do not target a speciﬁc

teaching activity (such as a lecture, seminar, work-

shop, practical session etc.). The content categories

may be combined during different teaching activities

as the educator sees ﬁt. However, the methodology

does provide a taxonomy of teaching activities and

how each content type may map on it.

Table 2 provides the broad categories of teach-

ing activities that we were able to identify given our

teaching experience.

Lectures are teaching activities that typically take

place with a larger audience and are asymmetrical: a

lecturer mostly talks while the audience mostly lis-

tens. Interactivity is important but is limited by the

larger audience: not all participants will be able to ac-

tively engage in discussions. Slides and media items

are used most frequently during lectures, however,

guides (in the form of demonstrations or tutorials)

may also be employed. Drills may be used in the form

of multiple choice questions to engage the audience

and have instant feedback on whether the audience

has understood the presentation.

Practical Sessions are represented by hands-on

activities that learners perform in order to acquire ex-

perience. These activities are typically supervised by

an educator who proposes the practical exercises and

evaluates the learner’s performance. The purpose of

the evaluation is to provide feedback as opposed to

grading the learner. Instances of practical sessions

are represented by lab sessions, seminars, or work-

shops. A practical session may employ, depending on

educator preference, reading materials for short ex-

planations, slides for educator presentations, media

items to support the previous two, guides that assist

the learner in learning, and drills that offer the possi-

bility for hands-on practice.

Assignments are unassisted practical activities

that learners undertake individually or in teams over

a longer time span. Their primary objective is to pro-

vide learners the ﬂexibility to work through tasks at

Using the Open Source Collaborative Model for Digital Educational Content

245

Table 2: Teaching activities mapped on content types.

Teaching Activity Category Content Type

Lecture Learning slides, media, guides, drills

Practical Session Learning reading, slides, media, guide, drills

Assignment Learning + Assessment projects, drills

Test Assessment drills

Self Study Assessment reading, media, drills, projects, guides

their preferred speed, allowing them to address gaps

in their understanding as they progress. Assignments

primarily use the project content type, however, they

may also be generated as a series of drills. Although

the primary purpose of an assignment is to deepen the

understanding of one or a series of topics, they are

usually graded, taking part in the assessment process.

Tests are assessments designed to evaluate learners’

understanding and application of the course material.

Tests are comprised of a series of drills.

Self Study is the process of independently learn-

ing outside of structural classroom settings or for-

mal instruction. Learners may individually access and

consume the educational materials, therefore all con-

tent categories, except for slides, may be used for this

activity. We have excluded slides as a learning mate-

rial for self study because slides are typically used for

presentation purposes, not self study.

Note that the teaching activities that involve skill

assessment might require that some parts of the re-

sources be private to the educator so that learners can-

not cheat. This comes in contrast to the open nature

of the educational content that is produced by using

our methodology.

For tests, our perspective is that the examination

should be done in a supervised environment to make

sure that the learner does not access the online solu-

tion. As for knowing all of the answers in advance, we

consider that the drill database should be sufﬁciently

large so that if a potential learner memorises all the

drills and their solution, then it can be considered that

the learner has mastered the course.

For assignments, however, we have not come up

with a solution. By their nature, projects are to

be solved in an unsupervised environment, therefore,

having a public solution makes it easy to cheat an as-

signment. We do not have a solution for this aspect,

other than expecting that the educators will modify

the project so that the public solution cannot be used.

4.5 Curating Content

Each repository of educational content is supervised,

or curated, by one or several content maintainers. Our

methodology describes the responsibilities of a con-

tent maintainer:

• Content management: to review and handle mod-

iﬁcation requests, to create tasks and organise the

work that should be done on the repository, to es-

tablish contribution guidelines, and acknowledge

the work of contributors.

• Community building: promote the content, en-

courage and organise discussions around it and

build an ecosystem of content users and contribu-

tors.

Content maintainers are typically the persons that

initially developed the educational materials, how-

ever, in time, proliﬁc contributors may also take on

this role.

4.6 Automation

Besides making it easy for educators to reuse exist-

ing content, another important aspect of our method-

ology is to design educational materials that alleviate

as much as possible the need for manual effort. As

such, we try to automate as much as possible content

generation and evaluation. Although this may not al-

ways be possible (for example, philosophical essays

still need to be graded by a human), most STEM ﬁelds

are amenable to automatic evaluation.

Automatic evaluation may be used for:

• Lectures, for multiple choice questions that are

engaging for learners. The result of a multi-

ple choice question is available live for both the

learner (to validate the understanding) and the

teacher (to aggregate the results and understand

whether a concept was understood or not).

• Practical sessions for drill categories such as

multiple choice questions, ﬁll-in-the-blanks, pro-

grams that need to be developed, etc. For en-

gineering ﬁelds, this should be easily imple-

mentable, as we show in Section 5.

• Assignments, provided that the assignment is

amenable to automated testing. In computer sci-

ence ﬁelds, for example, assignments are typically

in the form of computer programs therefore these

CSEDU 2024 - 16th International Conference on Computer Supported Education

246

Figure 5: Content deliverers’ level of agreement with the

statement ”The practical session materials were helpful in

understanding the presented concepts”.

Figure 6: Content deliverers’ level of agreement with the

statement ”The lecture materials were helpful in under-

standing the presented concepts”.

Figure 7: The evolution of exam grades over time for the

Operating Systems class. In 2023 we have used the educa-

tional resources developed using our methodology.

can be easily testable. We also provide guide-

lines on how to set up a generic assignment testing

framework that uses a versioning system.

• Tests, provided that the exam is in the form of

multiple choice questions or a practical assess-

ment (such as writing a computer program).

In our methodology, we provide guides on how to

set up an infrastructure for automated testing of all of

the above categories. In addition, our repository tem-

plate already contains default conﬁgurations that aid

in setting up the infrastructure. As such, content cre-

ators can easily set up the infrastructure when creating

a repository from scratch, educators typically are not

concerned with this (except for choosing content that

is amenable to automated evaluation) and content cu-

rators only have to maintain the initially conﬁgured

infrastructure.

5 EVALUATION

To apply our methodology we have created educa-

tional resources for 2 university courses - Operating

Systems (OS) and Computing and Calculus for Ad-

vanced Statistics (CCAS) - and 2 university summer

schools - Security Summer School (SSS) and D Sum-

mer School (DSS).

For the OS class, the materials were created

from scratch, whereas for the rest the materials were

”ported” to follow the methodology.

From this selection of courses, the OS class has

the largest audience, roughly 600 students each year,

and the largest team involved in maintaining and de-

livering the content. As such, we further present a

series of metrics to showcase the impact of using our

methodology for the OS class.

Figure 7 highlights the evolution of exam grades

for the OS class. Prior to 2020, we employed a writ-

ten exam and the average grade was around 6. In

2020, because of the pandemic, an oral exam was in-

troduced, and since it was the ﬁrst time we were us-

ing this type of exam, the examiners were very indul-

gent. The examination was calibrated so that it was

comprehensive enough without being too easy. As

a consequence, the average grade started decreasing.

However, when we introduced the new learning ma-

terials in 2023, the average grade increased. We note

that the team of examiners and the team of content

deliverers were roughly the same starting from 2020.

Although some of the persons who created the mate-

rials are also part of the examination team, the large

majority of them are not content contributors.

Furthermore, we wanted to evaluate the perspec-

tive of content deliverers on the materials we devel-

oped for the OS class. For that, we have devised a

feedback form and asked the content deliverers from

our course to complete it. The questionnaire con-

tained two statements and the respondents were asked

to complete the level of agreement with them. Figures

6 and 5 present the perspective of content deliverers

on the educational materials. As it can be observed,

the majority of respondents (28 participants) consider

that the materials help understand the presented con-

cepts.

Note that the content deliverers were not involved

in selecting the topics that were presented during the

course and practical sessions. The materials were al-

ready organised for them, they simply had to deliver

the content (in the form of lectures and practical ses-

sions). As such, we do not have any evidence of

whether the content that we provided is easy to con-

ﬁgure. However, there were content deliverers who

were not part of the content development team that

Using the Open Source Collaborative Model for Digital Educational Content

247

Table 3: Number of contributions for our Operating Systems educational materials.

Total Number Authored by Students Resolved

Pull Requests 222 32 (14.4%) 220

Issues 112 18 (16.0%) 72

have made contributions to the content. Furthermore,

we had student contributions to our educational con-

tent.

Table 3 showcases the number of ”Pull Requests”

(modiﬁcation requests) that were made to our content

repository. The number includes contributions from

content developers, content deliverers, and students

over the course of a year. A non-negligible percent-

age of those is made by students ( roughly 14.5% ),

most of which were accepted. The ”Resolved” col-

umn represents the number of requests, out of the to-

tal number, that have been processed. Processing a

pull request means that it has either been integrated

or closed without being merged. The majority of the

pull requests have been merged after changes were re-

quested by the content curators. A small fraction (25

pull requests) have been closed without being merged.

For issues, the ”Resolved” column signiﬁes the num-

ber of issues that have been closed as a result of a pull

request being merged.

This shows that once the content is publicly con-

tributable, both learners and educators/content devel-

opers beneﬁt from it: learners deepen their knowledge

by proposing modiﬁcations, whereas educators/con-

tent developers get to have their content improved.

In short, we have validated that the content de-

veloped following our methodology is indeed of high

quality, showing that educators appreciate it and

learners beneﬁt from it. Additionally, given the con-

tributions we obtained, we consider that we demon-

strated that the materials are also easy to contribute

to. However, we have not yet proven that our materi-

als are easily conﬁgurable.

6 NEXT STEPS

Currently, we are using the educational content that

we have developed following our methodology, how-

ever, we do not know if other educators ﬁnd it useful.

Therefore, our next step is to create an ecosystem of

educators around the content that has been developed

so far. Once educators start using the developed re-

sources, we will be able to obtain feedback outside of

the community from our university.

At the same time, we are continuously looking for

ways to improve our interfaces so that educators with

a non-technical background are capable of using and

developing materials for their courses.

Additionally, we want to create educational mate-

rials for more courses, ideally for non- technical ﬁelds

of study (such as geography, philosophy, etc.). Break-

ing the barrier of non-STEM ﬁelds of study will en-

able us to better understand what are the needs for

automation and usability for course materials that are

not naturally amenable to our methodology.

7 CONCLUSION

The technological revolution has enabled new ways of

learning and a multitude of tools have been developed

that aid the learning process. These tools primarily

target learners. No tools, to the best of our knowledge,

have been developed to ease the process of gathering

and organising teaching resources for educators.

To that end, we have presented a novel method-

ology that offers guidelines and recommendations on

how to create high quality educational content that is

easy to use, modify, and remix. Our methodology tar-

gets content creators, content deliverers, and content

curators.

We have used our methodology to develop edu-

cational materials for two university courses and two

university summer schools. We have presented met-

rics for one of the university courses that has been de-

livered that demonstrate the value of the created ma-

terials for both educators and learners. Further, we

have shown that content that is developed by using

our methodology can be easily contributed to.

We envision a future where educators, irrespec-

tive of the ﬁeld that they are teaching, may easily ac-

cess a platform and search for the topics they want

to teach, select a few, and push a ”publish” button,

and all of the course materials will be made available

to learners. Furthermore, all evaluation will be done

automatically. Once the burden of searching, gather-

ing, curating educational materials, and manual grad-

ing is lifted from educators, they will have more time

to spend on mentoring and coaching learners.

We consider that this work is a ﬁrst step towards

that future.

CSEDU 2024 - 16th International Conference on Computer Supported Education

248

ACKNOWLEDGEMENTS

This work has been supported by EEA Grants under

the 21-COOP-0016 Programe

REFERENCES

Angulo, M. A. and Aktunc, O. (2019). Using github as a

teaching tool for programming courses. In 2018 Gulf

Southwest Section Conference.

Benigno, V. and Trentin, G. (2000). The evaluation of on-

line courses. Journal of Computer Assisted Learning,

16(3):259–270.

Bhasin, T., Murray, A., and Storey, M.-A. (2021). Stu-

dent experiences with github and stack overﬂow: An

exploratory study. In 2021 IEEE/ACM 13th Interna-

tional Workshop on Cooperative and Human Aspects

of Software Engineering (CHASE), pages 81–90.

Black, P. and Wiliam, D. (1998). Assessment and classroom

learning. Assessment in Education: principles, policy

& practice, 5(1):7–74.

Chen, T., Peng, L., Yin, X., Rong, J., Yang, J., and Cong,

G. (2020). Analysis of user satisfaction with online

education platforms in china during the covid-19 pan-

demic. Healthcare, 8(3).

Chirikov, I., Semenova, T., Maloshonok, N., Bettinger, E.,

and Kizilcec, R. F. (2020). Online education plat-

forms scale college stem instruction with equivalent

learning outcomes at lower cost. Science Advances,

6(15):eaay5324.

Eleftheria, C. A., Charikleia, P., Iason, C. G., Athanasios,

T., and Dimitrios, T. (2013). An innovative augmented

reality educational platform using gamiﬁcation to en-

hance lifelong learning and cultural education. In IISA

2013, pages 1–5.

Glassey, R. (2019). Adopting git/github within teaching:

A survey of tool support. In Proceedings of the

ACM Conference on Global Computing Education,

CompEd ’19, page 143–149, New York, NY, USA.

Association for Computing Machinery.

He, Z., Xia, W., Dong, K., Guo, H., Tang, R., Xia, D., and

Zhang, R. (2022). Unsupervised learning style clas-

siﬁcation for learning path generation in online edu-

cation platforms. In Proceedings of the 28th ACM

SIGKDD Conference on Knowledge Discovery and

Data Mining, KDD ’22, page 2997–3006, New York,

NY, USA. Association for Computing Machinery.

Hyl

en, J. (2006). Open educational resources: Opportuni-

ties and challenges. Proceedings of open education,

4963.

Jonsdottir, A. H., Bjornsdottir, A., and Stefansson, G.

(2017). Difference in learning among students do-

ing pen-and-paper homework compared to web-based

homework in an introductory statistics course. Jour-

nal of Statistics Education, 25(1):12–20.

https://eeagrants.org/archive/2014-2021/projects/RO-

EDUCATION-0178

Jonsdottir, A. H. and Stefansson, G. (2014). From evalua-

tion to learning: Some aspects of designing a cyber-

university. Computers & Education, 78:344–351.

Judith Harris, P. M. and Koehler, M. (2009). Teachers’ tech-

nological pedagogical content knowledge and learn-

ing activity types. Journal of Research on Technology

in Education, 41(4):393–416.

Kebritchi, M., Lipschuetz, A., and Santiague, L. (2017).

Issues and challenges for teaching successful online

courses in higher education: A literature review. Jour-

nal of Educational Technology Systems, 46(1):4–29.

Mathias Decuypere, E. G. and Landri, P. (2021). Introduc-

tion: Critical studies of digital education platforms.

Critical Studies in Education, 62(1):1–16.

Moore, M. G. (1989). Editorial: Three types of interaction.

American Journal of Distance Education, 3(2):1–7.

Puggioni, M., Frontoni, E., Paolanti, M., and Pierdicca, R.

(2021). Scoolar: An educational platform to improve

students’ learning through virtual reality. IEEE Ac-

cess, 9:21059–21070.

Santos-Hermosa Gema, Ferran-Ferrer N

uria, A. E. (2017).

Repositories of open educational resources: An as-

sessment of reuse and educational aspects. Interna-

tional Review of Research in Open and Distributed

Learning, 18(5):84–120.

Tallent-Runnels, M. K., Thomas, J. A., Lan, W. Y., Cooper,

S., Ahern, T. C., Shaw, S. M., and Liu, X. (2006).

Teaching courses online: A review of the research.

Review of Educational Research, 76(1):93–135.

Zi-Yu Liu, N. L. and Korobeynikova, E. (2020). Online

learning platforms: Reconstructing modern higher ed-

ucation. International Journal of Emerging Technolo-

gies in Learning, pages 5–18.

Using the Open Source Collaborative Model for Digital Educational Content

249