Problem-Based Learning in a MOOC
Exploring an Innovative Instructional Design at a Large Scale
Daniëlle M. L. Verstegen, Annemarie Spruijt, Diana Dolmans and Jeroen van Merriënboer
Department of Educational Development and Research, Faculty of Health, Medicine and Life Sciences,
Maastricht University, P.O. Box 616 (Visiting Address: universiteitsSingel 60), 6200 Md Maastricht, The Netherlands
Keywords: Massive Open Online Course, MOOC, Problem-Based Learning, PBL, Innovative Instructional Design,
Collaborative Online Learning.
Abstract: This paper describes a MOOC about PBL which is designed –as far as possible in the setting of a MOOC-
in line with modern learning principles that are also at the basis of PBL: constructive, contextual,
collaborative and self-directed learning: Problem-Based Learning: Principles and design. Students at the
centre! The course is centred around a set of authentic ‘problems’ organized in three different tracks
(tutoring, designing, and implementing PBL) targeted at different types of participants. Small group work is
essential in this MOOC. Students can either form teams themselves or be assigned to a team automatically.
Each team has team space with chat facilities, file exchange, and facilities to schedule online meetings.
However, teams can decide themselves how they want to collaborate and communicate, synchronously or
asynchronously. A pilot study brought forward strong and weak points, which were used to further improve
the design. This paper describes the pilot study, the changes made in the design and some first impressions
of the first run of the MOOC. Preliminary conclusions are that MOOCs require careful instructional design.
Stimulating online small group learning in a MOOC, i.e. following PBL learning principles to an extent but
without tutors for each team, is possible, but not easy.
1 INTRODUCTION
Massive Open Online Courses (MOOCs) are a
global trend that will potentially change the whole
concept of higher education (Waldrom 2013; WUN
2013; Yuan and Powell 2013), however currently
dominated by key players: Coursera
(https://www.coursera.org/), EdX
(https://www.edx.org/) and Udacity
(https://www.udacity.com/). The UK Open
University offers MOOCs at Futurelearn
(https://www.futurelearn.com/courses). Within the
Netherlands the trend of MOOCs is recognized by
the government (Bussemaker 2013) and several
MOOC initiatives are starting up, Delft University
being most active up to now offering courses
through EdX.
The term MOOC has been applied to a variety of
online and blended courses (Hollands & Tirtaly
2014). Historically, so called ‘cMOOCs’ aim at
facilitating learning through participant interactions
with a network of individuals in which the teacher
has a far less prominent role. Participants are
encouraged to create, share, and build upon each
other’s artefacts. Another stream in MOOC
development, however, aims primarily at delivering
education at scale and involves more structured and
sequenced direct transmission of knowledge.
Hollands and Tirtaly (2014) researched how the
term MOOC is interpreted. The word ‘massive’ in
MOOCs usually refers to a large number of
participants. ‘Open’ usually refers to the possibility
for anyone with adequate internet to participate in
the course, typically also for free. Online refers to
availability via the internet, and most agreed that, to
be labelled a “course,” MOOCs should be bounded
by time, that is, have a beginning and an end point.
It should provide a coherent set of resources; and
follow a sequence of activities organized by an
instructor in order to address specific learning
objectives or goals.
Many existing MOOCs are criticized for lack of
sound instructional design (e.g. Holton 2013).
McAndrews and Scanlon (2013) stress that MOOCs
require careful instructional design using lessons
learned from other forms of distance education.
Verstegen, D., Spruijt, A., Dolmans, D. and Merriënboer, J.
Problem-Based Learning in a MOOC - Exploring an Innovative Instructional Design at a Large Scale.
In Proceedings of the 8th International Conference on Computer Supported Education (CSEDU 2016) - Volume 2, pages 369-377
ISBN: 978-989-758-179-3
Copyright
c
 2016 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
369

Alternative ideas are being developed stressing
learner participation and engagement (Ahn, Butler,
Alam & Webster, 2013) and the development of
connectivist MOOCs (cMOOC) (Mackness, Waite,
Roberts and Lovegrove 2013; Morrison 2013).
Maastricht University (UM) has a strong
tradition in Problem-Based Learning (PBL). PBL is
a powerful student-centred educational approach,
where learning is centred around authentic ill-
structured problems (e.g., Barrows 2002; Barrows &
Tamblyn 1980; Dolmans, De Grave, Wolfhagen &
Van der Vleuten 2015; Moust, Bouhuijs & Schmidt
2014). PBL focuses at small-group learning centred
around authentic problems. At first sight, this is in
contrast with the large-scale and often teacher-
driven set-up of MOOCs.
In a university wide project a MOOC about PBL
was developed which is designed –as far as possible
in the setting of a MOOC- in line with modern
learning principles that are also at the basis of PBL:
Problem-Based Learning: Principles and design.
Students at the centre! (https://novoed.com/problem-
based-learning/). The goals of this project were to
evaluate an innovative PBL-based instructional
design for MOOCs and to gain first-hand experience
with MOOCs, and explore potential implications for
Maastricht University and its students. More
information about the project can be found at
http://moocs.maastrichtuniversity.nl/.
This paper describes the design of this MOOC in
relation to the principles of PBL, reports results of
the pilot study, and first impressions of the first open
run of the MOOC in October-December 2015.
2 DESIGN OF THE MOOC
2.1 Topic and Project Team
At the start of the project there were a few
requirements: a university-wide project team, a topic
in the area of education and learning, and a format in
line with modern learning principles and the
educational vision of the university. The project
team consisted of 34 people, including
representatives from all faculties: Health, medicine
and life sciences, Law, Psychology and
neuroscience, Business and economics, Arts and
social sciences, and Humanities and sciences, and
some student-assistants.
The project team decided to take PBL also as
topic for the MOOC because it allows input from all
faculties and is in line with Maastricht University’s
educational vision. The defined target group consists
of people with a professional or personal interest in
education in general, and forms of problem-based
learning in particular. These will often be teachers,
tutors, instructional designers, curriculum
coordinators and other educational leaders, but may
also include current and future students of master or
PhD programs in the educational field or other
students interested in PBL.
2.2 Instructional Design
2.2.1 Duration and Workload
The MOOC Problem-Based Learning: Principles
and design. Students at the centre! is designed to last
9 weeks was designed to have a study load
equivalent to 2 ECTS. Figure 1 shows that the first
week is a ‘pre-week’ dedicated to learning more
about the structure of the course and forming teams.
(The team charter assignment was added later on
after pilot study, see Section 3). Subsequently,
participants work in groups on authentic problems in
a similar way as face-to-face PBL groups, except
that they work online and do not have a tutor.
Students who actively participate and finish the
course are given a Certificate of Participation, but
there is no formal exam.
2.2.2 Authentic Problems in Three Tracks
The course is centred around a set of authentic
‘problems’ organized in three different tracks that
are targeted at different types of participants. The
first and the last week participants in all tracks work
on the same problems, focusing on the learning
principles underlying PBL and the application of
Figure 1: The 'pre-week' in the MOOC Problem-Based Learning: Principles and design. Students at the centre.
CSEDU 2016 - 8th International Conference on Computer Supported Education
370

PBL principles in their own setting. In between they
work on track-specific problems (see Figure 2). In
the middle part the tracks split up:
- Track 1: The role of the tutor in PBL. This track
focuses on the teacher in the role of tutor. This is
often the first role that beginning teachers take in
a PBL curriculum.
- Track 2: Designing PBL problems and courses.
This track focuses on design aspects of PBL,
which might be interesting for instructional
designers and for more experienced teachers who
are taking up the role of PBL problem author or
course coordinator.
- Track 3: Assessment and organizational aspects
of PBL. This track looks into aspects of PBL at
the curriculum level, aligning assessment,
implementation and innovation of PBL curricula.
It targets educational managers or experienced
staff taking up the role of curriculum
coordinator.
Each problem is divided over 2 weeks, including a
brainstorm phase or pre-discussion, self-study, and a
reporting phase or post-discussion. With the problem
description some basic resources are given in the
form of video clips or public booklets or journal
articles. A larger set of references, some free and
some licensed, are provided on a Bibliography page.
The project team made an effort to provide examples
(in text and video) and learning materials covering
the five different domains of Maastricht University:
Healthy body, healthy mind, Economics, business,
trade and management, International relations,
politics and law, Arts, literature and philosophy, and
Science and technology.
2.2.3 Working in Small Groups
Small group work is essential to PBL and, therefore,
also in this MOOC. In the first ‘pre-week’
participants are asked to fill out their profile, study
the preparation module, and form learning teams
(see Figure 1). There are three ways to get into a
team:
- Start a new team and invite others to join, e.g.
inviting people you already know, or looking for
other participants that have a similar
backgrounds or interests.
- Join an existing team, looking for an interesting
team (based on the team’s name, tagline, or
profile) or for other interesting participants that
you would like to work with (based on individual
profiles).
- Wait until you are automatically assigned to a
team at the end of the ‘pre-week’, based on the
chosen track and other preferences (only if your
profile is filled out).
All members of the team have to take the same
track. In principle, the teams stay together during the
whole course. However, since anyone can enter and
the course is free a large drop-out of up to 95% is to
be expected in a MOOC (Devlin 2013) Therefore,
some regrouping is foreseen. Teams that become too
small, i.e. do not have enough active members to
work effectively, are encouraged to merge with
different teams in the same track. As far as possible,
the facilitators try to track down inactive teams and
to provide suggestions for merges.
Figure 2: Three different tracks in the MOOC Problem-Based Learning: Principles and design. Students at the centre.
Problem-Based Learning in a MOOC - Exploring an Innovative Instructional Design at a Large Scale
371

Figure 3: Team space with public profile page and private chat facilities, file exchange and facilities to schedule meetings.
Figure 4: General facilities and networking options.
Intentionally
covered
CSEDU 2016 - 8th International Conference on Computer Supported Education
372

2.2.4 Peer Review and Evaluation of Team
Members
After each problem, i.e. at the end of every second
week, participants are asked to peer review the
products of three other teams. They are also asked to
rate their own contribution to the assignment and to
rate the contribution of their team mates on a scale
from 'No contribution' to 'Very devoted'. Filling out
this evaluation helps us to get insight in who is still
active and who is not, and to track down teams that
have become very small.
2.2.5 Platform and Tools
The MOOC Problem-Based Learning: Principles
and design. Students at the centre! has been
implemented in NovoED (https://novoed.com/)
mainly because this platform explicitly supports
small group work.
Each team can avail of a public profile page with
chat facilities and a private team space with chat
facilities, file exchange, and facilities to schedule
meetings (see Figure 3). However, teams can decide
themselves how they want to collaborate and
communicate, synchronously or asynchronously,
using the tools provided in their team space or
others, if they prefer.
2.2.6 General Facilities and Activities
The most important part of the MOOC is the small
group work on authentic problems, as described
above. There are some general activities, however,
illustrated in Figure 4:
- A set of mini-lectures about important aspects of
PBL which correspond roughly but not exactly to
the PBL problems in the different tracks
- General discussion fora accessible to all
participants, with some prespecified topics, but
also the freedom to start new threads.
- Regular Google Hangouts sessions by the
facilitators: sessions of 20-30 minutes where the
facilitators react on questions, elaborate specific
topics (e.g. related to the tasks of the week), react
to main issues in the discussion fora, or give
concrete tips for often encountered problems.
These sessions are recorded and made available
for those who could not attend live.
- Networking opportunities in NovoEd allow
participants to search for other participants based
on profile information and to contact them, and
to follow the public page of other teams.
3 PILOT STUDY
3.1 Method of the Pilot Study
A pilot study was conducted from May 12 2015 to
July 12 2015 (9 weeks). Participants were recruited
mainly among Maastricht University colleagues
using internal media like newsletters and e-mails
from faculty management. Some external contacts
reacted to an announcement on a mailing list for
lunch lectures. A few participants found the pilot
course via the NovoEd web address (although it was
never announced in newsletters or catalogues).
Three facilitators were involved in the pilot study.
They responded to questions, and organized three
Google Hangouts sessions.
During the pilot study the following data were
collected:
- Log data,
- Results of two questionnaires: one after 4 weeks
and one after the course had finished,
- Assignments handed in by the team,
- Messages on the general discussion fora,
- Experiences of the facilitators, and
- Verbal feedback from three participants.
The data were analysed by members of the project
team involved in the evaluation work package, and
subsequently discussed during a plenary meeting
with the facilitators and representatives of other
work packages. Statistical analyses on log data and
questionnaire results are not reported here given the
low number of participants at the end of the course.
3.2 Results of the Pilot Study
3.2.1 Participation and Drop-out
As expected there was a large drop-out rate. The fact
that the end of the course fell in the start of the
summer holidays may have been an extra factor. A
total of 104 participants subscribed to the course as
students and an extra 23 subscribed as ‘auditors’
(which in NovoEd gives them access to the course,
but not the right to join a group and work on the
assignment). Of these students 35 joined one of the 7
teams that were formed. Automatic team formation
was not used because only 5 people had not joined a
team at the end of the ‘pre-week’. They were
contacted individually giving a suggestion of the
team they could join. At the end of the course 19
participants (18%) were awarded a Certificate of
Participation.
Problem-Based Learning in a MOOC - Exploring an Innovative Instructional Design at a Large Scale
373

3.2.2 Content and Structure of the Course
In general, the PBL tasks seem to have been
understandable and suitable. The assignments that
were handed in showed that groups were discussing
the topics that we wanted them to discuss. In the
questionnaires, the discussion fora and the verbal
feedback, participants were positive about the
assignments and the provided self-study materials.
The different tracks seemed to be appreciated,
but the fact that all assignments of all tracks were
always visible on the assignment page caused some
confusion.
There discussions on the general discussion fora
were quite active early in the course. Later on, there
was less discussion, presumably because the number
of active participants had dropped. Participants
interacted with the facilitators and with each other,
but discussions were not always placed under the
most logical header. They just seemed to continue
wherever they had started. Some of the discussion
was dedicated to online PBL, maybe because most
participants in the pilot study were staff members of
Maastricht University and already had ample
experience with face-to-face PBL.
Unfortunately, it was very hard to detect which
participants were active or inactive, because the
platform only provides data at the team level and
facilitators have no access to the team space of the
teams. There were some other specific issues related
to the interface of the NovoED platform, which we
will not report in detail here.
3.2.3 Working in Small Groups
Observations and inspection of assignments and
discussion fora showed that teams worked in
different ways, working on the assignments
seriously or at a more superficial level. Some groups
put a slightly different focus on what they discussed,
e.g. focusing on online PBL. That is not unexpected
in a MOOC, were participants can have different
backgrounds and interests and, therefore, different
learning questions. It may have caused some barriers
in the peer review, because the participants could not
always follow what another team had done. We also
observed that the peer reviews tended to be just an
assessment using the sliders, without any comments.
The evaluation of team members’ contribution was
skipped by most participants. When asked, the
participants that we spoke to said the task was easily
overlooked in the interface and its function was not
clear to them.
Another important observation, based on
questionnaire data, verbal feedback and questions to
the facilitators, was that some groups found it
difficult to start up because there is no prescribed
way to collaborate or communicate. Some students
remarked that they needed more information up
front about PBL and the assignments in the course,
and about role division and online collaboration in
teams. Participants explicitly asked for a clear
scheme with all activities and deadlines.
3.3 Changes to the MOOC Design
Based on the results of the pilot study no major
changes were made in the content of the course. To
reduce confusion the project team decided to provide
an explicit time schedule for each track and to open
up assignments gradually, so that not all of them are
visible at the start. The self-study resources were
added to both parts of each assignment to avoid
unnecessary navigation.
For the peer review a different format was
chosen with open questions in a grid format where
participants are asked to briefly describe the
strengths and weaknesses, to explain what was not
clear and to give some suggestions or new ideas.
The explanation of peer review and evaluation of
team members’ contribution was improved.
In the ‘pre-week’ an extra assignment was added.
The newly formed teams are asked to fill out and
hand in a Team Charter, asking them to divide roles
(who will lead, who will plan, who will hand in
assignments) and rules for collaboration. In the team
charter the teams also describe how they will
communicate (synchronously or asynchronously)
and which tools they will use. Some open questions
at the end ask them to discuss how they will deal
with unequal participation and lurkers, unwanted
behaviour, etc. The text of the team charter
assignment also gives more elaborate information
about the PBL-assignments in the course and about
what they will be expected to do in the course,
giving a few examples of how they might want to
work, rather than being prescriptive.
To stimulate and structure the discussions on the
general fora the project team decided to reduce the
number of fora and to stimulate participants to make
new threads with clear names themselves. The
facilitators should try to keep an eye on the
discussions and ask participants to move to a
different forum if that seems more suitable.
CSEDU 2016 - 8th International Conference on Computer Supported Education
374

4 THE FIRST REAL MOOC:
FIRST IMPRESSIONS
The MOOC Problem-Based Learning: Principles
and design. Students at the centre! started on
October 5 2015 and lasted until December 12. The
execution of the MOOC was advertised using the
NovoED catalogue and newsletter, Maastricht
University communication channels including
newsletters and alumni networks, a press release,
and social media like Twitter and LinkedIn, using
both personal contacts from project team members
and joining groups related to MOOCs. There are
four facilitators in the course (see Figure 5).
First impressions are reported here since the
analysis of data is not completed yet. The MOOC
started with 2989 subscribers: 2653 students and
336 auditors. Just over a quarter (26%) filled in their
profiles and became part one of the 111 teams.
There is an overrepresentation of teams following
Track 2. Tracks 1 and 3 are less popular. The
Google Maps in the forum ‘Introduce yourself’
shows that they are from all over the world with
concentrations in Europe and South America (see
Figure 6).
The majority of teams was formed by the
participants themselves (98 teams) and 13 teams
were formed automatically at the end of the ‘pre-
week’ from participants who had filled in their
profile but not joined a team yet. Some of the self-
formed teams are region-based, or even formed by
colleagues from the same institute. Others formed
around a certain area of interest, such as professional
education or language teaching. These interests also
come back on the heavily used general discussion
fora where hefty discussions take place about a
variation of topics, ranging from ‘what does a good
tutor do?’ to ‘can I use PBL for mathematics,
primary school children, disadvantaged students,
etc.’
49 of the 111 teams finished the course (i.e.
handed in the last assignment) and 264 participants
were received a certificate of participation. It
became clear very quickly that self-formed teams
function better than automatically formed teams.
Most of the 13 automatically formed teams never
even handed in the first assignment. Especially at the
beginning of the MOOC contributions to the
discussion for a showed that some people found it
difficult to join a team.
Inspection of the assignments shows that the
quality of the assignment varies, but that, in general,
the teams have followed a PBL like process
(brainstorming, formulating learning questions,
reporting and discussing results). The project team
members who were responsible for writing the
problems were surprised by the quality of a large
part of the assignments.
Discussions on the fora and inspection of the
assignments also show that teams collaborated and
communicated in very different ways. For some
teams it took time to find a good way to collaborate,
and some teams clearly struggled. Other teams
seemed to have no trouble to establish a way of
working and showed great creativity in the tools and
methods they used. The project team has observed a
number of teams in more detail, following the
interaction between team members in their team
space by joining their team as an observer (after
informed consent). The results are currently being
analyzed.
Figure 5: Flyer page of Problem-Based Learning: Principles and design. Students at the centre.
Problem-Based Learning in a MOOC - Exploring an Innovative Instructional Design at a Large Scale
375

Figure 6: Participants of Problem-Based Learning: Principles and design. Students at the centre.
5 CONCLUSIONS
Designing and running the MOOC Problem-Based
Learning: Principles and design. Students at the
centre! has shown that it is possible to apply some
aspects of PBL in a MOOC: constructive and
contextual learning centred around authentic
problems, and collaborative learning by asking
participants to work on these problems in small
teams. For a MOOC this is an innovative design, and
distinctly different from the usual designs for
xMOOCs and cMOOCs.
Whether this MOOC can be called PBL remains
questionable; it differs from more traditional forms
in some important aspects, such as the absence of a
tutor and very limited amount of feedback and
support.
Online collaboration in virtual teams remains a
challenge, and requires dedicated support. A large
drop-out is to be expected, and not all teams will
succeed. More research into factors determining
team success or failure is required.
ACKNOWLEDGEMENTS
The authors would like to thank the entire project
team of the MOOC project.
REFERENCES
Ahn, J., Butler, B.S., Alam, A. & Webster, S.A. 2013.
Learner participation and engagement in open online
courses: insights from the Peer 2 Peer University.
MERLOT Journal of Online Learning and Teaching,
vol. 9, no. 2. Available from http://jolt.
merlot.org/vol9no2/ahn_0613.htm.
Barrows, H.S. 2002. Is it truly possible to have such a
thing as dPBL? Distance Education, vol. 23, no. 1, pp.
119-122.
Barrows, H. S. & Tamblyn, R. M. 1980. Problem-based
learning: an approach to medical education. Springer:
New York.
Bussemaker, J. 2013. Open en online hoger onderwijs
(Brief aan de Tweede Kamer der Staten-Generaal,
Referentie 581269) [Open and online higher education
(Letter to the parliament, Reference 581269)].
Ministerie van Onderwijs, Cultuur en Wetenschap:
Den Haag, The Netherlands (In Dutch).
Devlin, D. 2013. MOOCs and the Myths of Dropout Rates
and Certification. Available from: http://www.
huffingtonpost.com/dr-keith-devlin/moocs-and-the-my
ths-of-dr_b_2785808.html (3-2-2013).
Dolmans, D.H.J.M., de Grave , W. , Wolfhagen, I.H.A.P.
& van der Vleuten, C.P.M. 2005. Problem-based
learning: future challenges for educational practice and
research. Medical Education, vol. 39, pp. 732 – 41.
Hollands, F.M, & Tirthali, D. 2014. MOOCs:
Expectations and reality. Full report. Columbia
University, Teachers College, Center for Benefit-Cost
Studies of Education: New York.
Holton, D. 2012. What’s the “problem” with MOOCs.
Available from http://edtechdev.wordpress.com/2012/
05/04/whats-the-problem-with-moocs/ (4-5-2012).
CSEDU 2016 - 8th International Conference on Computer Supported Education
376

Mackness, J., Waite, M., Roberts, G., & Lovegrove, E.
2013. Learning in small, task-oriented connectivist
MOOC: Pedagogical issues and implications for
higher education. The International Review of
Research in Open and Distance Learning, vol. 14, no.
4. Available from http://www.irrodl.org/index.php/
irrodl/article/view/1455/2531.
McAndrews, P. & Scanlon, E. 2013. Open Learning at a
Distance: Lessons for Struggling MOOCs, Science,
vol. 342, pp. 1450-1451.
Morrison, D. 2013. The Ultimate Student Guide to
xMOOCs and cMOOCs. Available from http://
moocnewsandreviews.com/ultimate-guide-to-xmoocs-
and-cmoocso/ (22-4-2013).
Moust, J., Bouhuijs, P. & Schmidt, H. 2014. Introduction
to Problem-Based Learning: A Guide for Students.
Noordhoff Uitgevers B.V.: Groningen, The
Netherlands.
Waldrom, M.M. 2013. Campus 2.0: Massive open online
courses are transforming higher education – and
providing fodder for scientific research. Nature, vol.
495, pp.160-165.
Yuan, L. & Powell, S. 2013. MOOCs and Open
Education: Implications for Higher Education (A
white paper). University of Bolton: CETIS: Bolton,
UK. Available from http://publications.cetis.ac.uk/
Worldwide Universities Network. 2013. Annual Report
2012-2013: Tackling Global Challenges through
International Collaboration. Available from:
http://www.wun.ac.uk/about.
Problem-Based Learning in a MOOC - Exploring an Innovative Instructional Design at a Large Scale
377