Learner Generated Content

Fostering and Valuing User Generated Content in eLearning using Social Feedback

Gabriel Reimers

Quality and Usability Lab, Technische Universit¨at Berlin, Ernst-Reuter-Platz 7, 10587 Berlin, Germany

1 RESEARCH PROBLEM

User Generated Content (UGC) is any form of con-

tent that is created by the users themselves in order

to make it available to others, usually on an online

platform. When UGC is used in a learning context

and learning material is created by students or learn-

ers in general, we talk about Learner Generated Con-

tent (LGC) as a subset of User Generated Content.

A very common form of LGC are Wikis, but the

concept is not limited to text generation. The term

applies to creative works on all media. For exam-

ple creating videos for YouTube, virtual worlds for

Minecraft or sharing pictures on Flickr, all are forms

of Lerner Generated Content when performed in an

educational context.

Having learners create and share content can be

advantageous in several cases:

• creation of large collections of content

• peer assessment of content

• motivation of learners

When applying the concepts of User Generated

Content, questions in several areas arise:

1.1 Acceptance of LGC

As UGC lives from its users, it is crucial to get

high acceptance by the users. That means, the plat-

form with learner generated content must be accepted

among learners as a valuable source for material, and

a critical mass of learners must be willing to create

and contribute content to the platform.

So the questions are:

• How can users be motivated to actively contribute

LGC?

• How big is the critical mass of contributors?

• What criteria must be fulﬁlled so users consider a

LGC platform as valuable?

1.2 Measuring Quality

Having lots of users create content is only half the bat-

tle. A learner browsing the repository of learner gen-

erated content must be able to quickly judge which

contributions are valuable to her and which are not.

As there is no central quality control, a community

based system is needed to sort, categorize and rate

each content contribution.

Problems are:

• What do learners consider relevant criteria for

quality?

• What forms of rating systems are possible and ac-

cepted by users?

• How do LGC ratings compare to evaluation by an

expert/teacher?

1.3 The Role of Social Feedback

Some UGC platforms, like Wikis, have no kind of re-

ward or point system. Others are strongly gamiﬁed,

like Waze

, where every contributor is awarded points

by the system.

StackExchange.com

is also gamiﬁed, but points are

not received from the system but from other users. As

users rate content, the creators of that content receive

or lose points depending on the rating.

The author believes that such a system of peer as-

sessment can not only be used to efﬁciently rate con-

tent, but is also a strong factor of motivation. Getting

social feedback by other users is a completely differ-

ent experience than contributing without recognition

(as in Wikis) or receiving automatic, and often mean-

ingless, rewards by the system.

The Self Determination Theory (SDT), developed

by Deci and Ryan, points out three important require-

ments for intrinsic motivation: Autonomy, Compe-

tency and Relatedness. (Deci and Ryan, 2000) User

https://www.waze.com

http://stackexchange.com

Reimers G..

Learner Generated Content - Fostering and Valuing User Generated Content in eLearning using Social Feedback.

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

Generated Content appeals to autonomy as users are

free in how, what, when and whether they create con-

tent. It can also easily be used to foster competency

with any kind of rating system together with user

proﬁles displaying progress and level of contribution.

Now, adding direct social feedback and peer assess-

ment not only improves the accuracy of the rating

for the competency, but also adds aspects of a social

network. As long as users are not completely anony-

mous, that adds the dimension of relatedness to LGC

and might increase intrinsic motivation of users.

However, it might well be that users perceive social

feedback as social pressure, or that social feedback

actually takes effect as extrinsic motivation, under-

mining intrinsic motivating.

Considering social feedback in learner generated

content leads to the following questions:

• Howdo users value social feedback in comparison

to system feedback?

• How need social feedback systems be balanced to

prevent abuse like spamming or bullying?

• Does social feedback yield higher motivation than

regular systems?

• Under what conditions do users consider social

feedback as neutral measure of their competency,

and when do they perceive it as controlling?

1.4 Effect on Learning Success

Collection and sharing of knowledge and content is

probably in most cases the driving reason to use con-

cepts of Learner Generated Content. However, having

a LGC platform is only useful if learners beneﬁt from

it. It is likely that actively creating content results in

better learning success. Making content available to

others should also help those consumers learn more

easily.

These assumptions remain to be proven, and the

following questions arise:

• Does creating content yield higher learning suc-

cess?

• Can the accessibility of LGC improve learning ef-

ﬁciency?

• Can social feedback make learners adjust and cor-

rect content they created?

2 OUTLINE OF OBJECTIVES

2.1 User Studies

The ﬁrst step is to develop a questionnaire that can

measure a users attitude towards a LGC in general and

towards speciﬁc platforms. That should cover aspects

like why and how often the platform is used, the con-

text in which the user is set, the willingness to create

and share content and especially the effect of social

connection and feedback on that platform.

The questionnaires will be based on interviews led

with users and on existing scales and concepts from

the literature like the Self Determination Theory.

2.2 Implementation

Off-the-shelf Wiki and Question & Answer (Q&A)

systems shall be set up and provided to students of the

courses supervised by the author. This is a cheap and

fast approach to collect impressions and questionnaire

results from students on Learner Generated Content.

However, it is important to not only investigate ex-

isting Learning Generated Content platforms. Testing

of new concepts and detailed tracking of users is only

possible when full access to the technical and edito-

rial components of the platform is guaranteed.

Accordingly, two platforms for LGC are to be devel-

oped within the context of this thesis.

One is PaperMesh, a platform allowing students

to upload and share scientiﬁc works they created dur-

ing their studies. For instance a student can upload

her bachelor thesis, seminar presentations or project

reports. These might be valuable, citable sources for

other students. A reliable rating and feedback system

will be essential for that platform.

The other planned project, PearRank, is an LGC

platform which can be used in context of courses to

peer review tasks. As a course instructor, one can cre-

ate rooms for speciﬁc topics, which are only acces-

sible to the (invited) course members. Within such a

room, students can post results of assignments in form

of pictures, videos or text. Any submission is visible

to all other participants, who are encouraged to rate

their peers’ submissions. This results in a ranking of

the works and small competition. The platform can

be used for open, creative assignments. For example,

students could be asked to take pictures of wild ani-

mals in their neighborhood and upload it to the LGC

platform. The contribution with the best peer-rating

can then be awarded a little price or bonus points. The

social feedback is expected to increase participation

and also have students retouch their submission if it is

not rated high enough.

2.3 Evaluation

Using the results and tools obtained during the user

studies, continuous iterative evaluation and adjust-

CSEDU2015-DoctoralConsortium

ment of the created platforms will be conducted.

The goal is to compare the effect of different feed-

back approaches on motivation and participation on

the platforms as well as measuring the effect on learn-

ing outcome of different LGC concepts.

3 STAGE OF THE RESEARCH

Currently, the development of appropriate question-

naires is ongoing. That includes performing inter-

views with representative users and building a model

of important factors inﬂuencing acceptance, motiva-

tion, contribution, quality and learning effect of LGC.

Development of PaperMesh, the LGC platform for

students to share scientiﬁc works, has started in win-

ter. An early beta version is expected for summer.

Also, a cooperation with the Berlin based startup

qLearning

has been established. The company is of-

fering a learning app, targeted at university students,

and wants to shift content creation to a learner gener-

ated model. The author consults in performing user

studies and will be granted access to study results and

usage data in return.

4 STATE OF THE ART

4.1 LGC Platforms

In classrooms and online courses, wikis and blogs are

a very common form of Learner Generated Content.

They can be used as a sandbox in which students sum-

marize what they learned or create articles on supple-

mental topics. Yet, wikis offer hardly any feedback

to contributors and don’t offer mechanisms to track

quality of content. Blogs usually have a comments

section but are very unidirectional.

To compare acceptance of Wikis with that of plat-

forms applying social feedback, the author will setup

off-the-shelf wikis for courses. Usage of these and of

other existing wikis will be evaluated as a baseline.

StackExchange.com is a collection of several

question and answer (Q&A) sites on topics such

as programming (stackoverﬂow.com), languages,

physics, math, cooking and many more. Questions

and answers are user generated, and a peer rating sys-

tem is used to rank questions and answers by qual-

ity. Users build up reputation points based on these

ratings and can follow a gamiﬁed path of levels and

badges awarded for their engagement. The StackEx-

change sites are very well implemented examples of

http://qlearning.de

how social feedback can be used to motivate users and

rank content by user voting.

While content is user generated, the Q&A tech-

nology behind this platform is held by stack exchange

inc. and is not publicly available. Accordingly this

platform cannot be customized or be used in closed

environments like classrooms. StackExchange plat-

forms can be used in institutional learning as a source

of information similar to Wikipedia. They are not pri-

marily designed or generally considered as eLearning

tools, though. Within this work, it is planned to test

the applicability of the Open Source Question An-

swer system (OSQA) in a course context. OSQA is

an open, but no longer maintained, reimplementation

of the StackExchange platform. It shall be measured

if social feedback in such a Q&A system yields better

acceptance or engagement than wikis.

On Graasp.eu

users can create private rooms for

their course or group. Within these rooms users pub-

lish text, images, videos or links and can rate other’s

content. It is intended for use in education contexts,

and its creators of EPFL Lausanne describe it as “a

social media platform, to setup a peer assessment ac-

tivity”. Graasp lacks any form of user proﬁles or gam-

iﬁcation and can be considered a content sharing plat-

form with ratings.

As part of this thesis the peer ranking platform

PearRank will be created, which will offer a similar

room based system as Graasp. In contrast to Graasp,

PearRank will put the social interaction and ranking

in focus. Graasp is more like a portfolio platform on

which users collect and share complementary infor-

mation. PearRank shall be more competitive as all

users upload similar content in response to a speciﬁc

task and vote to select the best.

4.2 Research

Wikis have been used and evaluated in various learn-

ing contexts and have become and essential part of

learning platforms such as Moodle. Some small stud-

ies have been done on the use of Wikis in higher edu-

cation courses, reporting positive feedback from stu-

dents, higher engagement and more social interaction

among participants. (Coutinho and Bottentuit Junior,

2007) (Wheeler et al., 2008)

Others argue, in these studies participation was

only achieved by pressure of points or grades, and

that students would not contribute voluntarily. (Ebner

et al., 2008)

Hardly any data is available on what students who

freely create content in Wikis or Blogs, or on how

those who don’t, can be motivated.

http://graasp.eu

LearnerGeneratedContent-FosteringandValuingUserGeneratedContentineLearningusingSocialFeedback

Q&A platforms, especially StackExchange, have

been researched, too.

At Trinity College Dublin the, now ofﬂine, ex-

plore.su analysing tool has been developed. It visu-

alizes user demographics, social interaction and edit-

ing behavior of the SuperUser.com platform (a child

site of the StackExchange ecosystem, for computer

enthusiasts and power users). (McAuley et al., 2012)

Anderson et. al. conducted a detailed big data

analysis on the complete usage data of the StackOver-

ﬂow.com platform and gave insights on how the pro-

cess of question answering takes place and on how

rating of questions and answers can be predicted.

(Anderson et al., 2012)

Both projects conclude that users with higher rep-

utation are more active and that distribution of repu-

tation resembles a pyramid with few very reputable

users and many rather inactive users.

A very interesting insight into the effect of social

feedback on user behavior is given in (Cheng et al.,

2014). Commenting on four major news communi-

ties was investigated, and it was found that users who

received negative voting on their contributions (com-

ments) produced more and worse comments after-

wards. Commentors who received positive feedback

did not change their behavior, whereas those who did

not receive any feedback contributed less afterwards.

It remains to be shown that this pattern can also

emerge on platforms that are more knowledge driven

and not as emotional and political as news comment-

ing is.

5 METHODOLOGY

To obtain real world data, four LGC platforms will

be analyzed in detail. That is the custom peer rating

platform PearRank, the student paper sharing plat-

form PaperMesh, off-the-shelf wikis and Q&A, and

the learning app created by the qLearning startup.

For each of these platforms users will be surveyed

with questionnaires about their usage of LGC and

why they create content or not. Log data will be col-

lected to neutrally measure participation and behav-

ior. Where feasible learning outcome will be mea-

sured on students of TU Berlin in parallel to regu-

lar student assessment. Additionally, learners will be

asked to provide grades or subjective impressions of

their learning success.

While it would be desirable to see positive effects

of LGC combined with social feedback on learning

outcome and motivation, it certainly is not the holy

grail of learning. Therefore, the null hypotheses re-

mains that LGC has no effect on learning outcome,

and social feedback has no effect on motivation or

participation. It might well be that other factors, such

as design, ease of use or personal preferences have

much stronger inﬂuence on how users interact with

LGC.

6 EXPECTED OUTCOME

The thesis will provide an overview of existing con-

cepts for Learner Generated Content. A selection of

these concepts will be analyzed on how well they are

accepted by users, how correctly their content is cate-

gorized and rated, and what effect they have on learn-

ers motivation and success.

As a result, certain concepts might proof more effec-

tive than others and factors having strong inﬂuence on

acceptance and motivation can be isolated.

A special focus will be put on the effect of social

feedback in terms of peer assessment, voting, rating

or ‘liking’. It is expected to positively affect users’

willingness to contribute content and increase their

intrinsic motivation to learn the topics covered by the

LGC platform. It shall also be shown that user ratings

— in any form — are an ideal tool to measure quality

of learner generated content and can be used to rank

search results on LGC platforms.

Motivation and reasoning of LGC users will be ex-

plored. Factors making users contribute will be iden-

tiﬁed, as well as factors that hold users back from par-

ticipating.

It will be investigated if there are any deﬁnable groups

of users that are more willing to contribute content

than others. The author assumes that users of UGC

platforms can be characterized by their motivation to

produce or consume content and their disposition to

be social or altruistic. If users can clearly be cate-

gorized, LGC platforms will be able to address users

more concretely.

A toolset of questionnaires and metrics will be

developed to measure motivation, participation and

learning success on LGC platforms. This toolset shall

be as platform independent as possible so it can be

used to by others to investigate different platforms as

well.

While several aspects inﬂuencing LGC accep-

tance and success will be uncoveredwithin this thesis,

many more will remain or arise. An important goal of

this work is to be a basis for future research to build

upon.

CSEDU2015-DoctoralConsortium

REFERENCES

Anderson, A., Huttenlocher, D., Kleinberg, J., and

Leskovec, J. (2012). Discovering value from com-

munity activity on focused question answering sites:

a case study of stack overﬂow. In Proceedings of

the 18th ACM SIGKDD international conference on

Knowledge discovery and data mining, pages 850–

858. ACM.

Cheng, J., Danescu-Niculescu-Mizil, C., and Leskovec, J.

(2014). How community feedback shapes user behav-

ior. arXiv preprint arXiv:1405.1429.

Coutinho, C. P. and Bottentuit Junior, J. B. (2007). Collab-

orative learning using wiki: A pilot study with master

students in educational technology in portugal.

Deci, E. L. and Ryan, R. M. (2000). The ’what’ and

’why’ of goal pursuits: Human needs and the self-

determination of behavior. Psychological inquiry,

11(4):227–268.

Ebner, M., Kickmeier-Rust, M., and Holzinger, A. (2008).

Utilizing wiki-systems in higher education classes: A

chance for universal access? Universal Access in the

Information Society, 7(4):199–207.

McAuley, J., O’Connor, A., and Lewis, D. (2012). Explor-

ing reﬂection in online communities. In Proceedings

of the 2nd International Conference on Learning An-

alytics and Knowledge, pages 102–110. ACM.

Wheeler, S., Yeomans, P., and Wheeler, D. (2008). The

good, the bad and the wiki: Evaluating student-

generated content for collaborative learning. British

journal of educational technology, 39(6):987–995.

LearnerGeneratedContent-FosteringandValuingUserGeneratedContentineLearningusingSocialFeedback