Human-centered Design Model in the Development of Online

Learning Tools for International Security Training

CASE IECEU New Media based Learning Application (NMLA)

Kirsi Hyttinen

Research and Development Unit, Laurea University of Applied Sciences, Vanha maantie 9, Espoo, Finland

Department of Information Technology, Cognitive Science, University of Jyväskylä, Finland

Keywords: Human-centered Design, Human-technology Interaction, Technology Solutions, Learning Environments,

Online Learning, e-Learning, Peacebuilding, Crisis Management, Security Training, International Security.

Abstract: A need to enhance online training solutions among peacebuilding and conflict prevention actors in the field

of international security has been addressed in order to raise the collaboration, information sharing and

provide more effective training. The implementation of educational technologies is inherently difficult

especially if it is technology driven. This paper presents the idea of human-centered design approach in line

with the principles of human-technology interaction in order to tackle the possible training challenges that

may occur in the development and implementation of online and Elearning environments for adult students.

As part of research and development project, the design case for peacebuilding and crisis management

online-based training was conducted and studied. The case study showed that before providing the

technological solutions there is a need in dept analysis on the requirements as well as iteration of

algorithms. Moreover, functionality and completeness of the instantiations, such as services, raised the

motivation among the user community. The created human-centred design model for the learning

environment development processes supports to meet not only with current training needs in security but

also in identifying which parts of education and training can be facilitated by technology.

1 INTRODUCTION

Department for International Development (2005)

listed 46 countries considered fragile states, in which

870 million people, or 14% of the world’s

population live and 250 million children worldwide

has no access to education (UNESCO, 2014). The

new threats from weak states, asymmetric conflicts,

organised crime, and terrorism, traditional

peacekeeping has frequently given way to complex

peacebuilding in protracted conflicts. These kinds of

global challenges and threats have led to the

complexity of crisis and conflicts. It has been argued

that international community needs to focus more on

strengthening the emerging fragile or unstable state

with preventive activities in early stage (IECEU-

project, 2016. D1.2). The Lisbon Treaty highlights

conflict prevention as key objective for EU’s foreign

policy and external relations (Treaty of Lisbon,

2009). The conflict prevention has been part of EU’s

Common Foreign and Security Policy (CFSP) and

the development of European Security and Defence

Policy (since 2009 the Common Security and

Defence Policy, CSDP) with the aim of the EU to

take action through its capacities in conflict

prevention and crisis management. The complexity

of EU engagement and current conflict and crisis

areas requires continuously assessment on the

effectiveness of different conflict prevention and

crisis management activities (see for example

European Parliament, 2012) also training and

technology. The development of the international

community’s perception has changed towards

international security, shifting from robust

peacekeeping towards the stabilization of fragile

states via conflict prevention and peacebuilding.

(Morton and O’Hagan, 2009).

At the same time, around 80% of the world’s

population will have mobile connectivity and 60%

will enjoy broadband access. Like any use of

technology in the classroom, there is a need to unify

content, technology and pedagogy (Mishra and

Koehler, 2006). The digital possibilities enable the

Hyttinen K.

Human-centered Design Model in the Development of Online Learning Tools for International Security Training - CASE IECEU New Media based Learning Application (NMLA).

DOI: 10.5220/0006559902750282

In Proceedings of the 9th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (KMIS 2017), pages 275-282

ISBN: 978-989-758-273-8

learning distribution around the globe online (Zhang

et al, 2004, p. 75-79) and already in the beginning of

century the online learning as a field has been

described as complex, diverse, and rapidly evolving

(Anderson and Elloumi, 2004). Technology has a

significant influence on teaching and learning

processes and the technology has re-organized the

human life, communication and learning (Siemens,

2004). Due to current trends, Information

Technology (IT) tools and resources have significant

influence on teaching and learning processes

(Sevillano-García and Vázquez-Cano, 2015. p. 106-

118). With its potential for providing flexible access

to content anywhere and anytime, it has become

popular worldwide (IECEU –project, 2016. D5.2)

and can benefit local counterparts as well as

peacebuilding organisations. Information and

Communication Technology (ICT) can provide the

access to education and enhance the ability of minor

and vulnerable groups to attend to education and

training (Gulati, 2008).

Even with the opportunities that technology can

provide, the researchers have seen the implementa-

tion of technology-supported collaborative learning

environments as a challenge (Zheng, 2014. p.357).

In addition, there is a need for academic practice-

based design research when developing learning

tools (Leinonen, 2010). Like any use of technology

in the classroom, there seems to always be a need to

unify content, technology and pedagogy (Mishra and

Koehler, 2006). The Web 2.0 offers features, which

are not always considered with human learning

processes. Moreover, the flexible media options of

digital environments are not currently fully

benefitted in e-learning contexts (Clark and Mayer,

2016). It has been argued that the online learning

should move beyond from traditional Learning

Management Systems (LMS). Sevillano-García and

Vázquez-Cano (2015) argue that “institutions need

to orientate methodologies toward the use of new

mobile devices, from the possibilities offered

primarily through open educational resources

(OERs) distributed on wikis, blogs, mash-ups,

podcasts, social software, virtual worlds, personal

learning environments (PLEs), massive open online

courses (MOOCs), and other emerging online

practices”.

These identified needs, challenges and

recommendations provide the basis for this research

and online learning application case study. Human-

Technology Interaction (HTI) with its broadest sense

covers forms of interaction between technical and

human interfacts and includes all roles in to the

design process (Saariluoma et al., 2016, p.2). This

approach could be benefitted in the online training

of peacebuilding and conflict prevention.

1.1 Objective and Purpose

Overall purpose of this paper is to analyse the

learning technology that could be utilized for the

peacebuilding and international security training. In

order to tackle the challenges with learning

technology, this study aims to provide the

understanding between human and learning

technology interaction. This study provides the

analysis of the findings in the design and

development of New Media based Learning

Application (NMLA) during Improving the

Effectiveness of the Capabilities in EU conflict

prevention (IECEU) –project in years 2015-2017.

Finally, this study provides a human-centered design

model for the development of Elearning solutions

and information sharing especially for adult

education purposes in the field of international

security, peacebuilding and conflict prevention.

Ideally this will finally lead to research on

successful implementation of online training

initiatives in the field of international security and

peacebuilding. The research purpose is to support

the educational capacity building that concentrates

to the societal education structures and enhancing

the protection and successful delivery of education

in online basis according to the principles of

effective human technology interaction.

2 HUMAN-TECHNOLOGY

INTERACTION IN CHANGING

LANDSCAPE OF TRAINING

The technology has re-organized the human life,

communication and learning (Siemens, 2004). In

practice, the learning takes place partially or entirely

over the Internet. With its potential for providing

flexible access to content anywhere, it has become

popular worldwide. The computers made the

delivery of education possible and the material were

able to deliver both print and electronical media.

(Moore, 1990). The online learning includes

different sets of learning applications, web

resources, web-based applications and new

collaboration technologies. Moreover, new type of

hybrid approaches regarding online learning

activities are increasing. (Means et al, 2009). The

term distance learning evolved towards online

learning, virtual learning, e-Learning, mediated

learning, web-based learning etc. (Conrad, 2002, p.

1-19). In the European countries especially tablets

and smartphones are considered as an important

development on the field of education (Eimeren and

Frees, 2012). From the pedagogical point of view,

the higher education institutes benefits of e-learning

e.g. by revision of teaching methods, monitoring

study progress and student learning, interactive

collaboration among students and by enhancing

learning and teaching in foreign languages (Gaebel,

M. and al. 2014). In 21 century, the theoretical

insights have raised the collaborative practices in

online learning (Siemens, 2004). Dalsgaard (2006)

argues that social software tools can support a social

constructivist approach to e‐learning by providing

students with personal tools and by engaging them

in social networks, thus allowing learners to direct

their own problem‐solving process. He points out

that e-learning should move beyond from Learning

Management Systems (LMSs).

The technologies in digital context enable new

ways of narrating contents. The users are actively

involved to building the content to different media

and channels. (Katz, 2013. p. 129-133). It has been

also argued that collaborative learning tools can be

used from both a cognitive constructivist and social

constructivist perspective (Bonk and Cunningham,

1998, p.35). It has been issued that the design of

learning environments can be based on the learning

objective, target audience, access (physical, virtual

and/or both), and type of content (Moore and

Dickson-Deane, 2011. p. 129). Different technology

applications are used to support different models of

online learning. Ideally, the online learning

components are combined or blended with face-to-

face instruction, in order to provide more learning

outcomes. (Means et al, 2009). The engagement in

Web 2.0 environments provides more avenues for

self‐representation, expression or reflection and

more organized forms of collaboration and

knowledge building. Web 2.0 tools can support

associative pedagogies and be used effectively in

terms of providing structured guidance through tasks

and through provision of effective and timely

feedback. (Conole and Alevizou, 2010). New media

and technology resources are based on principles of

mobility, collaboration and active participation.

Ubiquity and mobility become recurrent principles

for educational performance in this century.

(Sevillano-García and Vázquez-Cano, 2015).

The overall purpose of human-centered design is

on making systems or applications easy to use (ISO,

1999). Jounghyun (2015) defines the goals of

Human–Computer Interaction (HCI): (a) functional

completeness, (b) high usability, (c) aesthetic appeal

and (d) compelling user experience. Firstly, the

research interest can focus on technology-driven

design with emphasis on products, systems or

interphases. Secondly, the human-center or human-

driven approach is seen as important as technology-

driven. Further, the Human-Technology Interaction

(HTI) research aims to support the design and

develop usable technology for humans. The most

emphasied aspect of HTI has been that difficult-to-

use technology is easily rejected and weather people

find it easy to use the systems and products

(Saariluoma et al, 2016) and people should know

how to use the products, services or systems. (Leikas

2008). Despite the early focus of HTI has been how

to design interaction and implement interfaces for

high usability, it has been used for specific user

community. The critical components of successful

integration of technology innovations within

education and training settings and influences the

adoption rate of such technologies are transparency

in user interface design (Charalambos et al, 2004).

3 METHODOLOGY

In the research and development program of IECEU

(H2020) –project, the methodology concept and

quality assurance system for technology systems

were created by co-created methods in early stage of

the project, in the year 2015. The group of

developers, IT specialists and researchers worked

together during years 2016 and 2017 and design

process included the engagement of end users.

This particular study followed the principles of

Design Science Research (DSR) approach and

focused on the HTI during the design and

development of online learning solution for EU

conflict prevention and international security

training. DSR approach has been used for develop

and evaluate IT artifacts in order to understand,

explain and improve them. Artifacts within DSR are

constructs (vocabulary and symbols), models

(abstractions and representations), methods

(algorithms and practices) and instantiations

(systems and services). (Hevner et al., 2004). This

paper analyses the design and development process

of IECEU New Media based Learning Application

(NMLA) and findings from the data collection by

end users. The overall framework of NMLA

included the selection of pedagogical approach,

technology in design of NMLA, understanding of

new media interactive solutions and intercultural

competences.

The selected user community representatives

participated to participatory learning application

testing and provided the self-assessment on the

learning and evalauted the application interface. The

test groups consisted of both men and women of

different age. The majority presented experienced

academic researchers from different disciplines. The

participants were divided into three groups: group of

educators, professors and trainers (n=5), a group of

end user experts (n=14) and higher education

students (n=43). Firstly, each participant tested the

NMLA with use of device (mobile, iPad or laptop)

and answered to the survey. The survey included

multi-choise and open questions and the results were

analysed through quantitative and qualitative

methods. Secondly, the discussions around user

experiences were facilitated. The survey gathered

the first impressions and user experiences. The

discussions focused on analysing the learning

objectives and relevance of the NMLA with end user

needs in peacebuilding training.

The researchers analysed the interaction between

users and the different devices (PC, laptop, mobile

devices) based on research findings and

observations. The findings were further tested

(triangulation) in the discussions with the selected

key representatives in order to further improve the

learning application. The collected information was

then assessed and fed into development process

3.1 Design Case: IECEU Learning

Application

The purpose in the NMLA was to transfer the

original IECEU (Improving the Effectiveness of

capabilities in EU conflict prevention, H2020) –

project research findings and identified good

practices in the online format. In order to the learn of

key competences needed in the field of crisis

management and peacebuilding, the NMLA was

developed. The objective of the tool was primary to

exhibit the selected eight IECEU case studies and

further disseminate the findings of the comparative

analysis. Secondly, the development aimed at

creating an interactive training tool which could be

used in various crisis management training or as a

basis to introduce the new Common Security and

Defence Policy (CSDP) personnel to the roots and

dynamics of different conflicts either in their future

deployment areas or in general. During the

development process it was identified that the tool

can also be used among the larger audience of

stakeholders and other communities of interest (civil

servants, trainers, higher education, students etc.)

The IECEU NMLA offers a complimentary tool to

enhance the knowledge of a broad end-user audience

on themes related to selected crisis case studies and

crisis management decision making and planning.

The term new media was used to differentiate

between old media (print press, TV, radio) and the

new information technology based media. The

learning content is also supported by the social

media and rich digital contents as selected

algorithms. These algorithms often include

interactive elements and possibility to communicate

and provide feedback. Examples of new media

include websites, games, augmented reality,

multimedia and various learning platforms. (IECEU,

2017, D5.2). This also because in spite of the

presented benefits Eearning can provide to various

training, “digital tools do not substitute face-to-face

methods, but complement them by enabling more

constant interaction with users and lowering certain

users’ participation thresholds.” (Friedrich, 2013).

Figure 1: The framework of IECEU NMLA (IECEU,

2017, D5.2)

3.1.1 Instantiation: The Context for Service

The selection of service as part of instantiation

process was clarified after the analyses on the

context of use in early stage of design and

development. The project reseachers and developers

interviewed, made observations of the context and

participated to project workshops in order to build

comperehensive picture of the context. The special

attention was addressed to functional completeness

in this solution design process. Moreover, the

criteria for the solution selection was that the

platform needed to based on open source code

featuring desired functions: options for gamified

learning, mobile friendly (meaning a website or web

application that displays correctly in mobile

devices), multi-browser compatible (meaning that it

must be displayed correctly in multiple browsers).

The platform can be easily gamified with artifacts

such as badges, user scoring, certification awards

and H5P module integration, all of which are pre-

installed in the package or obtainable via the free

app store. H5P is an extension developed for

WordPress, Moodle and Drupal from a separate

organization that is used to gamify Elearning with

ease, all done in HTML5, including video quizzes.

The NMLA, which can also be downloaded to

laptop, tablet or mobile phone, could be utilized as a

part of the training conducted in fragile and crisis

areas. The IT instantiations were identified by the

developers. The service as instantiation was selected

with in depth understanding of the context, end users

and other requirements.

3.1.2 Construct Framework: Vocabulary

and Symbols

After instantiation, the key end users and

stakeholders of NMLA were analysed by the project

expert group. The IECEU Advisory Board provided

further guidance and support to understand key

training and education providers that may benefit

with the use of learning application in the future.

IECEU NMLA focused on European security and

defence network training initiatives as well as higher

education institutes and university. The research of

IECEU built the overall constrcuts for the NMLA in

line with end users’ vocabulary and symbols. The

learning objectives and outcomes were set according

to the needs and gaps identified during the IECEU –

project research. The end users group discussions

supported the identification of current training needs

that technologies could support. Learning objectives

are regognised among the training and education

community and they provided the contruct

framework for NMLA.

3.1.3 Quality Assurance Requirements

The requirements for the NMLA were also set in

IECEU Quality Assurance plan according to

technology development. At this phase, the designer

analysed wheather the instantiation, constructs and

representations were in line with quality assurance

requirements.

3.1.4 Iteration of Methods (Algorithms and

Practices)

There is a possibility to organize the technology

supported learning and training in decentralized or

centralized way (IECEU -project, 2016. D5.2). The

methods were agreed by the designers with

understand of algorithms and practices. It was

decided to use cartoon like visualisations from one

hand to keep the developing and programming time

manageable and on the other hand rather flexible to

accommodate any possible changes. Also the field

of international security and peacebuilding support

the use of animations rather than authentic pictures

or videos. Although the serious topic of crisis

management is linked to human suffering, the

visualisations were preferred to keep light. Finally,

perhaps most importantly, peacebuilding and crisis

management topics are often sensitive.

3.1.5 Models: Transfer of the Content

The research studies and analysis (content) were

transferred to rich digital online learning content.

Instead of written texts, the learning content finally

included more visual solutions such as videos,

animations, gamified quizes, and other more visual

material. These representations of information

provided the key models for NMLA design.

3.1.6 Evaluation and Assessment

In order to guarantee a systematic and comprehensive

assessment of the NMLA and its impact, IECEU

Consortium representatives conducted different types

of evaluation that correspond to the four levels:

NMLA evaluation by the producers, NMLA evalua-

tions by the users, In- and Out-Tests for the users and

possibly a NMLA Impact Evaluation Mission (IEM).

The evaluation of NMLA against the requirements

(needs, gaps, curriculas) included the analysis based

on the end users and quality assurance review. The

end user groups finalised the testing, completed the

individual survey and attended to group discussions.

The data collection led to analysis to further improve

the NMLA. The most important step from the

pedagogical point of view in the development process

of NMLA was to test and assess the learning out-

comes when using the tool in trainings and education.

Before and after the use of NMLA, their knowledge

and skills were tested to identify the first learning and

development. The final learning outcomes will be

further identified in the real life context.

4 FINDINGS

Firstly, the findings of the case study interviews of

experts working currently in the conflict setting

(IECEU Case Studies) corroborated the assumption

that up until now eLearning and learning applica-

tions have been not been broadly and systematically

utilized in the peacebuilding training. The main

shortfalls in the EU conflict prevention and

peacebuilding training focus on information sharing

and the content of Elearning modules. The general

findings of the NMLA design elaborated

experiences of NMLA use by end users. Especially

the evaluation and assessment phases gave valuable

feedback for the NMLA design process and its

further development. The survey findings identified

the benefits and shortfalls of initial version of

NMLA by end users. The introduction of the NMLA

(landing page) paid crucial role in motivation of the

users. The higher motivation raised the level of

continuing the use of service. The users described

the functionality and completed interface of NMLA

to support their active use of service. For the master

level higher education students the content was seen

rather advanced and hard to understand. It was

identified that higher education students were not

identified as key end users in construct framework.

The qualitative analyses pointed out that experts

experienced positively the rich media contents,

especially visualisations and video animations. The

positive implications were addressed on succesful

descriptions of the challenging and culturally

sensitive topics of peacebuilding. The methods and

algorithms used were seen easy to understand.

In the group discussions it was observed that the

content of peacebuilding training material must be

updated and changes must be completed to the

application in line with the global situation. It was

addressed that research could contribute to online

training with providing new knowledge and updated

content. NMLA enabled learning taking place by

adapting knowledge via different methods such as

reading, watching, and hearing. There was also a

possibility to use learners’ earlier experiences and

knowledge and combine them with NMLA content.

Opposite than teacher-centered pedagogy, the

learner-centered pedagogy acknowledges students’

needs, ability and learning styles (Weimer, 2013).

The digital rich media contents were identified

motivating compared to traditional learning

contents. The survey findings described how video

scenarios described well the conflict setting (24

mentions) and enabled the possibility of problem-

based learning. The gamification features (stages,

certification) raised the learners’ motivation when

the objective was clearly mentioned. The shortfalls

of NMLA were discussed in both end user groups.

Massive reading materials (such as research articles

or research reports) were analysed challenging to

adapt by the learner during learning process.

Moreover, the necessity of the possibility for

feedback and reflection was underlined in order to

ensure learning. The researcher observed that

the learning tool must be easy to use in order to

ensure positive learning experience through

technology. Most of the users (17 out of 19) felt the

NMLA easy to use. The clear navigation and

structure help to ensure humans interacting with

learning tools. The end users using only mobile

devices paid attention to scaling of the learning

content such as videos. As an additional key finding,

the user group that did not receive the information

on how to use the learning application, mainly

produced feedback on clarity of navigation in the

platform. From the HTI point of view, it can be

discussed that human aims to receive the guidance

on how to use the technology solutions before the

actual use. The clear navigation helps the learner to

focus on learning rather than guidance. The

relevance of guidance to be available for online

learning tool users would be beneficial to further

study. According to the interviews, the practitioners

of peacebuilding and conflict prevention mainly

look for technical instatiantions for training that are

ease to use without any further guidance needed.

During the NMLA design, the The Human-

Centered Model for Online Learning Tool Design

was created. The case study of IECEU NMLA

design and development provided an overall model

for successful online learning tool development,

which follows the principles of DSR, human-

centered design and successful human-technology

interaction. The online based and new media

supported educational tools aim to reach well the

end user needs and ensure several learning methods.

It was identified that if different communities of

practice are engaged through test, review and

feedback during the design, it will provide

motivation and possibility to better integrate online

tools for training. Figure 2 presents the Human-

Centered Model for Online Learning Tool Design.

Figure 2: The Human-Centered Model for Online

Learning Tool Design (Hyttinen, 2017).

5 CONCLUSIONS

The education technology accesses the people to the

larger community and provides learning

opportunities via online solutions for people in a

cost effective way. As a conclusion of this study, it

can be suggested that the development of online

learning and education tools for peacebuilding

should follow the principles and theories of adult

learning and human-technology interaction. This

study argues that human-centered approach supports

with better interaction between human and

technology. This study shows the benefits of

considering Human-Technology Interaction (HTI) in

the development of learning solutions for

international security, conflict prevention and

peacebuilding. The importance of the interaction

was observed high especially when the motivation to

use technology in learning is in low level. The

commitment of end users may support the

engagement of human to continue the use of

services. Saariluoma et al (2016, p.148) discuss the

motivation beign rather relevant in explaining the

use of technologies by human. This study showed,

the functional and completed user interface was seen

important to raise the motivation. A relation of the

motivation to use of learning tools could be further

assessed among different communities.

Before development of the online solutions it is

suggested to conduct in dept analysis on the current

needs and requirements, context as well as end

users. Therefore, I argue that implementation of

educational online technology should focus more on

user centric approach and Data Science Research.

The created model for development process of

learning tools support to meet with current training

needs and to identify which parts of education and

training can be organised in an online learning basis

the most effectively. Moreover, the model supports

when seeking the services for online learning. The

emphasis must also focus on transfering the learning

content to rich online format (videos, animations,

gamified solutions). Also, blended training possibly

consisting feedback and interaction with a trainer,

teacher or mentor deepen the understanding of the

learner and/or users of the content. It would also

provide a fast way to collect feedback to improve the

content or functionalities of the platform further.

Instant hot-wash-up with a trainer/mentor could be

considered beneficial in order to deepen users’

understanding and learning as well as to identify

possible knowledge gaps to be covered better in

various training activities. Based on this paper

analyses and desk study, there is still need for deeper

analysis regarding online education possibilities in

conflict prevention and crisis management training

and education, such as:

1) How can the online learning solutions more

effectively be implemented in current

training and education practices?

2) How will the artificial intelligence and

machine learning affect to the training and

information sharing in international

security?

REFERENCES

Adebahr, C. 2011. The Comprehensive Approach to Crisis

Management in Concerted Weimar Effort. Stiftung

Genshagen, 2011.

Bonk, C. J. and Cunningham, D. J. 1998. Searching for

Learner-Centered, Constructivist, and Sociocultural

Components of Collaborative Educational Learning

Tools.

Buckland, P. 2004. Reshaping the Future: Education and

Post-Conflict Reconstruction. World Bank

Publications. ISBN 9780821359594

Burbules, N. 2000. Does the Internet constitute a global

educational community? In Burbules, N and Torres,

CA (eds) Globalization and Education: Critical

Perspectives, New York, Routledge, pp. 323–355.

Bruner, Jerome. 1990. Acts of Meaning, Harvard

University Press, 1990

Carlson, S. and JBS International. 2013. Using technology

to deliver educational services to children and youth

in environments affected by crisis and/or conflicts.

United States Agency for International Development

Charalambos, V., Michalinos, Z. Chamberlain, R. 2004.

The Design of Online Learning Communities: Critical

Issues. Educational Media International. ISSN 1469-

Educational Media.

Clark, R. and Mayer, R. 2016. E-Learning and the Science

of Instruction: Proven Guidelines for Consumers and

Designers of Multimedia Learning. John Wiley and

Sons, 2016. ISBN 1119158664.

Conole, G. and Alevizou, P. 2010. A literature review of

the use of Web 2.0 tools in Higher Education. A report

commissioned by the Higher Education Academy.

August 2010, The Open University, Walton Hall,

Milton Keynes, UK.

Conrad, D. 2002. Deep in the hearts of learners: Insights

into the nature of online community. Journal of

Distance Education, 17(1), 1−19.

Department for International Development (DfID). 2005.

Why we need to work more effectively on Fragile

States (London, DfID).

European Union External Action, 2016. Strategic

Assessment of 2016 EU Global Strategy on Foreign

and Security Policy. Accessed in February 2016.

European Parliament, 2012. CSDP Missions and

Operations: Lessons Learned Processes. European

Union, 2012. Directorate-General for External

Policies, Policy Department. ISBN: 978-92-823-3756-

1. Accessed September 21, 2017.

Friedrich, Pirjo. 2013. Web-based co-design. Social media

tools to enhance user-centered design and innovation

process, University of Tampere, 2013.

Gaebel, M. and al. 2014. E-learning in European higher

education institutions. Results of a mapping survey

conducted in October-December 2013.

Gulati, S. 2008. Technology-Enhanced Learning in

Developing Nations: A review.

Hevner, A., March, S. T. Park, J. 2004. Design Science in

Information Systems Research. MIS Quarterly (28:1)

2004, pp. 75-105.

IECEU –project. 2016. D1.2 Analysis of the current

preventive activities. Accessed in April 2016.

http://www.ieceu-project.com

IECEU -project, 2016. D5.2 Design and Development

Processes of the NMLA. Accessed March 2017.

www.ieceu-project.com

ISO, 10015:1999. Quality Management- Guidelines for

Training.

Jounghyun, G. K. 2015 Human–Computer Interaction

Fundamentals and Practice. Taylor and Francis

Group. ISBN 978-1-4822-3389-6

Katz, J. 2013. Mobile gazing two-ways: Visual layering as

an emerging mobile communication service. Mobile

Media and Communication, 1, 129-133.

Kirk, J. 2007. Education and fragile states. Globalisation,

Societies and Education. Vol. 5, No. 2, July 2007,

pp. 197

Leinonen, T. 2010. Designing Learning Tools.

Methodological Insights, Aalto University, 2010.

Leikas, 2009. Life-Based Design – A holistic approach to

designing human-technology interaction. VTT

Publications 726. ISBN 978-951-38-7375-2.

Marsick, V., and Watkins, K. 2001. Informal and

Incidental Learning. New Directions for Adult and

Bass, A Publishing Unit of John Wiley and Sons, Inc

Means, et al. 2009. Evaluation of Evidence-Based

Practices in Online Learning: A Meta-Analysis and

Review of Online Learning Studies.

Mishra, P., and Koehler, M. 2006. Technological

pedagogical content knowledge: A framework for

integrating technology in teacher knowledge.

Moore, J. Dickson-Deane, C. Galyen, K. 2011. e-Learning,

online learning, and distance learning environments:

Are they the same? Internet and Higher Education 14

(2011) 129–135. 2010 Elsevier Inc.

Moore, M. G. 1990. Background and overview of

contemporary American distance education.

Contemporary issues in American distance education

(pp. xii−xxvi). New York: Pergamon Press.

Morton, K. and O’Hagan, J. 2009. Humanitarianism then

and now: Exploring the boundaries of civil–military

relations in a post-9/11 world in Humanitarianism and

civil– military relations in a post-9/11 world, ed.

Jacinta O’Hagan and Katherine Morton, (Canberra:

The Australian National University, 2009), 1-7.

Rutkowski, 1982. New Soft Computing Techniques for

System Modelling, Pattern Classification and Imafe

Processing. Springer.

Roto, V. Kaasinen, E. Heimonen, T. Karvonen, H.

Jokinen, J. Mannonen, P. Nousu, H. Hakulinen, J. Lu,

Y. Saariluoma, P. Kymäläinen, T. Keskinen, T.

Turunen, M. Koskinen, H. 2017. Utilizing Experience

Goals in Design of Industrial Systems. Proceedings of

the 2017 CHI Conference on Human Factors in

Computing Systems. ACM, 2017.

Saariluoma, P., Cañas, J., Leikas, J. 2016. Designing for

Life - A Human Perspective on Technology

Development. Macmillan Publishers Ltd. London.

ISBN 978-1-137-53047-9 (eBook)

Sein, M., Henfridsson, O. Purao, S., Rossi, M. and

Lindgren, R. 2011. Action Design Research. MIS

Quarterly, (35: 1), pp. 37-56.

Sevillano-García, M. L., and Vázquez-Cano, E. 2015. The

Impact of Digital Mobile Devices in Higher

Education. Educational Technology and Society,

18 (1), p. 114.

Siemens, G. 2014. Connectivism: A learning theory for

Digital Age.

Smith, J., 1998. The book, The publishing company.

London, 2

edition.

The Treaty of Lisbon, entered into force on 1 December

2009. Accessed September 25, 2017.

UNESCO, 2014. Teaching and Learning – Achieving

Quality for All. ISBN 978-92-3-104255-3,

Weimer, M. 2013. Learner-centered teaching: Five key

changes to practice. John Wiley and Sons.

Zhang, D., Zhao, J. L., Zhou, L., and Nunamaker Jr., J. F.

2004. Can e-learning replace classroom learning?

Communications of the ACM , 45 (5), 75-79.

Zheng, B., Niiyab, M. and Warschauerb, M. 2014. Wikis

and collaborative learning in higher education.

Technology, Pedagogy and Education, 2015. Vol. 24,

No. 3, 357–374.