Estimating the Effort in the Development of Distance Learning Paths

Milena Casagranda

Luigi Colazzo

and Andrea Molinari

Department of Economics and Management, University of Trento, Via Inama, 5, 38122 Trento, Italy

Department of Industrial Engineering, University of Trento, Via Sommarive, 2, 38122 Trento, Italy

Keywords: e-Learning Cost Modeling.

Abstract: The quantification of the effort required to produce e-learning material has been always subject to the

proposal of some “magic numbers” not always supported by a method or analytical data. In this paper we

propose a method for calculating the effort required to design and develop e-learning paths. Our model is

based on a systematic gathering of data regarding the effort of the different actors involved (teachers, tutors,

instructional designers etc.) from a series of e-learning projects carried out at the Laboratory of Maieutics

over the last five years. In the first two years we collected ex-post data on design and development times,

being careful to include a variety of different teaching methodologies. In the following years we identified

critical variables which allowed us to abstract and generalize a possible costing model. The model proposed

could be used as a reference point for professionals working on the development of content in their

estimation of costs linked to the design and development of learning materials, providing a calculation basis

which takes a number of methodological approaches and educational objectives into account.

1 INTRODUCTION

Over the years the fortunes of Distance Learning

(DL) have see-sawed, for reasons both cultural and

technological: among them, a reluctance to replace

face-to-face interaction with instructors; inadequate

communication lines for the transmission of

complex learning objects (LOs); insufficient

bandwidth for the reliable streaming of multimedia

material and also a lack of standards for tracking and

certification of on-line courses. Most of these

problems have now been overcome: it therefore

seems that the hesitant progress of e-learning –

always on the point of spreading extensively and

then for some reason never really succeeding -

should not have to continue any longer, given the

disappearance of most of the technological barriers.

In fact, we believe, there is another -

considerably more insidious, not being linked to the

inevitable progress of technology - obstacle which

threatens to further slow the spread of DL. This

obstacle is the economic advantage of instructors in

the production of LOs in relation to the traditional

method of being paid an hourly or daily rate, linked

to their verifiable, physical presence in a classroom.

This point seems rather crass and mercenary, but we

have observed early signs of such resistance,

especially as it is now clear, at the industrial level

above all, that e-learning will become indispensable.

We refer to those cases where e-learning has

unquestionable advantages, for example: a) large

numbers of people to be trained in a short time; b)

people widely dispersed or hard to access physically;

c) training regarding software or ICTs usage.

All the above cases include factors which could

be considered to “justify” fair pay for the creators of

LOs (instructors, instructional designers, tutors,

etc.). A comparison of the production costs of the

LOs with how much it would have cost to run a

face-to-face course normally justify the choice for an

e-learning initiative. On the other side, however, we

find the perspective of the creator of the LOs, who

should be highly paid due to the number of users

involved and due to the strategic value of the

training itself. We are thus witnessing an extension

of the online course offer, which is going to involve

an increasing range of subjects, ever more closely

linked to those areas (like soft skills) traditionally

not “favorable” to e-learning. Nevertheless, this

potential burst of growth in LOs production presents

a number of criticalities:

 a more detailed planning is required in order to

create interactive, reflective, self-rating situations

for the learning of soft skills;

264

Casagranda M., Casagranda L. and Molinari A..

Estimating the Effort in the Development of Distance Learning Paths.

DOI: 10.5220/0004533602640272

In Proceedings of the 4th International Conference on Data Communication Networking, 10th International Conference on e-Business and 4th

International Conference on Optical Communication Systems (ICE-B-2013), pages 264-272

ISBN: 978-989-8565-72-3

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

 LOs are delivered by instructors who are often

unfamiliar with ICTs ;

 LOs represent an area of the market which is still

very profitable for traditional classroom based

education.

Regarding the last item, resistance on the part of

instructors, who have always been the authors of the

destiny of e-learning initiatives, is presumable – and,

as far as we are concerned, already verifiable. By

instructors we mean everybody involved in the

development of courses and related educational

objects. The argument is very simple: considering

for example the business of professional training,

teaching a given subject requiring 20 classroom

hours to 100 people allows an instructor to suppose

N repetitions of the course, each multiplied by 20

hours, multiplied by their hourly pay. The economic

final reward (FR) for the instructor, by which to

calculate the threshold of convenience in creating

the LOs, is function (at least) of the following

parameters:

FR = f( r , h , hr , d , ph , hpc)

where

r = No. of course repetitions;

h = No. of hours of each edition;

hr = hourly rate of the teacher for that course

d = rate for design activities

ph = preparation hours needed for LOs creation

hpc = hourly preparation costs

Other elements should be considered, like the

location of the course, travel expenses, credibility of

the organization etc. Sometimes, moreover, a flat

rate is paid for design, or it is not always

recompensed, since instructors receive a good hourly

rate for classroom hours (particularly as they

become more senior).

In the simplest situations, the function can be

easily calculated as follows:

FR= r * h * hr + d – ph * hpc

There’s no doubt that unless a pay scheme for e-

learning courses at least as attractive to instructors as

that for traditional courses is provided, there will be

a further brake on the spread of e-learning. A

number of studies have focused on the determining

of parameters, and many authors have already

pointed out the complexity resulting from

methodologically and educationally based choices in

the construction of e-learning courses. (Bacsich eand

Asch, 1999); (Bacsich and Asch, 2001); (Lewin,

1995).

To a great degree, over-simplifying the idea

risks, we believe, the reoccurrence of the initial

problem, the “inexpedience” for experts of

transferring their knowledge through DL. We have

already drawn attention (Casagranda et al., 2010) to

criteria for calculating the production costs of LOs,

but extensive subsequent trials have revealed the

need to refine and integrate the model, especially in

the LOs design stage. In this paper we will propose

these extensions and refinements to our e-learning

costs model. We have added more importance to the

design stage, in order to allow the a-priori estimation

of the work done by the different experts involved in

project development.

2 THE DESIGN OF

EDUCATIONAL PROJECTS

The detailed preparation of an e-learning course or

environment needs to be done in advance. The

instructional designer should identify the subject

matter, define it operatively, decide how to evaluate

students’ knowledge and skills and introduce

occasions for feedback in order to support learning.

Therefore, we begin by identifying the different

phases of planning stages and then suggest how times

and costs might be calculated.

All e-learning projects include at least four basic

stages which lead to the delivery of a course. We

believe that in the evaluation of the effort involved,

we should quantify, optimize and recognize:

 the design of the educational project;

 the design of LOs through an analysis of subject

matter, which we introduce as modeling in this

article;

 the development of educational material, the

utilization of which has already been described

by other researchers (Bartley e Golek, 2004);

 the management of interactions with students in

order to further learning, mostly following web

2.0 approaches and tools (forums, blogs, wikis,

social network interactions etc.), as already

mentioned in previous articles (Casagranda et al.,

2010)

When discussing the design phase, we include a

number of dimensions, already mentioned elsewhere

(Raineri, 2005):

 demand assessment: information about clients’

expectations is collected, clarified and selected,

through hypotheses which designers develop

considering technological issues about the

creation of LOs ;

 the proposal of hypotheses: the variables and

EstimatingtheEffortintheDevelopmentofDistanceLearningPaths

265

points of view involved are multiple and it is

hoped that decisions / solutions can lead to the

presentation to clients of a number of

possibilities;

 fine tuning of the project: when a possibility has

been chosen the – almost inevitable – next step is

the development of the project, a recursive

research process in which the direction of the

project is defined.

The starting point for the design of both the course

structure and content is an analysis of its educational

objectives. We have based our work on the model

which guided us in our operative planning choices

for the project (Battaglia et al, 2008). Through a

consideration of three critical variables we can create

a 3-dimensional matrix representing the educational

models:

 the principal teaching objectives;

 the importance of vertical interaction according

to the principle of instructors’ authority;

 the importance of horizontal interaction

according to the principle of collaboration and

dialogue.

The combination of educational objectives has been

graded in three segments: knowledge, technological

skills, the development of cognitive skills.

The second dimension focuses on the importance

of instructors as a source of knowledge and as an

expert in the field. The two ends of the continuum are

the “Central figure” or “reference point” at one

extreme, on whom the learning process depends, and

the “Neutral moderator” of students’ study, at the

other.

Figure 1: The vertical interaction dimension of an

educational model.

In our terminology, “Verticality” represents the

degree of interaction between instructor and student,

while “Horizontality” refers to the degree of

interaction between students. Note that the two

dimensions are independent, allowing four different

combinations: the weak vertical - weak horizontal

combination represents independent learning, while

the strong-strong combination represents the

development of elaborate educational projects, in

which the instructor assumes the role of project

manager and strong group leader. The combination

of the three dimensions generates an cubic

“educational space” in which twelve educational

models can be identified. Some of these models are

relatively “pure” and can therefore be used as general

points of reference.

Figure 2: The horizontal interaction dimension of an

educational model.

We can place the tutor in this educational space,

alongside the instructor. This allows us to see how

closely correlated and dependent on each other these

roles are: when an instructor is the central role for a

course, the tutor – to a large extent an observer –

supports him; conversely, when the instructor’s role

is that of moderator, the teaching role of the tutor is

potentially greater, particularly for any Distant

Learning versions of a course. In our studies, four

educational models have been identified:

1. models aimed at the acquisition of knowledge:

characterized by strong vertical interaction

between instructor and student, weak horizontal

interaction between students;

2. models focusing on method: characterized by

strong vertical interaction, weak horizontal

interaction and a focus on technological skills;

3. models aimed at increasing ability: characterized

by strong vertical interaction, strong horizontal

interaction and a focus on technological skills;

4. models aimed at increasing expertise:

characterized by weak vertical interaction, strong

horizontal interaction and with a focus on the

development of cognitive skills.

Note that the variable of technology is not one of the

dimensions of the model: the characteristics of the

technological tools used to deliver on-line courses

cannot be considered primary criteria for the

classification of educational models.

We could have different models for the same

educational proposal, depending on the methodology

that is considered more effective for the learning

ICE-B2013-InternationalConferenceone-Business

266

process. The detection of the predominant model is

useful to determine the educational objectives, while

it is relevant for our cost model in order to determine

the type and the extension of the support activities in

the educational processes, as presented in tables 1

and 2 at the end of this paper.

The central role at this stage and across the other

proposals is the e-learning project managers, who

support and coordinate those involved in the

development of the educational project. The overall

organization and implementation of the e-learning

system is their responsibility.

Figure 3: The “educational space” through the

identification of educational formats.

They supervise the planning, development and

administration of the project and are responsible for

its educational content. The project manager

undertakes to:

 schedule and coordinate the e-learning program

 identify users’ profiles of the e-learning system

 analyze the educational requirements / demand

of users

 define the objectives of the e-learning system

 define the type of service to set up

 define the criteria and indicators for the

monitoring and evaluation of the service

We examine the roles of the collaborators later on, in

particular those of the content manager and

instructional designer.

3 LEARNING OBJECTS DESIGN

THROUGH CONTENT

ANALYSIS

When LOs are being planned, after the appropriate

educational model has been identified, we can start to

apply our costing model. The most significant

variables, which we introduce for the first time in this

model, are (Raineri, 2002):

 content “definability”: can the content be

processed in a standard way, or does it require

discussion and comparison and need to be

created for a specific group of participants?

 content “interactivity”: can the content be

conveyed through text, images and graphics, or

does it require interactivity? Simulations, for

example?

The definition of the content (how much structured

subject matter there is, on a scale: low – medium –

high) lies on the vertical axis. The lower the value,

the less time the instructor will need to plan specific

content and design time can be devoted to the

methodological structure and to the search for stimuli

that can be offered along a study path (e.g.

facilitation of the study community, where the

educational task and support while the work is

ongoing will be more recognized) (Cohen and

Nachmias, 2008).

The indicators reported in this paper are

multipliers that result from an ex-post analysis

conducted over five years of design and creation of

learning objects. We collected the detailed data from

the roles involved in the process regarding design

times and creation times. Afterwards, we identified

some recurring variables in the process, and their

average influence on the time used to produce the

learning objects.

Figure 4: Model for the calculation of multipliers in the

planning of material.

We have placed the level of interactivity on the

horizontal axis. According to our formulation, here

follows some examples of LOs with low levels of

interactivity:

 narrative LOs, which introduce suggested

courses and/or single modules;

EstimatingtheEffortintheDevelopmentofDistanceLearningPaths

267

 expository LOs, which refer to text based

information and concepts, images.

The following are examples of medium levels of

interactivity:

 Demonstrations: animations which illustrate a

series of operations

 Interactive LOs: participants are required to

interact with the material – providing answers,

commands, links, etc.

 Tests: drag and drop, classification tests that start

with given possibilities, problem solving tests,

etc.

 Guided exercises: step by step instructions on

what to do

 Case studies

The following involve high levels of interaction:

 Simulations: presentations of simulated

situations in which the student must achieve an

objective (e.g. using particular software)

 Role based LOs: contexts in which participants

are asked to take decisions, etc.

Each cell in this model is assigned a different value

which is multiplied by the expected length of the

material being developed. We have excluded

“serious games” from the calculation / estimate that

can be made at this stage of our experiments / trials.

The serious games category is a broad one,

nevertheless it always entails, at all stages of

development, a quantity of work which definitely

cannot be reduced to the elements represented in our

model. Currently, there are experiments / trials on-

going in this field, but they are still at too early a

stage to provide accurate data.

At least two figures are involved at this stage of

design, their contributions can be identified using the

same parameters (from the point of view of co-

planning), using the matrix above:

- the instructor, who:

 participates in the definition of course structure

 identifies the possible testing and evaluation

methods to be included in the program

 supports and collaborates with the instructional

designer in the creation of LOs

- the instructional designer, who:

 gives advice on educational methods and

strategies for the delivery of content and

resources for e-learning

 decides which software to use for the

generation of content

 manages the multimedia resources

 identifies the strategies and tools for evaluation

and practice most appropriate to the purposes of

the course.

4 THE DEVELOPMENT OF

EDUCATIONAL MATERIAL

Regarding another component in our model for

determining the cost of production, i.e., the

development of educational material, in other studies

(Casagranda et al., 2010) we suggested a method for

estimating the amount of time spent by instructors on

the creation of LOs. We will shortly present this part,

considering the following questions: a) will all the

material be new, or is there any material available for

reuse? b) to what extent might the material be

reusable in the future?

The possibility of reusing existent material for the

development of LOs has a great impact in the

perception that end users have about the originality

of the creator’s work, and sometimes is even subject

to negotiation between the educational institution and

the instructor. In our model, this aspect lies on the

vertical axis, on a scale: absent – low – medium –

high. The higher the value, the less time the

instructor needs for the preparation of the LO.

The replicability of material – how possible it is

to reuse the same LOs for other courses or users

(Huddleston and Pike, 2005) – lies on the horizontal

axis. The scale we use is the same as the last one: the

higher the replicability the bigger the multiplier, as

the same material can be used for many versions and

subjects and costs can thus be recouped. In contrast,

if there is little or no possibility of replicating the

material and an LO is useful only in one, or a limited

number of, context/s, instructors will be paid

proportionately less since they will soon have either

to generate new material or update old.

We hypothesize that the values of the variables of

the multiplier on this axis will be lower than those on

the vertical. All the cells in the model are assigned a

different value which serves as a multiplier for the

expected useful life of the material being developed.

Moreover, the concept of “reuse” can be based on an

estimate of the number of versions expected and on

the number of possible users, or even on estimates of

re-combinability in other contexts.

The following factors are to be considered when

determining the “reuse of material” variable:

 Absent: it is the first time that instructors have

developed a course like this, so they have neither

classroom nor online material available with

which to begin the preparation and must start

ICE-B2013-InternationalConferenceone-Business

268

from zero, or near zero;

 Medium: instructors can adapt material which

they have previously used in the classroom for

use online, or have a limited amount of relatively

unstructured online material available;

 High: instructors have some structured material

available, suitable for their chosen type of

distance education, and only small changes or

updates are required.

The variable regarding the replicability has been

constructed referring to the average participation to

courses held for University and public

administrations in our territory. It is complicated to

“universalize” this parameter of replicability, that in

our case sets a “high replicability” when courses

involve up to 100 participants.

Figure 5: Model for calculation of multipliers in the

creation of material.

This parameter could hence be a limit in different

contexts respect to those used in our analysis, and

should be recalculated and adapted. At the same

time, having collected data from several courses

editions, in environments very close to what the

market nowadays provides, we are confident to have

a good basis for future analysis.

At this stage we are also faced with the following

questions:

 What are the best technologies for the running of

the course? For content management? (We are

here referring to LMS and LCMS.)

 What technical support is needed for the

production of the material?

Sometimes the instructor him/herself will be able to

produce the educational material, sometimes a

specific person will carry out this function. The

person producing material will have the time

recognized using the model in Fig. 5. It is important

to add, however, that it would be helpful to add an

extra multiplier to our model, both to quantify the

work involved in reviewing the content / work, in

checking that the material meets established LO

standards, and for the professional editing of the

material itself, as set out below.

At this stage we add another value, when two

roles dominate the work: the content manager and

the LO editor / LO production expert, who guarantee

the quality of the LO at all stages of its production,

and especially:

 when the stages of production and their durations

are being decided;

 for the writing of LO storyboards;

 for managing professional standards in the

multimedia content (videos, narrators, graphics,

etc.)

 for editing of the LO

From our evidences, we derived these multiplying

factors, that must be confirmed in further analysis:

Content manager

Responsible for the review of video

content: video time multiplied by 3

LO editor

Responsible for editing: video time

multiplied by 33

Figure 6: Model for calculating the multiplier for creating

material.

The incidence of this element on the production

time of LOs is clearly relevant, and in our experience

this is particularly true for those institutions that do

not have a LO editor at their disposal, thus forcing

them to turn to external expertise.

Accessibility of learning objects is a serious

issue, and the attention devoted to people with

disabilities is never too much. Thus, we observed in

our tests to which extend the impact of enriching /

modifying material to be usable by this category of

users was. When accessible materials are being

created, we have identified a further increase in time

to be recognized for this activity:

Accessible materials generated by the LO editor

- Addition of Text: Creation of material + Video time

- Addition of Subtitles: Creation of material + Video time

multiplied by 1.5

Figure 7: Model for calculating multipliers in the case of

accessible material.

5 SUPPORTING THE LEARNING

PROCESS

Finally, as already discussed in other papers,

(Casagranda et al., 2010), we consider the choice of

teaching model to be very important for the

estimation of the maximum number of hours that

EstimatingtheEffortintheDevelopmentofDistanceLearningPaths

269

tutors and instructors will spend on the delivery of a

course. The calculation of these hours is based on the

application of a percentage on the total number of

course hours for face-to-face and DL, shown below:

Table 1: Adjusting factor for distance learning courses

based on teaching model for teaching.

This table presents the Percentage of the

maximum number of instructor’s hours recognized

based on combinations of classroom and/or DL hours

(supporting students’ learning and communication

during the course). The following figure, instead,

represents the percentage of the maximum number of

tutor’s hours recognized based on combinations of

classroom and/or DL hours (supporting students’

learning and communication during the course)

Table 2: Adjusting factor for distance learning courses

based on teaching model for tutoring.

The percentage is doubled if it is expected that an

instructor’s presence in the medium term will be

required, either to allow students more time to

complete certain activities (e.g. in blended courses:

within 3 months of the last classroom based lesson),

or to guarantee that the course web pages are updated

(e.g. the instructor agrees to update FAQs,

regulations, etc.).

6 APPLICATION OF THE

MODEL

In this article we describe the application of the

model to a self-study course run in 2011 and 2012.

Our results were extremely encouraging: a deviation

of about 10% between the “ex-ante” calculation,

based on the a priori application of the suggested

parameters, and the “ex-post” statement of accounts

(allowing an a posteriori evaluation of the

hypothesis).

Choice of Educational Model

- Information component: model directed towards knowledge

- Application component: model directed towards method

Figure 8: Educational model.

Calculation of planning time

- Information component: narrative and expository LOs

o Instructor/expert: 6 hours (effective duration) * 12

(multiplier: high definability of the content/ low levels of

interaction) = 72 hours

o instructional designer: 6 hours (effective duration) * 12

(multiplier: high definability of the content / low levels of

interaction) = 72 hours

- Application component: simulations

o instructor/expert: 2 hours (effective duration) * 20

(multiplier: high definability of the content/ high

interactivity) = 40 hours

o instructional designer: 2 hours (effective duration) * 20

(multiplier: high definability of the content/ high

interactivity) = 40 hours

o total planning hours: 224

Figure 9: Calculation of planning time.

See below a short summary of the course

characteristics:

 Course title: “Digital signatures and Certified

electronic mail”;

 Participants: almost 22,000 private businesses;

 Course objectives: divided into two areas;

 Information component: a presentation of the

main characteristics of digital signatures and

certified electronic mail, the related legal aspects,

the main uses of these tools, the necessity of

ICE-B2013-InternationalConferenceone-Business

270

adopting them, limits to their use, opportunities

for their use;

 Application component: showing how digital

signatures and certified electronic mail work and

making available practical demonstrations of

their use.

 Effective duration of the educational material–

information component: 6 hours, application

component: 2 hours.

Calculation of production time

- Information component

o instructor: 6 hours (effective duration) * 5 multiplier

(material replicability high and reuse of existing material

absent) = 30 hours

o content manager: 6 hours (effective duration) * 3

(multiplier) = 18 hours

o LO editing: 6 hours (effective duration) * 33 (multiplier) =

198 hours;

o accessibility 6 hours (effective duration) * 1.5 (multiplier)

= 9 hours

- Application component

o instructor: 2 hours (effective duration) * 5 multiplier

(material replicability high and reuse of existing material

absent) = 10 hours

o content manager: 2 hours (effective duration) * 3

(multiplier) = 6 hours

o LO editing: 2 hours (effective duration) * 33 (multiplier) =

66 hours;

o accessibility 2 hours (effective duration) * 1.5 (multiplier)

= 3 hours

o Total LO production hours: 100

Figure 10: Calculation of production time.

In summary, the original course has been planned

as a 6 hours + 2 hours course. With the application of

our model to this specific instance, the global effort

has been calculated as follows:

- Planning Time: 72h+72h+40h+40h = 224h

- production Time: 30h + 18h + 198h + 9h + 10h +

6h + 66h +3h = 340h

- Support time: n/a

Total Effort: 564h respect to an effective duration of

the educational material of 8 hours.

Calculation of time spent in supporting the educational

process

- Information component:

o instructor: max 5% of total hours * number of participants

- Application component:

o instructor: max 6% of total hours * number of participants

In the case study, given the number of users, no educational

support was factored in.

Figure 11: Calculation time spent supporting the

educational process.

7 CONCLUSIONS

This article shortly describes a model for the

estimation of the time spent by the various people

involved in the planning and production of DL

material. We attempt to address the problem of

calculating total numbers of hours worked by all the

actors, providing a general framework for this

complex calculation (in the sense of quantity of

variables and situations to be considered).

This is largely a result of the lack of an adequate

model for standardizing the measurement of the

effort required to create online material, leading to

instructors receiving completely inadequate

recompense for their work. We believe that the lack

of this calculation could be one of the possible

factors which could slow the spread of DL. Our

model calculates, using multipliers and reference

tables which have been trialed in the field in some

EstimatingtheEffortintheDevelopmentofDistanceLearningPaths

271

revealing projects, the number of hours worked by

the creators of DL courses to be recognized against

the number of hours scheduled for the online course.

Most of the trials / experiments with this model were

carried out under the auspices of the DL program of

the Autonomous Province of Trento and the

University of Trento, where we applied the model on

a set of 37 editions of different courses run in

2011/2012.

Our results were extremely encouraging: a

deviation of about 10% between the “ex-ante”

calculation, based on the a priori application of the

suggested parameters, and the “ex-post” statement of

accounts (allowing an a posteriori evaluation of the

hypothesis), based on the systematic gathering of

data from the roles involved (teachers, tutors,

instructional designers, etc.). Further experiments are

needed to confirm or adjust the multipliers stated in

the model, especially regarding new media and new

educational models and approaches. The evaluation

of the multiplier for multimedia learning objects is a

crucial component: we are already working on a

method that progressively decreases progressively

the effort respect to the length of multimedia learning

objects to be produced.

REFERENCES

Bacsich P. e Ash C. (1999), The Costs Of Networked

Learning, Sheffield, Sheffield Hallam University.

Bacsich e Asch, 2001, The Costs Of Networked Learning

Phase Two, Sheffield, Sheffield Hallam University.

Bartley S. J. E Golek J. H.(2004), Evaluating the Cost

Effectiveness of Online and Face-to-Face Instruction.

Educational Technologies & Society, 7 (4), 167-175

Battaglia G., Serpelloni G., Simeoni E. (2008), Apprendere

e lavorare nell’era digitale – Online collaborative e-

learning per le organizzazioni sanitarie, Verona.

Casagranda M., Colazzo L., Molinari A., Tomasini S.

(2010), E-learning as an opportunity for the public

administration: results and evolution of a learning

model. In International Journal of Teaching and Case

Studies, Vol. 2, N. 3/4, 2010, ISSN (Online): 1749-

916X - ISSN (Print): 1749-9151, InterScience

Cohen A. e Nachmias R. (2008), A case Study of

Implementing a Cost Effectiveness Analyzer for Web-

Supported Academic Instruction: An Example from

Life Science. Paper for the EDEN 2008 Annual

Conference –New Learning Cultures, How do we

learn? Where do we Learn? Portugal

Huddlestone J. Pike J. (2005), Learning Object Reuse – A

Four Tier Model, IEEE and MOD HFI DTC

Symposium on People and systems - who are we

designed for.

Levin H. M. (1995), Cost-effectiveness analysis. In M.

Carnoy (Ed.), International encyclopedia of economics

of education (2nd ed., pp. 381-386). Oxford:

Pergamon.

Ranieri M. (2005), E-learning: modelli e strategie

didattiche, Trento, Erikson

Rumble G. (2001), The costs and costing of networked

learning, “JALN”, vol. 5, n. 2.

ICE-B2013-InternationalConferenceone-Business

272