QUALITY ASSURANCE IN DISTANCE LEARNING

A Study in Higher Education

Gianna Oliveira Bogossian Roque

Central Coordination for Distance learning, Pontificia Universidade Católica do Rio de Janeiro

Rio de Janeiro, 22453900, Brazil

Gilda Helena Bernardino de Campo

Dept. Education and Central Coordination for Distance learning, Pontificia Universidade Católica do Rio de Janeiro

Rio de Janeiro, 22453900, Brazil

Marcus Vinícius de Araújo Fonseca

Programa de Engenharia de Produção, COPPE-UFRJ, Pólo de Xistoquímica, IQ-UFRJ, Rio de Janeiro, Brazil

Keywords: Quality, Higher education, Distance learning, Formation consistency, Knowledge applicability.

Abstract: The preoccupation with forming graduate students of distance learning courses in Brazil, and those which

aim at forming teachers, have raised concerns, especially with regards to formation consistency. According

to our viewpoint, formation consistency is linked to two factors: reliability, which relates to the ability of the

Higher Education Institution to provide a service in a correct, safe and careful manner; and applicability,

which we understand as the possibility of applying the knowledge acquired during the professional and

social life of the graduate student. This study presents the preliminary results of a research aimed at

verifying the students’ perception of a Licentiate in History about formation consistency in higher education

offered at the distance learning modality.

1 INTRODUCTION

To assess the quality of an educational program, we

must ponder over the definition of the term quality.

On a broader sense, it can be defined as the ability of

an object or action to correspond to its purpose.

The different mechanisms for assessing the

courses adopted by the Education Ministry (MEC) in

Brazil aim at revealing the programs’ quality. For

presential or distance undergraduate courses, the

quality is verified using a few instruments, such as

the National System for Assessing Higher Education

(SINAES) and the National Student Development

Exam (ENADE). Although the instruments exist,

issues related to the quality of Higher Education are

still broadly discussed and incite a few concerns. In

Distance Learning (EAD), it is even more

preoccupying, given the very few results available–

considering that distance graduation in Brazil is still

very recent–and the accelerated growth of this

teaching modality. In addition to the assessment by

SINAES and by ENADE, distance learning courses

in Brazil must comply with the Quality Referentials

for Distance Learning in Higher Education defined

by the Distance Learning Education Secretary of the

Ministry of Education (SEED/MEC).

The purpose of this article is to verify the issues

involving quality assessment of undergraduate

distance learning courses and to present an on-going

research aimed at understanding the students’

perception of the quality of the course in which they

are enrolled.

2 QUALITY IN DISTANCE

EDUCATION

The preoccupation with the quality of graduate

students, whether in presential or distance courses, is

not exclusive to Brazil. Around the world, new

293

Oliveira Bogossian Roque G., Helena Bernardino de Campo G. and Vinícius de Araújo Fonseca M..

QUALITY ASSURANCE IN DISTANCE LEARNING - A Study in Higher Education.

DOI: 10.5220/0003306702930298

In Proceedings of the 3rd International Conference on Computer Supported Education (CSEDU-2011), pages 293-298

ISBN: 978-989-8425-50-8

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

society demands have required the implementation

of several measures that seek to improve

undergraduate degrees. For example, the Bologna

Protocol, signed on June 19, 1999 by 29 countries of

the European Union (EADTU, 2007). One of its

objectives is to develop a quality assurance process

that somehow ensures a qualification and

performance standard for undergraduate and

graduate degrees in all signatory countries.

Nevertheless, the issues at the core of the debate

about quality in Higher Education, according to

Santana (2007), are discussed under two

perspectives: the critical viewpoint, which reflects

on social inclusion and human development; and the

instrumental viewpoint. The author affirms that “the

instrumental vision makes education a business

comparable to any other commercial enterprise. But

the critical vision is concerned with understanding

quality as having a social function to be fulfilled”

(Santana, 2007, p. 61). In other words, in the

instrumental perspective, quality in education is

geared towards processes and results, while in the

critical vision it refers to the social relevance and the

pertinence of the formation. This distinction affects

the evaluation that the educational programs are

subjected to.

This research focuses on undergraduate

programs offered in the Distance Learning (EAD)

modality. This is justified because the programs in

this modality have been presented as a quality

alternative to a higher education degree for a large

number of individuals seeking both insertion in the

labor market and improvement of their initial

formation.

In the last 6 years, there has been a 1,980%

increase in the number of enrollments in distance

learning undergraduate degrees in Brazil (INEP,

2008), while for presential programs this percentage

was only 25%. This fast growth in EAD leads us to

inquire about the way in which the quality

assessment procedures in these courses have been

implemented and whether these programs promote

innovations noticeable to the students and that result

in a transformation in their professional life.

In order to guide and regulate an EAD program

in Brazil, a document has been developed, titled

“Quality Referentials for Distance Higher Education

Learning” (MEC, 2007). These referentials outline

the guidelines for certification, offer and follow-up

of distance learning courses, and are based on the

following assumptions: there is no single distance

learning model; the best technology and

methodology depend on the nature of the course and

the target audience.

3 FORMATION CONSISTENCY

The Brazilian Government has invested massively in

the distance learning modality. From 2008 to 2010,

there has been a 78% increase in the number of

undergraduate programs offered by public

institutions in this modality. Current higher

education distance learning programs in Brazil

include, primarily, the Pro-Licentiate I and Pro-

Licentiate II programs, which began in 2005, in

addition to the programs linked to the Brazilian

Open University System (UAB), which began in

2006. One of the objectives of the UAB System is to

offer, through the use of information and

communication technologies (ICTs), higher

education to a large number of teachers of the public

school system.

Gatti and Barreto (2009) raise a few issues with

regards to governmental programs oriented to

teachers formation, more specifically, the licentiate

programs offered through EAD. Among the issues

raised, they mention the accelerated and

disorganized growth of program offers in this

modality, considered by the authors as an attempt to

establish a system that reverts the situation of

inequality in access to higher education in Brazil. It

is evidenced by the multiplication of consortiums,

presential facilities, and offer of courses without a

consistent political and pedagogic project for teacher

formation.

In practice, one verifies that the evaluation of

these programs emphasizes the infrastructure, i.e.,

classroom conditions, libraries, laboratories, and

technological resources; faculty; technical

qualification of human resources, for example,

librarians and technicians; management, which

includes coordination of the facilities, the pedagogic

project, the geographic location, and others.

However if the government proposal is to develop a

policy to improve education and to reduce social

inequalities, besides the infrastructure we must

concern ourselves with the issue of formation

consistency of graduates. This, in our view, relates

to two factors: a) Reliability of the higher education

institution (IES) and the program offered, which is

related to the ability of the IES to provide a service

in a correct, safe and careful manner, and b) the

applicability of acquired knowledge, considered as

the possibility of applying it to the student’s social

and professional life.

Each of these factors, in turn, is verified through

different analysis dimensions, as shown in Figure 1.

So, in addition to the infrastructure, which

certainly is fundamental for the good development

CSEDU 2011 - 3rd International Conference on Computer Supported Education

294

Figure 1: Diagram of Formation Consistency and its relation to the factors of reliability and applicability.

of a course, and is represented by reliability factor,

whose indicators are distributed into different

dimensions, including “tangible elements” and

“pedagogic and political project”, we also consider

as important categories for the assessment of a

course quality for a distance learning undergraduate

program, which are directly related to the

applicability factor,

through its analysis dimensions:

• The efficient use of technology, not only

instrumentally, but in a way that the student

may benefit from the technology for his/her

own personal and professional growth;

• Knowledge creation, not only in the discipline

but, above all, in a practical context;

• Relevance of knowledge, competencies, and

abilities acquired to the reality of the work

environment;

• Impact of using the EAD modality in relation to

effective professional learning.

4 RESEARCH SCOPE

The pilot case study developed for this researchused

as a reference a Licentiate Program in History in the

distance learning modality. The program was created

with the support of MEC’s Pro-Licentiate Program–

through the Fundamental Education Secretary (SEB)

and the Distance Learning Secretary (SEED), and is

part of the Pro-Licentiate II Edict. Its objective is to

improve the fundamental education quality through

consistent and context-based initial formation of the

teacher in his/her work area. The target audience

was initially formed by 1,000 (one thousand) laymen

teachers, that is, with no legal qualification to

practice the profession, and who lectured History in

the fundamental and high school public system,

distributed among several Brazilian states. To be a

student in this program, the applicant must be

working as a teacher of the public school system. It

is, therefore, an in-service formation.

Currently, the program counts a total of 790

students, having began its activities in August 2006

and should end them in December 2010.

The program objective is not only to enable

teachers but, above all, to invest in their academic

formation, giving the students a better understanding

of the world from the knowledge acquired in

experiences of different societies, in diverse time

and space conditions. This formation involves the

production and transmission of historical

knowledge, through differentiated practices, such as:

school-teaching, historical research, advisory and

consulting activities. It also involves the use of

several supports: History books and magazines;

annals of congresses; museographic or non-

museographic exhibits; catalogs, videos, films, CD-

Roms, etc.

The program was developed with an emphasis in

the adoption of EAD procedures, and the teaching-

QUALITY ASSURANCE IN DISTANCE LEARNING - A Study in Higher Education

295

learning process was structured in a bi-directional

way, with media predominance.

To assess the students perception about the

formation consistency offered in the distance

learning modality, which is the focus of this article,

we developed a questionnaire, composed of 3 (three)

open questions and 48 (forty-eight) closed questions.

This questionnaire was sent on-line to all active

students. We received 208 answers, representing

26.3% of the total students regularly enrolled at the

time of research.

The procedure for analysis includes quantitative

and qualitative methods. Quantitative analysis was

performed considering the 208 answers to the 48

closed questions. Quantitative data handling has

been done with the statistical software package

Statistical Package for the Social Sciences (SPSS).

This step is done to verify the validity and

trustworthiness of the data collection instrument,

that is, the applied questionnaire, by means of the

analysis of data variability and its internal

consistency. This analysis, however, is not part of

this article scope. The qualitative analysis includes

contents analysis, i.e., the interpretative analysis of

respondents’ textual records, from the open

questions in the form. The contents analysis is

achieved with the Analyse Lexicale par Contexte

d’un ensemblement de Segment de Texte

(ALCESTE)

software. This software product

analyzes co-occurrences of words in the text, to

organize and summarize the information it deems

more relevant.

In this article, we present the preliminary results

obtained by analyzing the contents of one of the

open questions in the form: In what way has the

acquired knowledge in the program disciplines

enhanced your professional performance?

5 RESULTS

To compose the research corpus, object of this

study, each response by a student-teacher to the open

The ALCESTE software was developed in France in

1979, by Max Reinert, of the National Scientific

Investigation Center (CNRS), at Jean Paul Benzécri’s

laboratory, and was made available to the market by

IMAGE society. In the glossary of Pesquisa Qualitativa

com texto, imagem e som: um manual prático, by Martin

Bauer and George Gaskell, ALCESTE is defined as a

“computer program for qualitative research which

distinguishes different types of speech in natural text,

through an automatic statistical analysis” (BAUER, 2007,

p. 491).

question in the questionnaire is considered a Initial

Context Unit (UCI). The set of all UCIs constitutes

the analysis corpus, which will then be processed by

ALCESTE. For the study in question, the corpus is

formed by the set of answers to the first question.

According to Bauer (2007), the procedures for

contents analysis recreate representations in two

primary dimensions: syntactic and semantic. The

former can identify, by means of “the way

something is said or written”, a likely type of

audience. But the semantic dimension verifies “what

is said in a text”, identifying connotative and

denotative meanings in a text by analyzing co-

occurrences of words that, to this author, “is a

statistical analysis of frequent pairs of words in a

corpus of text” (Bauer, 2007, p. 211). The analysis

will also identify “words with function,” such as

articles, verbs, pronouns, etc. and “words with

contents,” which denote relevant information.

ALCESTE classifies in a semi-automatic manner

the words to the inside of a corpus. The words

classification, named by the software product as

“reduced forms,” occurs after the establishment of

matrices. To achieve this, the text is segmented and

the similarities between the segments and

hierarchies of word classes are defined. This method

is called the hierarchical descending classification

method. To the study analyst, these categories are

not the objective per se, but they establish

assumptions or interpretation trajectories.

In the analysis of the resulting corpus of the set of

answers to the question under study, we established

243 UCEs (Elementary Context Units).

Figure 2: Dendrogram of the classes found.

These are defined according to text size (number

of words parsed) and/or sentence score. Of the 243

defined UCEs, 93% were parsed and 4 (four) fully

hierarchical classes were identified (Figure 2), based

on the reduced forms found in each of them (Table

1).

CSEDU 2011 - 3rd International Conference on Computer Supported Education

296

Naming these classes in function of the reduced

forms found, we have:

• Class 1 – Professional growth

• Class 2 – History knowledge

• Class 3 – Confidence in the contents

• Class 4 – Teaching material

Table 1: Reduced forms of each class.

Reduced Forms

CLASS 1 CLASS 2 CLASS 3 CLASS 4

being historic+ contents+ also

is+ fact+ confidence of the

how before student+ colleagues

profession

+ do work subject+

learning new+ to the tutor+

my teach in experience+

liv+fe many have didactic+

stop+ping one critical helped

improve+ this quality elaboration

growth vision more exchange+

were no past+ had

professor+ be+ with doubts

th+ studied+ transmit am

already one mouvoir. book+

performan

ce learn+ of the enough

import+an

t history oth+er didactic+

great+ are because

in the relation in the

improving history mainly

my+ vision material

good+ helped

our+ by

all+ especially

worked+ but

Notice in the dendrogram (Figure 2) that Classes

1 and 3 contrast with Class 4, and these three classes

contrast with Class 2. The grouping of classes 1 and

3 denotes that both have common meanings which

distinguish them from the other classes. This

meaning (professional growth and confidence in the

contents) points exactly to the issue of improvement

of the quality of student formation and his/her

effective professional performance.

Despite having the lowest number of co-

occurrences of words (only 7.08% of u.c.e.), Class 2

is formed by the reduced forms “fact” and “historic”.

“Historic” is the first dimension present in the

answers, and it signals to the focus of the Licentiate

in History program. It is worth reminding that the

question analyzed inquires about the way in which

the knowledge acquired in the disciplines of the

Licentiate in History distance program have

enhanced the student’s professional performance.

5.1 Knowledge Applicability

The question analyzed by ALCESTE is directly

related to the “applicability” factor, more

specifically, the following analysis dimensions: a)

coherence between theoretical and practical

knowledge; b) social relevance of the knowledge; c)

pertinence of the acquired knowledge (Figure 1).

The dimension of coherence between

theoretical and practical knowledge is concerned

with encouraging the student, throughout the

program, to build a body of knowledge in the

disciplines, but especially one that can be applied in

the context of the work practice.

Because this program is about in-service

formation, the dichotomy between theory and

practice is minimized by the daily activities of the

students themselves. They are teachers who teach

History and at the same are pursuing a Licentiate in

History. Thus, theoretical knowledge acquired

throughout the program is used in practice, resulting

in improvements in their performance. This

dimension touches the speech of the interviewees,

especially those classified by ALCESTE in Class 2,

in which the reduced forms identified provide a new

look at the theoretical knowledge of a historical fact,

and that is reflected in new History teaching and

learning practices.

“The differentiated look at History, considering there

is no single explanation for one historic fact

(an

absolute truth) and that by researching, investigating,

collecting data and analyzing sources we can make

History

. This is not to be kept only with me, because I

can encourage my students to always do a critical

reading of historic facts.” (Interviewee no.143)

The dimension of the social relevance of

knowledge is related to the importance to society of

the knowledge produced by the students and the

relevance of the effects it produces on the social and

economic development of the populations for which

it is generated.

For the analyzed program’s target audience,

which is comprised of History teachers from the

public school system, the importance of the effects

of the acquired knowledge is reflected in their

pedagogical practice, and is perceived through the

analysis of the context in which Class 1 reduced

forms are inserted.

“This course was very important in my professional

life, has opened up now horizons that further

improved my professional development, providing me

with more knowledge and a more critical look at

things” (Interviewee no.119)

Therefore, the social relevance of knowledge is

QUALITY ASSURANCE IN DISTANCE LEARNING - A Study in Higher Education

297

based on the formation of technically competent

subjects.

The dimension of pertinence of the acquired

knowledge verifies if the social values of the

knowledge that are produced, selected, and

transmitted do correspond to the expectations of the

society which it proposed to create. In other words,

it verifies the adequacy of the knowledge built

throughout the course.

Considering that our target audience is composed

of practicing teachers, the reduced forms included in

Class 3 analyzed by Alceste point to the confidence

students acquire with regards to History contents.

This becomes more evident when analyzing these

forms in context, i.e., in the answers given in the

questionnaire. For instance:

“It helped me to have more confidence when

transmitting the contents

to be worked on and to

present it with assurance, and I find myself more

prepared to research and develop them in class”

(Interviewee no. 200).

The dimension of pertinence of the acquired

knowledge, in our understanding, is closely related

to the way in which student-teachers have managed

to improve their lessons, since this dimension

signals the adequacy of the acquired knowledge

throughout the Licentiate in History program.

The reduced forms in Class 4 refer to the course

itself, i.e., to the didactic materials employed, the

adopted methodology, the tasks that were

performed, among other aspects. This class,

therefore, does not add value to the analysis of the

question at hand.

6 FINAL CONSIDERATIONS

In recent discourses about Education in Brazil, a

strong connection is noticed between the quality of

Fundamental Education and Higher Education,

through the offer of distance learning courses geared

towards teachers continuous education. The offer of

these courses has grown very rapidly in the last few

years. Nonetheless, the assessment of their quality

has focused on the infrastructure offered by Higher

Education Institutions, especially the presential

support infrastructure that is available to the

students.

We believe, however, that it is essential to assess

the course program in terms of how adequate the

knowledge acquired by students and if they are

effectively applied in the student professional life.

This issue is evident in our research results, in which

the students express how important it is to apply the

knowledge acquired throughout the course.

When the students are asked how it has affected

their professional performance, they have pointed to

four speech categories: their professional growth;

the theoretical knowledge of a historical fact; the

confidence in relation to the contents they learned;

the quality of the course’s didactic material.

In 3 (three) of these classes found, it is clear how

important the acquired knowledge pertinence and

relevance are to the reality of the work environment,

from the moment in which this knowledge is put to

practice. The students' speech signals to a sense of

personal fulfillment. Therefore, we consider it

important to verify the manner in which the

assessment of the quality of distance learning

courses has been implemented, so that we can

propose new indices that point to formation

consistency.

REFERENCES

Bauer, Martin W. Análise de Conteúdo Clássica: uma

revisão. In: BAUER, Martin W.; Gaskell, George.

Pesquisa qualitativa com texto, imagem e som: um

manual prático. 6 ed. Petrópolis: Vozes, 2007. p.189-

217.

European Association of Distance Teaching Universities.

Available at <http:// www.eadtu.nl/conference%

2D2007/sup_docs.asp>. Accessed on: Oct. 24, 2007.

Gatti, Bernadete Angelina; Barreto, Elba Siqueira de Sá

(Org.). Professores do Brasil: impasses e desafios.

Brasília: UNESCO, 2009.

Instituto Nacional de Estudos e Pesquisas Educacionais

Anísio Teixeira. Resumo Técnico - Censo da

Educação Superior. 2008. Brasília: INEP, 2009a.

Available at <http://www.inep.gov.br/download/

censo/2008/resumo_tecnico_2008_15_12_09.pdf>.

Accessed on: April 21, 2009.

Ministério da Educação. Secretaria de Educação a

Distância (SEED). Referenciais de Qualidade para

Educação Superior a Distância. Brasília, August 2007.

Available at <http://portal.mec.gov.br/seed/arquivos

/pdf/legislacao/refead1.pdf>. Accessed on: Sept. 06,

2007.

Santana, Flavia Feitosa. A Dinâmica da Aplicação do

Termo Qualidade na Educação Superior Brasileira.

São Paulo: Editora SENAC, 2007.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

298