The Development of an e-Portfolio for Competency based Training
Assessment for a Malaysian Skills Diploma Program
Hafizan Matsom, Tim Stott
and Frances Tracy
Faculty of Education, Community and Leisure, Liverpool John Moore University, Liverpool, U.K.
Keywords: Competency Based Training, Electronic Portfolio, Skills Training Program, Formative Assessment.
Abstract: This paper presents a comprehensive Electronic Portfolio (e-Portfolio) Framework developed for
Competency Based Training (CBT) assessment in a Malaysian Skills Diploma Program which is certified
by the Ministry of Human Resources in Malaysia. The proposed framework is adapted from an existing
assessment system, which was outlined by the Ministry of Human Resources with the addition of other
features from current E-Portfolio technologies. The aim of this study is to present a new comprehensive
framework that contains a combination of e-portfolio applications that align with the current methodology
of CBT assessment as a blended approach to formative assessment. Senior officers at the Ministry of
Human Resources, Principals and Instructors of Training Institutes have been questioned via the medium of
email based interviews to establish their views on the need for this kind of e-Portfolio as well as the possible
constraints that would be faced. All interviewees agreed that the e-Portfolio is well suited for
implementation as an evaluation method to improve the IT skills of the students. However, they also
highlighted constraints that should also be considered before implementation to ensure this system will be
effectively installed and completely functional.
1 INTRODUCTION
Competency-Based Learning/Training (CBL/T) is a
current method used to bring together the gap
between learning in educational settings and future
workplace performance, which represents a
challenge for institutions of higher and further
education (Sluijsmans et al., 2006). In competency-
based learning methods, the models and learning
strategies used must be able to encourage reflection
and reactions from students in an effort to solve the
problems and challenges they face during training.
The students are often tested with problems that
require skills and knowledge based on previous
experience or what they have learned. Competence
is important so that students will master the skills
needed in ‘real world’ industry (Bastiaens, 2010).
Technically, in CBT students need to collect proofs
of their skills during training so that their
competence can be recognised. Normally in
developing countries, evidence collection is done
manually through the compilation of a paper based
file. Research into the use of electronic portfolios in
developed countries in higher and further education
indicates that there are potential benefits to the use
of e-portfolios. The main aim of the e-portfolio is to
encourage inclusive learning with the use of ICT
technology and promoting learning ownership for
students or trainees as well as instructors. This paper
proposes a framework for adapting the concept of
CBT in the e-portfolio for Skills Training program
in Diploma Courses that allows this system to be
embedded in training methodologies. There are
many potential benefits to be gained through the use
of e-portfolios in CBT; such as an improvement in
industrial recognition of the product of students’
work, encouraging an active learning environment,
as well as cultivating the IT skills of students whilst
completing assignments. Moreover, the instructors
could also monitor the performance of students
through their e-portfolio progress to take proper
steps to support students who experience problems.
The development of this framework can also be a
guide for any institution or any parties who intend to
use an e-portfolio in their training program.
386
Matsom H., Stott T. and Tracy F..
The Development of an e-Portfolio for Competency based Training Assessment for a Malaysian Skills Diploma Program.
DOI: 10.5220/0004350003860392
In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 386-392
ISBN: 978-989-8565-53-2
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
2 COMPETENCY BASED
TRAINING (CBT)
In Malaysia, the Skills Training Program was
developed in alignment with this CBT approach.
These programs were organized and coordinated by
the National Vocational Training Council under the
jurisdiction of the Ministry of human Resources. To
implement CBT, a national occupational skills
standard (NOSS) was developed. For every NOSS-
based training program, the learning outcomes to be
achieved are stipulated in the task profiles of the
NOSS, which include performance standards to be
met at the end of the training program (Sachs, 1998).
The training objectives, or outcomes, are specified
and made known to trainees in advance so that
trainees can progress at their own optimum rate. In
other words, the duration of time spent on training
can be a variable but the learning outcomes to be
achieved are considered to be constant (NVTC,
2001, p.8). The focus on outcomes is clearly
reflected in the interpretation of the ‘competency’
concept that underpins the training system based on
NOSS in Malaysia:
“The concept of competency focuses on what
is expected of a worker in the workplace rather
than on the learning process. It embodies the
ability to transfer and apply skills and knowledge
to new situations and environments” (MLVK,
1995, p.1).
The outcome-based orientation of the
competency-based approach is usually characterized
by its strong emphasis on assessment. This can be
seen from an interpretation of assessment which is
typically used in competency-based training as the
“process of collecting evidence and making
judgments on the extent and nature of progress
towards the performance requirements set out in a
standard, or a learning outcome” (Hager et al., 1994,
p.5). By adopting the competency- based approach,
the training system based on NOSS in Malaysia
clearly favours an outcome-based orientation. This
orientation has been accentuated by the
implementation of a national skills certification
system which adopts a criterion- referenced
assessment approach, focusing on performances as
the key basis for assessment and certification
(NVTC, 2001, p.4).
Figure 1: Existing CBT Assessment Process in Malaysia
Skills Diploma Program.
Figure 1.0 shows the process of assessment in the
Malaysian Skills Training Diploma. Each student
must meet the criteria of competence in modular
training programs as well as in the
complement/elective programs. They also need to
accomplish the final project and six months of
industrial training before they are qualified to be
awarded with the Diploma certificate. All
assignments, documents and evidence of
competencies are compiled in a large file called the
student portfolio.
3 E-PORTFOLIO
e-Portfolios can be defined as "a personal, web-
based compilation of work, assignment feedback,
and reflection used to indicate key skills and
achievements in a variety of contexts and time
periods" (Barrett, 2005); (Reardon et al., 2005);
(Turhan and Demirli, 2010). This collection consists
of elements of text, graphics, or multimedia which
can be accessed through a web site or other
electronic media such as CD-ROM or DVD. The
learning portfolio is a flexible, evidence-based
process that combines reflection and documentation.
It engages students in ongoing, reflective, and
collaborative analysis of learning. It focuses on
purposeful, selective outcomes for both improving
and assessing learning. In vocational and education
training which employs the Competency Based
Training approach, few countries have implemented
this application as part of their e-learning education
support program. The use of a portfolio is an
alternative form of learning and assessment that is
particularly attractive to vocational educators
TheDevelopmentofane-PortfolioforCompetencybasedTrainingAssessmentforaMalaysianSkillsDiplomaProgram
387
because it includes the assessment of active learning
and performance rather than the mere recall of
memorized facts (Turhan and Demirli, 2010).
Turhan & Demirli (2010) reported from their study
of vocational education teachers’ and students’
perception of the use of e-portfolios in the United
Kingdom, Denmark, Romania and Turkey showed
that both teachers and students found the e-portfolio
process was necessary in vocational education as
part of the learning session. The outcomes of a
Leonardo da Vinci project (MOSEP - More self-
esteem with my ePortfolio) showed that E-portfolios
can be used as a technology supported learning
method for the documentation of competency
development (Hallam, 2008). This project outlined a
new training concept for teachers and tutors using
open source e-portfolio software tools.
4 E-PORTFOLIO IN MALAYSIAN
SKILLS TRAINING PROGRAM
The Malaysian Skills Certification Program does not
currently make use of any Virtual Learning
Environments (VLE) such as e-portfolios or any
training applications implemented in public or
private training institutions. However, some
countries such as the USA (Abrami et al., 2008);
Lorenzo et al., 2005) UK (Madden, 2007) , Australia
(Gerbic and Maher, 2008); (Hallam, 2008), Portugal
(Queirós et al., 2011) and the Netherlands
(Bastiaens, 2010) have adopted various types of
specialized VLE for students in higher education
including the Vocational Education Training (VET)
sector enhance the quality of the program and to
appeal to parents and prospective students. To obtain
views regarding the development of e-portfolio in
Malaysian Skills Training Programme, an interview
was conducted as part of this project to assess the
opinions of five senior officers from the Department
of Skills Development, Malaysia Ministry of Human
Resource, two managers of Private Accredited
Centre that run Malaysian Skills Training Program
under the Department of Skills Development; and
three instructors cum verifier officers of Accredited
Centre that teach Skills Training Programme (two
were from the private institutions and one was from
Public/Government Institution).These interviews
were conducted via email and the questions related
to an evaluation of current methods of portfolio
documentation, views on the need for a VLE, and
any barriers to implementation that they could
predict. According to senior officers, Department of
Skills Development (DSD) Ministry of Human
Resource at the moment is in the process of
developing a system of ICT applications, which are
known as the National Skills Credit Bank System.
This system intends to upgrade the Skills Training
Program to adopt information technology into the
learning and training methodology (Dollah et al.,
2012). While the instructors and the principals stated
that their campus, currently has not applied a VLE,
they do utilize a common system for recording
student grades and observations on the performance
of trainees (Abd Aziz and Haron, 2012); (Zulkefli et
al., 2012). Analysis of the interviews highlighted
several reasons why e-portfolios should be
introduced to trainees. The requirements were based
on the situation and circumstances as follows (1) To
upgrade the skills training program to be at par with
professional training programs that are recognized
by well-known organizations in and outside the
country; (2) To promote skills training to youth and
adults with increasingly interested in the field of
information technology in line with the current
technology; (3) To facilitate monitoring of the
certification body so that the training process and
requirements of the learning outcomes of the
training centers’ and the trainees will be conformed
according to the standard set.; and (4) To facilitate
the training program in providing promising
opportunities through links with other programs in
other institutes as options for the trainees’ further
education path.
Moreover, according to the instructors, teenagers
now prefer a more flexible training environment and
on-demand learning. They prefer reading materials
and references that give more insight into
professional reality than traditional textbooks. Thus,
e-learning has the potential to motivate students to
explore the internet to find constructive and
beneficial information relating to their learning
rather than wasting time visiting social sites and
playing video games (Zulkefli et al., 2012). The
Principal of the training center also agreed to
implement e-learning such as the e-portfolio in the
training system mainly to provide the performance
report and results of the trainees to parents or a third
party. This system will also maintain the quality of
learning and training by upgrading and enhancing
the training methodology in line with current
learning technologies. Regarding the costs to be
considered, they understand that in order to
implement any kind of reform process, costs and
risks will certainly exist. These costs can be
considered according to the needs and current
situation (Abd Aziz and Haron, 2012). In
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
388
conclusion, the results of interviews with various
parties involved in the skills training program
showed that there is general agreement that the e-
portfolio should be implemented in the program to
the advantage of trainees, instructors, the awarding
body, parents and any other parties that are directly
or indirectly involved with skills training based
education. Furthermore, they suggested other kinds
of e-learning methods, which may have various
benefits for instance a learning management system
or mobile learning.
4.1 Proposed CBT Blended Assessment
Model
The purpose of developing this E-portfolio is to
enhance current assessment methodologies in the
Skills Training Program to a method called CBT
Blended Assessment. This blended assessment will
combine online participation as well as face-to-face
evaluation by students. For online participation,
students have to create an electronic portfolio in the
system provided. Figure 2.0 shows components of
assessment that will be involved in the E-portfolio
system in CBT Blended Assessment Model.
Figure 2: Proposed CBT Blended Assessment Model in
Malaysia Skills Diploma Program.
4.1.1 e-Portfolio Development Framework
The purpose of the CBT e-Portfolio Framework
development is to pave the way for the development
of well planned and organized specific e-portfolio
for CBT. A framework of e-portfolio based CBT is
developed based on criteria of CBT. This framework
consists of three elements which are modules and
task performance objectives, performance
assessment and construction of trainee’s e-
portfolio for each module.
Module and Task Performance Objectives:
Each module has several tasks that should be
accomplished by the trainee in order to achieve
competence. The modules have certain performance
objectives as well as containing tasks. These
objectives will be aligned with a skills and
competencies matrix for respective modules.
Trainees must comprehend this matrix in order to
plan the progression of their work. Performance
Assessment: Each module must have a complete set
of performance assessments to audit skills
competencies. This assessment must allow formative
as well as the compulsory summative assessment.
A formative evaluation (sometimes referred to as
internal) is a method for judging the worth of a
program while the program activities are forming (in
progress). CBT learners must know the work
process in solving a given task. Therefore, they have
the responsibility to work through the process in the
correct order. Progress must also been evaluated and
commented on with the aim of raising learning
standards. Trainee’s e-Portfolio: After completing
the phases such as identification of skills
competencies and assessment needs, trainees can
begin to construct a collection of pages for each
module. Each module collection must contain the
skills matrix, progress of work and the completed
final work. The skills matrix is also used as a
performance rubric or field monitor for the instructor
to identify the level of competence of trainees for
each given assignment. In the student’s personal e-
portfolio account, they must include all
documentation, instructions and resources. Table 1.0
shows an example of evidence required in the e-
portfolio.
Table 1: Examples of Contents in e-Portfolio.
5 RESULTS OF PILOT TEST
A three months pilot test of E-portfolio system had
been implemented to a group of students from one
private Skills Training Institute in Malaysia. The
pilot session involved 23 students and 2 instructors
which are from Diploma Skills in Culinary Art
TheDevelopmentofane-PortfolioforCompetencybasedTrainingAssessmentforaMalaysianSkillsDiplomaProgram
389
Program. The scope of this session begins with both
student and instructor will sign up for a personal
account and fill the basic information required,
instructor then publish a performance assessment
question with dates of submission/progress, student
starts to develop plan using ‘PLAN’ function in the
system and insert the dates given as well as the other
content like materials for assignment preparation,
student place their progress of work in their personal
E-portfolio account and finally get feedback from
the instructor. From the analysis based on user
statistics in E-portfolio monitoring system, all 23
students were successfully signed up on to the
system. However, only 15 students were involved in
assignment tasks with only 60% from them
submitted until final submission. Then, students
were given a questionnaire that asking their opinion
about how they perceived about the system. From 23
students overall, only 8 students were filled in the
online questionnaire provided. Below are the
summary of the participation in the pilot test session.
Table 2: Summary of Students’ Participation in e-Portfolio
Pilot Test.
Result from questionnaire is presented in the
histogram (Refer Figure 3.0 ). Majority of these
graphs were representing the modes of frequency
after accumulating the result from sub-questions.
There are eight main questions asked which are
Perceived Usefulness, Perceived Ease of Use,
Computer Self-Efficacy, Image, Facilitating
Conditions, Competence Expectancy, Perceived
Playfulness and Behavioral Intention in 5 scales
rating questionnaires (1:Strongly Disagree,
2:Disagree, 3:Neither agree or disagree, 4:Agree,
5:Strongly Agree).
Overall, by refer to the histograms of Perceived
Usefulness, Image, Competence Expectancy and
Behavioral Intentions, majority of students agreed
that the E-portfolios are useful to facilitate the
assignment completion and recognize the key
competencies required for the module. In addition,
they also agreed that by using this system, they can
enhance their image into a technologically advanced
student. Ultimately, they decided to continue to use
the system during the course and will invite their
friends to join using this system in the future.
However, for technical support issues, only some of
them felt that there is a support team available to
assist when in problems while others favor neutral.
The same result shows on issue Perceived
Playfulness which was only partially felt that by
using this E-portfolio system will adds more
pleasure and joyful during the training courses.
On the other hand, it can be found that most of
the students were having moderately skills in
computer usage and application. This was evident
when many of them choose intermediate for
Computer Self-Efficacy issues. In the same case, the
issue of Perceived Ease of Use also showed
moderate rating for mainly students. This clearly
indicates that with low skills in computer
applications, this system viewed rather difficult to
use during training.
Figure 3: Result of questionnaires.
6 IMPLEMENTATIONS ISSUES
AND CHALLENGES
Senior officers stated that the main constraint to
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
390
implementation of a VLE or online learning for
these training programs is difficulty in getting
internet access services in remote areas such as
Sabah and Sarawak and the upstream side of the
village in the peninsula. The instructors’ opinions
were that any kind of system could be implemented
but it would require a longer time to familiarize
trainees to internalize the system due to the attitude
and skills or weaknesses in the trainees themselves
in terms of adopting these technologies. Principals
also stated that in order to implement the reforms,
many things need to be considered by various
parties, such as time, cost and manpower. All of
which has to be approved by various levels of
meetings with stakeholders. In addition, the system
development process also requires review and
comprehensive study to produce a system that really
functions and satisfies the end user. Therefore, these
constraints should be considered before making the
decision to implement e-learning. These are
summarized as follows: (1)The supply of hardware
resources such as computer equipment, internet,
network, servers and other hardware involved in
every centre that participates; (2)The provision of
software and applications must be easy to develop
and maintain as well as low in cost if licensed; (3)
Intensive training to trainees and all institutional
personnel or departments which will be involved
with the system; (4) Moral and financial support
from the stakeholders and management; and (5)
Costs like resources, manpower and time periods
should be considered from the beginning of a
process of planning, implementation, testing up to
the assessment of effectiveness.
7 CONCLUSIONS
This paper has presented proposed changes to the
current evaluation model in Malaysia Skills Training
Diploma to a new CBT blended assessment model
which adopts the concept of an e-portfolio. This e-
portfolio system has received positive feedback from
various groups such as senior officers at the
ministry, principals and instructors of the training
institutes. Numerous people have proposed that there
may be advantages to the student like time and cost
saving, enhancing student’s active learning and
giving students access to the latest learning
technology. Although there are several constraints
that need to be addressed and taken into account
such as the high cost of short and long term
investment for the facilities and equipment, lack of
student’s and instructor’s awareness as well as the
political intervention, these people believed that the
new model would be of benefit and successful.
Proposals to develop e-portfolios have full support
from all parties involved with the Skills Training
program in Malaysia. This system will hopefully
help the trainees to be more reflective in their
training and deepen the objectives of their courses.
REFERENCES
Abd Aziz, A. S., & Haron, M. H. (2012). Personal
Communication. Sept 2012. Kuantan, Pahang,
Malaysia.
Abrami, P. C., Wade, C. A., Pillay, V., Aslan, O., Bures,
E. M., & Bentley, C. (2008). Encouraging self-
regulated learning through electronic portfolios.
Canadian Journal of Learning and Technology.
Retrieved from http://www.cjlt.ca/index.php/cjlt/
article/view/507/238
Barrett, H. C. (2005). Researching Electronic Portfolios
and Learner Engagement. October.
Bastiaens, T. (2010). Competency-based education in an
electronic- supported environment : an example from a
distance teaching university Markus Deimann and.
International Jurnal Cont. Engineering Education and
Life Long Learning, 20, 278–289.
Dollah, M. N., Saedon, M. A., Taja Arifin, H., Abd Razak,
M. L., Rakon, Z., & Abd Hamid, N. (2012). Personal
Communication. Sept 2012. Putrajaya: Department Of
Skills Development, Malaysia.
Gerbic, P., & Maher, M. (2008). Collaborative self-study
supporting new technology: The Mahara e-portfolio
project. Hello Where are you in the landscape of
educational technology Proceedings ascilite
Melbourne 2008, (2005), 320–324. Retrieved from
http://www.edna.edu.au/edna/referral/advsearch/http://
www.ascilite.org.au/conferences/melbourne08/procs/g
erbic.pdf
Hager, Paul, Gonczi, Andrew and Athanasou, James
(1994): General issues about assessment of
competence. Assessment & Evaluation in Higher
Education, 02602938, Apr94, Vol. 19, Issue 1.
Hallam, G. (2008). The Australian ePortfolio project and
the opportunities to develop a community of practice,
368–372.
Lorenzo, G., Ittelson, J., & Oblinger, D. (2005).
Demonstrating and Assessing Student Learning with
E-Portfolio. (D. Oblinger, Ed.)ELI Paper 32005.
Retrieved from http://www.educause.edu/ir/library/
pdf/ELI3003.pdf
Madden, T. (2007). Development , Adaptation and
Implementation of the e-Portfolio Framework.
October.
NVTC. (2001). Government Circular :National skill
certification in Malaysia. Kuala Lumpur: NVTC.
Queirós, R., Oliveira, L., Leal, J. P., Moreira, F., Inesc-
porto, C., & Ipp, K. E. (2011). Integration of
TheDevelopmentofane-PortfolioforCompetencybasedTrainingAssessmentforaMalaysianSkillsDiplomaProgram
391
ePortfolios in Learning Management Systems.
Portfolio The Magazine Of The Fine Arts, 500–510.
Reardon, R. C., Lumsden, J. A., & Meyer, K. E. (2005).
Developing an E-Portfolio Program: Providing a
Comprehensive Tool for Student Development,
Reflection, and Integration. NASPA Journal, 42(3),
368–380. doi:10.2202/1949-6605.1513
Sachs, R. (1998). Development of National Occupational
Skill Standards (NOSS). Workshop – Task Analysis.
Kuala Lumpur: MLVK
Sluijsmans, D. M. a., Prins, F. J., & Martens, R. L. (2006).
The Design of Competency-Based Performance
Assessment in E-Learning. Learning Environments
Research, 9(1), 45–66. doi:10.1007/s10984-005-9003-
3
Turhan, M., & Demirli, C. (2010). The study on electronic
portfolios in vocational education : The views of
teachers and students in. Romania, 5(11), 1376–1383.
Zulkefli, N. A., Yusoh, Y., & Ibrahim, R. (2012). Personal
Communication. Sept 12. Kuantan, Pahang, Malaysia.
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
392
