Exploratory Study of Effects of Learning System Acceptance on
Learning Program Outcomes
Fusing the Technology Acceptance and Technology Mediated Learning Models
Biswadip Ghosh and Jonathon Pries
CIS Department, Metropolitan State University of Denver, Denver, Colorado, U.S.A.
Keywords: Technology Mediated Learning, Unified Theory of Acceptance and Use of Technology.
Abstract: End-user learning is an important element of Information Systems (IS) projects inside organizations. End-
user learning can constitute roughly 5% to 50% of project budgets. To lower costs and make learning more
convenient for the end-users, organizations are largely utilizing online systems for the electronic delivery of
such learning programs, referred to as Technology Mediated Learning. In this scenario, before the end-
users are able to immerse themselves in the actual learning program, they are first required to adopt and use
an online learning system. Currently published IS research has two mature streams of publications: one
stream focused on models of technology acceptance and usage that has lead to the UTAUT (Unified Theory
of Acceptance and Use of Technology) model and a second stream focused on the TML (Technology
Mediated Learning) framework of learning structures and outcomes. This research study aims to build and
validate an empirical model to study of effects of learning system features, content and structure from the
TML framework on acceptance and adoption constructs from the UTAUT model and measure how they
impact learning outcomes By surveying users of an online learning system and their usage behaviour of
specific learning system capabilities, this study measures the acceptance and usage of the system and the
learning outcomes of mastering MS-Office productivity software. The results of this study have
implications for both the UTAUT and TML research streams and also the design and use of technology
mediated learning by practitioners.
1 INTRODUCTION
End-User learning is one of the most pervasive
methods for developing human resources within
modern organizations to effectively deploy and use
Information and Communications Technology (ICT)
in their business operations. Majority of learning
deals with teaching end-users how to use computer
applications and gain skills to do their assigned jobs
in the organization. There are three targeted goals of
most end-user learning programs (Gupta, et al.,
2010): (1) skill-based goals (tool procedural) that
target the user’s ability to use the system, (2)
cognitive goals (tool conceptual or business
procedural) that focus on the use of the system to
solve business problems and (3) meta-cognitive
goals that focus on building the individual’s belief
regarding their own abilities with the system. To
lower costs and make learning more convenient and
schedule friendly for employees, modern
organizations are currently utilizing online systems
for the electronic delivery of end-user learning
(ASTD, 2011). Recent reports suggest that upwards
of 40-50% of end-user learning is conducted through
technology mediated leaning (TML) systems
(ASTD, 2011). A comprehensive TML research
framework is elaborated in Gupta and Bostrom
(2009). In the TML framework, the learning
structures (or scaffolds) support the delivery of the
learning content, such as the rules, resources and
methods, the level of detail in the instructions given
to participants, the guidance provided by the
facilitator and the nature of the facilities and
equipment used in the learning session.
Most commercial TML systems typically are
feature rich applications that support various
learning tasks and learning scenarios. The set of
features allow end users to search learning content,
build a customized learning program by planning a
sequence of courses, manage their learning progress
and even receive a certificate on completion. With
95
Ghosh B. and Pries J..
Exploratory Study of Effects of Learning System Acceptance on Learning Program Outcomes - Fusing the Technology Acceptance and Technology
Mediated Learning Models.
DOI: 10.5220/0004382300950100
In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 95-100
ISBN: 978-989-8565-53-2
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
the popularity of TML appliations and an increase in
cloud based technologies, there is vast diversity in
these online learning systems, which employ various
platforms and software architectures that pose a
variety of challenges (Bensch and Rager, 2012). IS
researchers have long called for additional research
into the questions of how such technology enhances
the learning processes and outcomes (Alavi and
Leidner, 2001). Published research has found that
the learner’s acceptance of e-Learning technologies
and specific application features have been found to
be important factors that strive to address some of
these questions (Lee, Yoon, Lee, 2009; McGill and
Klobas, 2009; Piccoli, Ahmed and Ives, 2001).
The UTAUT framework models the factors that
govern the acceptance, behavioural expectations and
the ultimate usage behaviour of a technological
(Venkatesh, Thong and Xu, 2007). It has evolved
over 20 years from TAM (Technology Acceptance
Model Theory -Davis, 1989) as a vehicle for
evaluating factors that impact an individual’s
acceptance and use of technology. The TAM model
conceptualized the relationship between perceived
ease of use (the level of difficulty of adopting a
technology) and the perceived usefulness of the
technology (the user’s performance expectations) on
the user’s intentions to use the technology. Several
research studies have applied the Technology
Acceptance and Use Model to understand effects of
the pedagogical design of such e-learning systems.
The focus has been on the impact of learning system
features such as learning activities, security,
information and service quality, interactivity and
responsiveness, learner control and the ability to
self-organize their learning on the user’s acceptance
of those systems (Selim, 2003; Pituch and Lee,
2006; Roca and Gagne, 2008; Sun, et.al., 2008).
Recent TML based research studies about the
effectiveness of online learning systems on end user
learning task conformance and learning outcomes
has been ambivalent (Gupta and Bostrom, 2009).
Some have reported positive impacts, while others
have not. Such results support the need to merge
additional constructs into the TML framework to
represent the user’s technology acceptance and
usage behaviour.
1.1 Research Goals
The focus of this research study is to answer the
question “Does the level of acceptance and use of
features and capabilities of an online learning system
impact learning outcomes?” To answer this
question, the paper extends the TML framework
with constructs from the UTAUT model and their
impacts on learning appropriation and outcomes.
The goals of this study are:
To develop and empirically validate an extended
TML research model that also includes the users’
learning system usage behaviour and the
facilitating conditions supporting such usage.
To measure the impacts of the usage behaviour and
facilitating conditions on the users’ learning
outcomes.
2 BACKGROUND THEORY
Information technology deployed in typical learning
programs is used as a primary structural element in
the learning process (e.g. Simulations or exercises
that are part of the learning process) or as a
secondary tool in the learning process (e.g.
Computer based tests and quizzes) The latter
approach implies the technology is part of the
learning delivery process. However, the actual use
of the features and capabilities of an online learning
system have been found to differ across groups of
users (Bekkering and Hutchison, 2009). Individual
differences play a role in what features of these
systems are used and how the systems can impact
each end-users’ learning process and outcome
(Gupta, Bostrom and Anson, 2010). The current
research stream of IS end-user learning has studied
the impact of the above learning structures on
different learning outcomes along with various
confounding factors such as the individual’s learning
style, their motivation to participate and their
interest in the learning content (Bostrom, et.al.,
1990; Nogura and Watson, 2004). While the TML
model incorporates technology as a structural
element of learning delivery, it does not take into
account the usage behaviour of the specific
capabilities of the learning platform by the
individual users. Individual differences can impact
learning outcomes by generating a different mental
response to the learning content and influencing
their interactions with the learning delivery
structures. Learning style of the user plays an
important role in the user’s conformance to the
learning tasks embedded in the online learning
system (Bohlen and Ferratt, 1997).
Abstract learners perform better than users with
concrete learning styles in online technology based
learning. The trainee’s motivation and attitudes also
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
96
Figure 1: Research Model.
have been found to influence learning performance
in the TML context (Szajna, B. and Mackay, J.M.,
1995; Yi and Davis, 2003). Both intrinsic motivation
and extrinsic motivation played a role in the
adoption and appropriation of the learning system
capabilities and the completion of the online
learning regimen (Gupta, Bostrom and Anson,
2010). Intrinsic motivation has been found to
influence the personal innovativeness of the learner
that directly impacts how they deal with obstacles
faced with the learning system.
The technology acceptance model (TAM) is one of
the most widely used models used in Information
systems research to study the adoption and usage
intensions of individual users towards information
systems (Ajzen, 1988; Ajzen, 1991). TAM was
developed by Davis (1989) to explain the
determinants of the intention to use computer
systems. The UTAUT model extended that TAM
framework to include facilitating conditions and
individual differences that can influence the user’s
intentions to use a technology (Venkatesh, Thong
and Xu, 2012). The UTAUT model also has factors
that are related to the working environment in the
organization, such as social influence and facilitating
conditions. The UTAUT model includes age, gender
and experience with technology as important
individual differences that moderate intension to use
and actual usage behaviour.
3 RESEARCH MODEL
The research model is displayed in Figure 1. The
research constructs are defined in the following
subsections. The dependent variable in the model is
Learning Outcomes (LO).
The independent variables are the three components
of the TML system (modelled as a formative second
order construct) – (i) Learning system features
(LSF), (ii) Learning Content (LC) and (iii) Learning
Structures (LS). The Individual characteristics (IC)
and Facilitating Conditions (FC) are also
independent variables in the model.
3.1 Learning Outcomes
Learning outcomes (LO) focus on the mental
awareness and judgements of the end-user and the
levels of application of acquired knowledge towards
operating business functions (Gupta, et al., 2010).
The learning outcomes is a formative construct that
consists of three types of outcomes – skill based,
cognitive and meta-cognitive.
3.2 Learning Content and Learning
Structures
Learning content (LC) refers to instructional
methods that encourage students to accomplish
learning goals. These allow end-users to fill gaps in
their understanding and builds skills (skill focus) and
knowledge about how they can use the system to
improve their productivity (cognitive focus). “Soft
skills” are also developed that allow members to
learn collective beliefs and norms that help them
develop confidence and knowledge in solving future
business problems.
ExploratoryStudyofEffectsofLearningSystemAcceptanceonLearningProgramOutcomes-FusingtheTechnology
AcceptanceandTechnologyMediatedLearningModels
97
Learning structures (LS) refer to the scaffolds that
support the delivery of the learning content. Also
referred to as appropriation support (Gupta, et.al,
2010), they include the rules, resources and methods
that support the elements of the collaborative
learning session. For this research study, the
learning structures include level of detail in the
instructions given to participants, the guidance
provided by the facilitator and the nature of the
facilities and equipment used in the learning session.
3.3 Learning System Features (LSF)
As the use of TML in learning programs intensifies,
the need to list the features of such applications as a
component of the overall learning system is more
important. Capabilities mentioned in the research
stream refer to responsiveness and quality (Lee,
Yoon & Lee, 2009), feedback and facilitation of
communications about assigned instructional work
(Putuch & Lee, 2006), flexibility, autonomy and
user control of the learning process and steps
(Piccoli, Ahmad and Ives, 2001).
3.4 Individual Characteristics (IC)
People prefer learning methods based on their
specific learning styles (Nogura and Watson, 2004).
Individual differences influence the formation of
mental models, which effects the learning process.
“States” are general influences on performance that
vary over time and include temporal factors such as
motivation level and interest level (Bostrom, et.al.,
1990). “Traits” are static aspects of information
processing affecting a broad range of outcomes.
Cognitive traits refer to learning styles such as a
preference for procedural or abstract knowledge and
an exploratory or reflective approach to instructional
content delivery format (Bostrom, et.al., 1990;
Nogura and Watson, 2004). For this research study,
the Individual characteristics (IC) variable is
measured using motivation and interest as states and
individual learning style as traits. Both intrinsic
motivation and extrinsic motivation influences the
learner’s state and is measured in the survey.
3.5 Performance Expectancy and
Effort Expectancy
Two key components were used in the original TAM
model – perceived usefulness an the perceived ease
of use of any technology innovation. The UTAUT
model includes two components – Performance
Expectancy and Effort Expectancy (Venkatesh,
Thong and Xu, 2012). Performance Expectancy
(PE) is referred to as the “degree to which a person
believes that using a particular system will enhance
their performance” (in a job or activity). Effort
Expectancy (EE) defines the “degree to which a
person believes that using a particular system would
be free of effort”. It is posited that intention to use
and actual usage of a system will positively depend
on both constructs (Venkatesh, et. al., 2003).
3.6 Facilitating Conditions (FC)
Facilitating conditions are environmental factors that
refer to the users’ perceptions of resources and
support to use the technology (Venkatesh, et. al.,
2008). In the context of a learning system,
facilitating conditions include resources,
accessibility, compatibility with other systems,
infrastructure quality and support (McGill and
Klobas, 2009; Venkatesh, et.al., 2008).
3.7 Behavioural Intentions and Usage
Behaviour
Behavioural intentions (BI) and actual usage
behaviour (UB) to use the technology are part of the
original TAM and the UTAUT models (Venkatesh,
et. al., 2003). Behavioural intentions imply the
plans and intentions to use the system. Such
intentions can be habit forming and also be
constituted from the users’ past experiences. Actual
usage behaviour refers to the duration, frequency
and intensity of the use of the system (Venkatesh,
et.al., 2008).
3.8 Research Hypotheses
The research hypotheses are listed below. Given the
exploratory nature of this study, the emphasis is to
model and test various possible relationships across
constructs in the TML and UTAUT models.
H1: The TML System has a positive effect on
Learning Outcomes.
H2a: Individual Characteristics will moderate
the relationship between the TML System
and Learning Outcomes.
H2b: Individual Characteristics will have a
positive effect on Learning Outcomes.
H2c: Individual Characteristics will moderate
the relationship between Use Behaviour
and Learning Outcomes.
H2d: Individual Characteristics will moderate
the relationship between Behavioural
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
98
Intention and Use Behaviour.
H3a: The Learning System Features will have a
positive effect on Performance Expectancy.
H3b: The Learning System Features will have a
positive effect on Effort Expectancy.
H4: The Learning Content will have a positive
effect on Use Behaviour.
H5: The Learning Structures will have a
positive effect on Behavioural Intention.
H6a: Performance Expectancy will have a
positive effect on Behavioural Intention.
H6b: Effort Expectancy will have a positive
effect on Behavioural Intention.
H6c: Facilitating Conditions will have a positive
effect on Behavioural Intention.
H7: Behavioural Intention will have a positive
effect on Use Behaviour.
H8: Use Behaviour will have a positive effect
on Learning Outcomes.
4 METHODOLOGY
4.1 Data Collection
Our data collection approach will consist of
surveying a collection of approximately 400
business school students, who use an online
learning system, “MyITLab”, to learn spreadsheet
and database software applications. The survey
consists of 3 questions for each construct and uses a
5 point Liekert scale (1 being strongly disagree and
5 being strongly agree) to measure user responses to
each item. The survey is included in the appendix.
A pilot study has been completed with 45 users
and reliability and validity of the survey instruments
has been calculated (Table 2).
4.2 “Myitlab” Learning System
MyITLab (www.myitlab.com) is a feature rich
learning application that allows users to complete a
variety of simulated tutorial exercises and case
studies with Microsoft excel and access software
packages. While some parts of the system can be
cumbersome and requires extensive scaffolding,
such as initial registration, login and a properly
configured browser for accessibility, yet the major
benefits of using the system are quick feedback on
assignments, interactive help on various procedural
aspects of Excel and Access software and
organization of the learning process.
5 PRELIMINARY RESULTS
The 45 surveys collected from the pilot study were
analyzed with SPSS version 20 (factor analysis,
Scree plot and Cronbach’s alpha) and results are
presented in Tables 1 and 2. Factor analysis with
Varimax rotation found 6 factors with eigenvalues
greater than 0.9 with an explained variance of
78.4%. Four constructs – Learning Outcomes (LO),
Learning System Features (LSF), Usage Behaviour
(UB) and Individual Characteristics (IC) are
formative and do not show as factors. The results
also indicate adequate reliability and (Cronbach’s
Alphas are above .67 for all constructs) and
discriminant validity to proceed with the full data
collection during Feb-March 2013. The full survey
will be completed and results and hypotheses test
outcomes will be presented at the conference.
Table 1: Demographic Variables (n = 45).
Variable Min Max Mean S.D.
Years of Edu (yr) 2 6 2.96 0.92
Prior Excel Use (yr) 0 8 2.36 1.95
Prior Access Use
(yr)
0 6 0.93 1.25
Weekly Usage in Hrs 1 15 3.57 2.66
Gender Male: 28 Female: 17
Table 2: Factor Analysis and Construct Reliability (CR).
PE LS FC BI EE LC
Cronbach’s
Alpha
.89 .74 .72 .79 .85 .67
Item 1 .81 .32 .74 .66 .37 .66
Item 2 .73 .82 .82 .81 .71 .32
Item 3 .77 .22 .21 .58 .63 .22
REFERENCES
Alavi, M. and Leidner, D.E. (2001). Research
Commentary: Technology-Mediated Learning – A
Call for Greater Depth and Breadth of Research,
Information Systems Research 12(1), pp. 1-10.
ASTD 2011, ASTD State of the Industry Report.
http://www.astd.org, ASTD.
Bekkering, E. and Hutchison, D. 2009, A Follow-up Study
of Using Remote Desktop Applications in Education,
Information Systems Education Journal 7 (55), June
12, 2009.
Behsch, S. and Rager, M. (2012). Cloud-Based Online
Learning Platforms, Proceedings of BIS 2012
Workshops, LNBIP 127, pp. 165-176.
Bohlen, G.A. and Ferratt, T.W. 1997, End User learning:
An experimental comparison of lecture versus
computer-based learning, Journal of End User
Computing 9 (3), pp 14-27.
ExploratoryStudyofEffectsofLearningSystemAcceptanceonLearningProgramOutcomes-FusingtheTechnology
AcceptanceandTechnologyMediatedLearningModels
99
Bostrom, R. P. and Huber, M. 2010. End-User Learning
Methods: What we know, Need to Know, The
DATABASE for Advances in Information Systems
41(4), pp. 9-39.
Bostrom, R. P., Olfman, L. and Sein, M. K. 1990. The
Importance of Learning Style in End-User Learning,
MIS Quarterly 14(1), pp. 101-119.
Davis, F. D. 1989. Perceived usefulness, Perceived ease
of use and user acceptance of information technology,
Management Information Systems Quarterly, 13(3),
pp. 319-340.
Gupta, S. and Bostrom, R. P. 2009, Technology-Mediated
Learning: A Comprehensive Theoretical Model,
Journal of the Association for Information Systems
10(9), pp. 686-714.
Gupta, S., Bostrom, R.P. and Anson, R. 2010, Do I
matter? The Impact of Individual Differences on
Learning Process, Proceedings of the 2010
SpecialInterest Group on MIS’s 48
th
Annual
Conference on Computer Personnel Research, pp. 112-
120.
Lee, B. C., Yoon, J.O., Lee, I. (2009). Learner’s
Acceptance of e-Learning in South Korea: Theories
and results, Computers and Education (53), pp. 1320-
1329.
McGill. T. J. and Klobas, J.E. (2009). A task-technology
fit view of learning management system impact,
Computers and Education (52), pp. 496-508.
Noguera, J. H. and Watson, E.F. 2004. Effectiveness of
Using an Enterprise System to teach process centered
concepts in business education. Journal of Enterprise
Information Management 17(1), pp. 56-74.
Piccoli, G., Ahmed, R. and Ives, B. (2001). Web-Based
Virtual Learning Environments: A Research
Framework and a Preliminary Assessment of
Effectiveness in Basic IT Skills Training, MIS
Quarterly 25 (4), pp. 401-426.
Pituch, K. and Lee,Y. (2006). The influence of system
characteristics on eLearning use, Computers and
Education 47(2), pp. 222-244.
Roca, J. C. and Gagne, M. (2008). Understanding
eLearning continuance intention in the workplace, A
self-determination theory perspective, Computers in
Human Behavior 24(4), pp. 1585-1604.
Selim,H. M. (2003). An Empirical Investigation of
Student Acceptance of Course Websites, Computers
and Education 53(3), pp. 588-598.
Sun, P. C., Tsai, R. J., Finger, G., Chen, Y. Y., and Yeh,
D. (2008). What drives a successful eLearning? An
Emperical investigation of the critical factors
influencing learner satisfaction, Computers and
Education 50(4), pp. 1183-1202.
Szajna, B. and Mackay, J. M. 1995. Predictors of Learning
Performance in a Computer-User Learning
Environment: A Path-Analytic Study, International
Journal of Human-Computer Interaction 7 (2), pp.
167-185.
Venkatesh, V., Thong, J. Y. L. and Xu, X. 2012,
Consumer Acceptance and Use of Information
Technology: Extending the Unified Theory of
Acceptance and Use of Technology, MIS Quarterly
36(1), pp. 157-178.
Yi, M. Y. and David, F. D. 2003. Developing an
Validating an Observational Learning Model of
Computer Software Learning and Skill Acquisition,
Information Systems Research 14 (2), pp. 146-170.
APPENDIX
The Survey Instrument is below:
Excel Usage Experience (in years) ____Years of Education _____
Access Usage Experience (in years) _______ Gender: M F
How many hours/week on average, did you use MyITLab?___
LO1-I understand how I can navigate Excel and Access
LO2-I am confident I can finish an assigned task with office
LO3-I can use features of Excel and Access to solve problems
UB1-I used all of the available features of MyITLab.
UB2-I used MyITLab a lot compared to other learning system
FC1-I had the resources necessary to use MyITLab
FC2-I had the knowledge necessary to use MyITLab.
FC3-I had all the support necessary to use MyITLab
BI1-I had a favourable attitude towards using MyITLab.
BI2-I never disliked using MyITLab
BI3-I am satisfied with the guidance provided by my
instructor in the learning process.
LC1-I would use MyITLab to learn another application.
LC2-The learning materials provided me with enough
details.
LC3-I am satisfied with the documentation of MyITLab
EE1-It was very easy for me to learn to use MyITLab.
EE2-It was easy to find information about MyITLab
EE3-I found MyITLab to be very easy to use.
LSF1-The output from MYITLab was presented in a
useful format.
LSF2-The information from MyITLab is accurate.
LSF3-MyITLab allowed me to take control of my learning
process
PE1-Using MyITLab enhanced my effectiveness in
learning.
PE2-Using MyITLab increased my productivity in the
course
PE3-I found MyITLab to be very useful
LS1-I am satisfied with the facilities and equipment that
were available for my use in the learning process.
LS2-MyITLab system fits well with the way I like to learn
LS3-I understood the policies around using MyITLab.
IC1-I was motivated to learn as much as I can from this
class.
IC2-I was very interested to take this class.
IC3-I was excited about learning the skills that were
covered
CSEDU2013-5thInternationalConferenceonComputerSupportedEducation
100
