THE IMPACT OF TECHNOLOGY-ENABLED ACTIVE
LEARNING ON STUDENT PERFORMANCE, GENDER
AND ACHIEVEMENT LEVEL
Ruey S. Shieh
Tatung Institute of Commerce and Technology, 253 Mi-Tou Road, Chia-Yi, Taiwan, R. O. C.
Keywords: Technology Enabled Active Learning (TEAL), Quasi-experimental Research.
Abstract: Technology Enabled Active Learning (TEAL) is an innovative pedagogical method emphasizing active,
interactive learning. It is implemented in a technology-rich, multimedia studio. The National Chung Cheng
University (CCU) is the first university in Taiwan to use the TEAL studio to teach science courses. The
purpose of this study is to assess the impact of TEAL on student learning at CCU, including the influence on
genders and achievement levels. A quasi-experimental research was designed to conduct this study. Data
sources consist of a pre-test, a post-test, and a survey. The preliminary results indicate that (1) the
experimental (TEAL) students significantly outperformed the control (traditional classroom) students, (2)
the experimental male students outperformed their counterpart male students, (3) the learning gain achieved
by the experimental female students was more significant than that of their counterpart female students, and
(4) the experimental low-achieving students achieved the highest learning gain among the different
achievement-level groups. Narrowing the learning gap between different achievement levels of students
through the use of this technology-enhanced learning approach appears promising. Nonetheless, continuing
to improve the teaching innovation is necessary, particularly in terms of more effectively integrating
technology-enabled features into teaching.
1 RESEARCH BACKGROUND
Technology-Enabled Active Learning (TEAL) was
developed by MIT (Massachusetts Institute of
Technology) in 2001. It is an innovative teaching
and learning format featuring multimedia-equipped
studios to facilitate students learning science and
technology-related courses. TEAL emphasizes group
discussion and interaction during the teaching
process. The group interaction and discussion is
accomplished through the support of built-in
assessment processes, such as the personal response
system (PRS), which some educators consider a
powerful tool for teaching science courses (e.g.
Beatty & Gerace, 2009; Beatty, et al., 2006). In
Taiwan, the National Chung Cheng University (CCU)
is the first university using TEAL to improve student
learning.
At CCU, students majoring in sciences and
engineering disciplines are required to take the
General Physics (GP) course in their freshman year
offered by the Physics Department. In order to help
students better understand the abstract concepts
associated with basic physics, the notion of
technology-enabled active learning was introduced
to its campus in 2004. Similar to the TEAL studio
established at MIT, the CCU TEAL studio was
equipped with tables, big-screen projectors around
the walls, and blackboards in between the projectors.
Figure 1 shows an overhead view of the TEAL
studio and Figure 2 displays a scene of a hands-on
activity.
Figure 1: An overhead view Figure 2: A hands-on Activity.
of the TEAL studio.
In 2005, CCU began to use the TEAL studio to teach
420
Shieh R. (2010).
THE IMPACT OF TECHNOLOGY-ENABLED ACTIVE LEARNING ON STUDENT PERFORMANCE, GENDER AND ACHIEVEMENT LEVEL.
In Proceedings of the 2nd International Conference on Computer Supported Education, pages 420-425
DOI: 10.5220/0002783504200425
Copyright
c
 SciTePress

the GP course to the physics major students. By the
year of 2007, there were three classes of
department-wide students studying the GP course in
the TEAL studio. Based on the students’ learning
outcomes in 2007, the experimental (TEAL)
students and the control (traditional classroom)
students achieved about the same learning gains in
the first semester (6.43% vs. 6.34%); however, the
experimental students achieved much higher
learning gain than their counterpart students in the
second semester (18.09% vs. 11.20%). Nevertheless,
this learning gain of below 20% was still considered
relatively small (Hestenes & Halloun, 1995).
Improvements for implementing the innovative
pedagogy were identified, including improving
teaching skills, reconsidering the content coverage,
and more deliberately integrating the lab activities
into the lecture content. More detailed findings are
to be published elsewhere.
The purpose of this study is to continue assessing
the impact of TEAL on student learning
demonstrated in 2008. In addition to comparing the
learning outcomes of the experimental and control
groups, this study also examines the learning gains
achieved by the different genders and different
academic achievement-levels. Three research
questions are addressed in the study:
1. To what extent does technology-enabled active
learning impact student learning for the
experimental and control groups?
2. To what extent does technology-enabled active
learning impact student learning between the
genders?
3. To what extent does technology-enabled active
learning impact student learning among high,
intermediate, and low achieving levels?
2 LITERATURE REVIEW
Many researchers have studied the social aspect of
cognitive activities, such as communities of
scientists and learners (Duschl & Hamilton, 1998).
The transformation from a focus on individuals to a
focus on participants of the learning community
requires both conceptual change (Strike & Posner,
1985) and a fundamental change in the educational
environments established for learning the desired
content (Dori & Belcher, 2005). Educational
technology plays an important role in supporting a
social, active, constructive learning environment
(Jonassen, Carr & Yueh, 1998). The capabilities of
new technologies and the methods that use them are
viewed as being better able to attract student
attention to the lectured topic (Beichner, et al., 1999)
and to facilitate students’ learning (Kozma, 1994).
Educational technology has the potential not only to
improve student performance, but also to prepare
students to be productive, employable citizens
(Dusick, 1998). Hake (1998) found that students
engaged in active learning significantly improved
their performance in undergraduate physics.
Nonetheless, some researchers have cautioned that
facilitating students to learn in an interactive, active
environment is a challenging task. For instance, the
instructor must acquire classroom management skills
to facilitate the process of the activities (Maclsaac &
Falconer, 2002), be able to identify an appropriate
extent and timing of intervention during discussion
(Bell & Gilbert, 1996), and be perceptive of the
content and context of students’ responses and
reactions to questions raised (Roth, 1996).
2.1 Gender Gap
The TIMSS report showed that the percentage of
high-achieving boys was significantly higher than
that of the counterpart girls on average science
achievement across countries (Martin, et al., 1999).
Science educators have been engaged in promoting
the participation rate of female students and
achieving a more equitable gender balance in the
past 30 years (Hodgson, 2000). It was found that
gender difference can be attributed to sociological
influences, such as culture (e.g. the creation of
gender identity and gender equity), attitude (e.g.
intrinsic interest and reading attitudes) and choice
(e.g. personal preference), and biological influences,
such as neurology (e.g. the structural shape of the
brain), chemistry (e.g. the levels and use of
hormones in genders) and imagery (e.g. the
availability of imaginal mediators) (Kitchenham,
2002). Kitchenham (2002) contended that
achievement differences between genders could be
reduced through sound pedagogical methods; for
example, alternating between group discussion and
structured instruction to accommodate different
learning needs for both genders, promoting
mixed-gender team teaching, and encouraging the
use of technology in the classroom for both genders.
Similarly, Hodgson (2000) urged that science-related
gender stereotyping issues be resolved through
curriculum and pedagogy. Lorenzo, et al. (2006)
found that interactive engagement instruction, such
as encouraging in-class peer interaction, effectively
eliminated the gender gap in the conceptual
understanding of an introductory calculus-based
physics course. Beichner, et al. (1999) reported that
THE IMPACT OF TECHNOLOGY-ENABLED ACTIVE LEARNING ON STUDENT PERFORMANCE, GENDER
AND ACHIEVEMENT LEVEL
421

female students were found to be as engaged in the
class discussion and group work as male students in
a highly collaborative, technology-rich,
activity-based learning environment. They
emphasized that socialization among peers played a
critical role in the success of students in the physics
component of the curriculum.
2.2 Low - and High - Achieving Gap
According to the National Assessment of
Educational Progress (NAEP) report reported by
Freeland (1983), high achievers’ skills in science and
math were declining, while low achievers’ skills
were improving. Lau and Chan (2001) found that the
gap among underachievers and high achievers could
be attributed to motivational variables, such as
having a low academic self-concept, placing low
attainment value on learning, and deficiency in using
learning strategies. Martin (1985) revealed that
among low achievers, negative motivation, such as
anxiety and frustration, was almost twice that of
high achievers. Bailey (1971) stated that
self-estimates and desired levels of college ability
were positively associated with the level of students’
actual achievement. Dori and Belcher (2005)
reported that although students studying in a
technology-rich, active learning environment
improved their performance significantly, the net
learning gain of the low-scoring group was the
highest compared with intermediate- and
high-scoring groups. Likewise, Lorenzo, Crouch,
and Mazur (2006) revealed that interactive
engagement in physics courses appeared to have
more effectively reduced the percentage of
low-achieving students, particularly female students,
than that of high-achieving students. In other words,
appropriate instructional approaches could
significantly improve the performance of low-
achieving students, especially female students.
However, She (1998) found that most female
students in Taiwan, particularly elementary and
middle-school students, were incapable of picturing
themselves pursuing a science-related profession due
to worrying about being labeled as too competent
when compared with their counterpart male students.
Therefore, whether the TEAL setting established at
CCU can assist female students and low-achieving
students to accomplish learning outcomes that are as
significant as those demonstrated in western
countries is thus of interest.
3 METHOD
3.1 Research Context
A quasi-experimental research was designed to
conduct the study. There were four classes of
students studying the GP course at CCU in the first
semester of 2008. Three of the classes studied the
course in the TEAL studio and were regarded as the
experimental group, whereas the class studying in
the traditional classroom was seen as the control
group.
3.2 Data Collection and Analysis
Three sources of data were collected, consisting of:
1. Pre-test: All students studying the GP
course were scheduled to take the pre-test at the
beginning of the semester. Force Concept Inventory
(FCI), developed by Hestenes, Wells, and
Swackhamer (1992), was used to assess the
students’ understanding of fundamental physics
concepts in mechanics. The FCI consists of 30
multiple-choice questions.
2. Post-test: The students were scheduled to
take the post-test (the same test content as the
pre-test) at the end of the semester.
3. Survey: A self-report survey was
administered at the end of the semester to gather
TEAL students’ learning experiences. A 5-point
Likert scale ranging from 5 (strongly agree) to 1
(strongly disagree) was used to collect the survey
data.
4 PRELIMINARY RESULTS
4.1 The Test Results
There were 410 students registered in the four GP
classes, including 281 experimental students and 129
control students. A total of 342 students completed
both the pre- and post-tests. Table 1 displays the test
results of the four classes, including the mean scores
of the pre- and post-test, and learning gains, where
learning gain is defined by Hake (1998) as:
〈g〉= (post test - pre test) / (100 – pretest))%
Table 1 indicates that the learning gain of the
experimental group (14.51%) is significantly higher
than that of the control group (2.19%).
Table 2 lists the learning outcomes broken down
by gender. Although the mean scores of the pre-test
CSEDU 2010 - 2nd International Conference on Computer Supported Education
422

Table 1: The tests results by group.
Group
Experimental (N=252) Control (N=90)
Pre-/Post Test Pre Post Pre Post
Mean 75.26 78.85 73 73.59
Std. Dev. 15.29 13.95 15.55 13.00
t-test p-value < .001
*
.584
<g> 14.51% 2.19%
*
P-value of the paired t-test < .001
Table 2: The test results by gender.
Semester
1
st
semester of 2008
Gender
Male Female
Group
Experiment
(N=209)
Control
(N=66)
Experiment
(N=43)
Control
(N=24)
Pre-/Post Test Pre Post Pre Post Pre Post Pre Post
Mean
*
75.50 79.28 74.95 75.35 74.11 76.74 67.64 68.75
Std. Dev. 15.88 14.12 12.99 12.98 12.10 13.02 13.92 12.03
t-test p-value < .001
*
.086 .744 .626
<g> 15.43% 1.60% 10.16% 3.43%
*
P-value of the paired t-test < .001
Table 3: The tests results by achievement.
Semester
1
st
semester of 2008
Achievement
High Intermediate Low
Group
Experiment
(N=122)
Control
(N=38)
Experiment
(N=99)
Control
(N=36)
Experiment
(N=31)
Control
(N=16)
Tests Pre Post Pre Post Pre Post Pre Post Pre Post Pre Post
Mean
*
87.35 86.86 85.53 82.28 69.73 74.65 69.17 70.28 45.38 60.75 51.88 60.42
Std. Dev. 26.43 60.66 32.00 78.14 34.39 146.67 37.06 131.03 119.02 246.45 16.25 102.04
P-Value .258 .007
*
< .001
***
.27 < .001
***
.002
**
<g> -3.87% -22.46% 16.25% 3.60% 28.14% 17.75%
*
P-value of the paired t-test < .05
**
P-value of the paired t-test < .005
***
P-value of the paired t-test < .001
of the experimental male group and the control male
group were about the same (75.50 vs. 74.95), the
experimental male group, performed better than its
counterpart control male group in the post-test
(79.28 vs. 75.35, p-value <.001 and .086
respectively). The learning gains achieved by the
two male groups were 15.43% and 1.6% respectively.
Compared with their pre-test scores (74.11 and 67.64
respectively), the experimental and control female
groups did not significantly improve their learning
outcomes in the post-test (76.74 and 68.75
respectively). However, the learning gain achieved
by the experimental female group (10.16%) was
much higher than that of the control female group
(3.43%).
The students’ learning outcomes were also
analyzed based on their achievement levels. Three
levels of achievement were categorized, using the
students’ pre-test scores: (1) high (80 or higher), (2)
intermediate (60-79), and (3) low (below 60). The
results are shown in Table 3. As indicated in Table 3,
both the high-achieving experimental and control
groups had negative improvement in their post-test,
-3.87% and -22.46% respectively. The intermediate-
and low-achieving experimental groups and the
low-achieving control group, however, improved
their post-test significantly (16.25%, 28.14%, and
17.75% respectively), with p-value < .001, <.001,
THE IMPACT OF TECHNOLOGY-ENABLED ACTIVE LEARNING ON STUDENT PERFORMANCE, GENDER
AND ACHIEVEMENT LEVEL
423

and <.005, respectively. In accordance with the
findings reported by Dori and Belcher (2005), this
study also found that the students in the
low-achieving group benefited most from engaging
in TEAL. Dori and Belcher (2005) stated that it was
probably due to more room for the lower achieving
students to make improvement.
4.2 The Survey Results
A total of 239 surveys were collected from the three
TEAL classes, representing an 85% (239/281) return
rate. The survey information reveals that the TEAL
students were inclined to agree that there was more
interaction taking place between the instructor and
students in the TEAL studio (mean=3.74), when
compared with traditional classroom learning
experiences. They also tended to agree that there
was more interaction taking place among peers in
the TEAL course (mean=4.02). Although they felt
more nervous studying the GP course in the TEAL
studio than in the traditional classroom (mean=3.61),
they, overall, preferred to study in the TEAL studio
than in a traditional classroom.
The three methods most frequently used by the
students to resolve problems encountered were: 1)
discussing the problems with peers, 2) self-studying,
and 3) reviewing the lecture clips posted online. The
top three items (among 14 items
1
) that the students
considered most helpful for their study are: (1) self
study, (2) the instructor’s
instructional style and
teaching skills, and (3) the teaching assistants’
in-class assistance. Suggestions made by the
students for improving future TEAL implementation
revealed that quite a few students were impressed
with the high-technology equipped studio. However,
a number of students stated that the instructional
pace was too fast, formula derivative occurred too
often, and more demonstrations and clearer
explanations of the lecture content were preferred.
5 CONCLUSIONS
Consistent with the findings disclosed in 2007, this
study found that the experimental students not only
1
The 14 items include (1) the instructor’s instructional style and
teaching skills, (2) the use of PRS, (3) attendance requirement (4)
the instructor’s after-class assistance, (5) teaching assistants’
in-class help, (6) recitation sessions taught by teaching assistants,
(7) peers’ in-class discussion, (8) classmates’ after-class assistance,
(9) self study, (10) video clips posted on the web, (11) 3D
simulation, (12) lab activities, (13) frequent tests, and (14)
homework.
showed positive attitudes toward the technology
enabled learning environment, but also achieved
higher learning outcomes than the control students.
This study also disclosed that the learning gain
achieved by the low-achieving TEAL students
(28.14%) was the highest among the groups.
Helping low-achieving students learn more
effectively, and narrowing the learning gap, has been
a common goal for many educators. The results
reported here appear encouraging in this regard.
In addition, the survey results reveal that when
students encountered problems they would discuss
them with their peers in class, which indicates that
the peer-discussion feature of TEAL did provide a
venue for students to solve their problems. However,
it is also noticed that the top three items that the
students considered most helpful to their study were
self-study, the instructor’s instructional skills, and
the teaching assistants’ in-class assistance. In other
words, the students do not yet seem to have fully
benefited from some of the TEAL features, such as
the use of personal response system and simulation
demonstrations. In sum, integrating technological
features into instructional design remains the most
challenging task to the instructors. Nevertheless, it is
hoped that the findings reported here could provide
useful insights and cautions to educators who are
interested in strengthening their teaching through the
use of technology reinforced learning approaches.
NOTE
The research was sponsored by the National Science
Council, NSC 97-2511-S-271-001.
REFERENCES
Bailey, R.C., 1971. Self-concept differences in low and
high achieving students. Journal of Clinical
Psychology, 27(2), 188-191.
Beichner, R., Bernold, L., Burniston, E., Dail, P., Felder,
R., Gastineau, J., Gjersten, M., & Risley, J., 1999.
Case study of the physics components of an integrated
curriculum. American Journal of Physics, 67,
S16-S24.
Beatty, I. D., & Gerace, W.J. (2009). Technology-
enhanced formative assessment: A research-based
pedagogy for teaching science with classroom
response technology. Journal of Science Education
and Technology, 18,146-162.
Beatty, I.D., Leonard, W.J., Gerace, W.J., & Dufresne, R.J.
(2006). Question driven instruction: Teaching science
(well) with an audience response system. In D. A.
CSEDU 2010 - 2nd International Conference on Computer Supported Education
424

Banks (Eds.), Audience Response Systems in Higher
Education: Applications and Cases, (pp. 96-115).
Hershey, PA: Idea Group.
Dori, Y. J. & Belcher, J., 2005. How does technology-
enabled active learning affect undergraduate students’
understanding of electromagnetism concepts? The
Journal of the Learning Sciences, 14(2), 243-279.
Duschl, R. A. & Hamilton, R. J., 1998. Conceptual change
in science and in the learning of science. In Fraser, B.
J. & Tobin, K.J. (Eds.), International Handbook of
Science Education (pp. 1047-1065). Dordrecht, The
Netherlands: Kluwer Academic.
Dusick, D. M., 1998. The learning effectiveness of
educational technology: What does that really mean?
Educational Technology Review, 10, 10-12.
Freeland, S. (1983). NAEP report. Education Digest, 48(8),
65.
Hake, R.R., 1998. Interactive-engagement versus
traditional methods: A six-thousand- students-survey
of mechanics test data for introductory physics course.
American Journal of Physics, 66, 67-74.
Hestenes, D. & Halloun, I. (1995). Interpreting the Force
Concept Inventory. The Physics Teacher, 33, 502-506.
Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force
Concept Inventory. The Physics Teacher, 30, 141-158.
Hodgson, B. (2000). Women in science – or are they?
Physics Education, 35(6), 451-53.
Jonassen, D., Carr, C., & Yueh, H.P., 1998. Computers as
mindtools for engaging learners in critical thinking.
TechTrends, 43(2), 24-32.
Kitchenham, A., 2002. Vive la difference: Gender,
motivation and achievement. School Libraries in
Canada, 22(2), 34-37.
Kozma, R. B., 1994. Will media influence learning?
Reframing the debate. Educational Technology
Research and Development, 42(2), 7-19.
Lau, K. L., Chan, D.W., 2001. Motivational characteristics
of under-achievers in Hong Kong. Educational
Psychology: An International Journal of Experimental
Educational Psychology, 21(4), 417-30.
Lorenzo, M., Crouch, C.H., & Mazur, E., 2006. Reducing
the gender gap in the physics classroom. American
Journal of Physics, 74(2), 118-122.
Martin, D. M., 1985. Relationship of motivational
differences of male and female community college
students to academic achievement (Abstract, Masters
Thesis, University of Missouri- Kansas City).
(ED262856).
Martin, M. O, Mullis, I.V., Gonzales, E.J., O’Connor,
K.M., Chrostowski, S.J., Gregory, K.D. et al., 1999.
Science Benchmarking Report: TIMSS 1999 – Eight
grade, achievement for U.S. states and districts in an
international context. IEA The Third International
Mathematics and Science Study, MA: Boston College.
Roth, W- M. (1996). Teacher Questioning in an
open-inquiry learning environment: Interactions of
context, content, and student responses. Journal of
Research in Science Teaching, 33(7), 709-736.
She, H. C., 1998. Gender and grade level differences in
Taiwan students’ stereotypes of science and scientists.
Research in Science and Technological Education,
16(2), 125-135.
Strike, K. A. & Posner, G. J., 1985. A conceptual change
views of learning and understanding. In L.H.T. West &
A.L. Pines (Eds.), Cognitive Structure and Conceptual
Change, New York: Academic Press.
THE IMPACT OF TECHNOLOGY-ENABLED ACTIVE LEARNING ON STUDENT PERFORMANCE, GENDER
AND ACHIEVEMENT LEVEL
425