Blended Learning Training for Mentors of STEM Team

Competitions

Sharon M. Locke

, Susan L. Thomas

, Stephen Marlette

, Georgia L. Bracey

, Gary Mayer

Jerry B. Weinberg

, Janet K. Holt

and Bradford R. White

Center for STEM Research, Education, and Outreach, Southern Illinois University Edwardsville,

Campus Box 2224, Edwardsville, Illinois, U.S.A.

Office of Academic Affairs, Truman State University, 100 East Normal Avenue, Kirksville, Missouri, U.S.A.

Department of Curriculum and Instruction, Southern Illinois University Edwardsville, Edwardsville, Illinois, U.S.A.

Department of Computer Science, Southern Illinois University Edwardsville, Edwardsville, Illinois, U.S.A.

Illinois Education Research Council, Southern Illinois University Edwardsville, Edwardsville, Illinois, U.S.A.

Keywords: Blended Learning, Mentoring, Robotics, K-12 Education, Self-Efficacy, STEM.

Abstract: This paper describes the findings of a research study of a blended-learning approach to train mentors of

teams in the Botball® Educational Robotics Program. Botball is an international team-based robotics

competition for secondary students designed to build skills in computer programming, robotics, teamwork,

and problem solving. For this study, we recruited new teams comprising 8-10 middle school students per

team and a mentor. Teams were randomly assigned to one of three treatment groups or a control group.

Mentors of teams in the experimental groups received training in one of three types of mentor practices: best

practices, mentoring for self-efficacy, or a combination of best practices and self-efficacy. The training

format consisted of web-based self-paced tutorials, a face-to-face workshop, and webinars. Dependent

variables were student post-test scores on three assessments: Efficacy for Science-Related Jobs, STEM

Achievement-Related Choices, and STEM Self-Efficacy. A priori statistical analyses showed no difference

between the groups; however, post hoc analyses showed that the use of self-efficacy techniques was

positively related to the three dependent measures. Post-competition surveys of mentor practices indicated

that students in the treatment groups did not appear to receive distinctly different treatments, revealing some

of the potential challenges of the blended learning approach for professional development of teacher-

mentors.

1 INTRODUCTION

Science and engineering competitions offer the

potential to generate interest in STEM (science,

technology, engineering, and mathematics) among

K-12 students (Dabney et al., 2012), identify and

develop STEM talent and influence choice of

postsecondary major (Sahin, 2013), and foster

participation by historically underrepresented groups

(Alvarez et al., 2010). Existing competitions vary in

scope, format, and learning goals; some of the most

popular range from academic Olympiads (e.g.,

Science Olympiad), to math contests (e.g.,

MATHCOUNTS), to science fairs (e.g., Intel

International Science and Engineering Fair, Google

Science Fair), to robotics competitions (e.g.,

FIRST

®, Botball®, VEX®). Each of these activities

involves a mentor, often a school teacher, who

guides a student or group of students to successfully

complete a project or intense program of study in

preparation for a competition day. In addition to

aiming for high academic performance, mentors who

work with groups of students must manage team

dynamics so as to ensure a positive experience for

all. “Group mentoring” for academic competitions

has not been well studied, and very few advice

guides are available for mentors. Given the ongoing

interest in academic competitions, further research is

needed to determine which approaches to group

mentoring lead to positive student outcomes.

This research study examined the components of

effective mentor training for an out-of-school

robotics program for middle-school students. The

training used a blended learning approach that

included a one-day face-to-face workshop, 3-5 Web-

based tutorials, and interactive webinars. Blended

331

Locke S., Thomas S., Marlette S., Bracey G., Mayer G., Weinberg J., Holt J. and White B..

Blended Learning Training for Mentors of STEM Team Competitions.

DOI: 10.5220/0005487003310337

In Proceedings of the 7th International Conference on Computer Supported Education (CSEDU-2015), pages 331-337

ISBN: 978-989-758-108-3

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

learning was chosen for its flexibility in delivering

content to mentors who were not co-located (e.g.,

Garrison and Kanuka, 2004). A total of four STEM

educators served as training facilitators: two

facilitators co-led the face-to-face workshop for

groups of mentors at two sites in the U.S., and three

facilitators shared responsibilities for leading the

webinars. The mentor training took place over a 10-

week period, beginning 6 weeks before the robotics

competition season started and ending 3 weeks prior

to the culminating competition day. This position

paper (work in progress) describes the benefits and

challenges of using blended learning to train team

mentors and reports initial research findings on

training outcomes.

2 THEORETICAL FRAMEWORK

The number of published studies of blended learning

has continued to increase since 2000 (Bliuc et al.,

2007). While several meanings for the term

“blended learning” have been proposed, a widely

accepted definition is that blended learning is a

combining of face-to-face instruction with web-

based instruction where learners are not co-located

(Garrison and Kanuka, 2004; Owston et al., 2008).

Furthermore, these researchers have argued that

effective blended learning is not simply the addition

of a technology-mediated component to an existing

traditional course, but rather an intentional

integration of face-to-face and web-based

components that capitalizes on the strengths of each

format. The online experience can add value by

allowing time for student reflection, extended

discussions, and personalization of content

(Ausburn, 2004; Bonk and Graham, 2006;

Laurillard, 2014). Researchers have reported that

compared to traditional courses, the blended learning

format promotes faculty-student interaction (Owston

et al., 2006), increases student satisfaction (Dziuban

et al., 2006), and increases student engagement

(Ziegler et al., 2006).

Much of the published literature on blended

learning focuses on university courses; however,

blended learning has also been applied to K-12 in-

service teacher professional development (e.g.,

Berger et al., 2008; Owston et al., 2008; Matzat,

2013; Eshtehardi, 2014; Ho et al., 2014). A primary

challenge of any teacher professional development is

to design a program that will facilitate lasting

change in teacher practice (Loucks-Horsley et al.,

2009). Using blended learning, facilitators can offer

support for teachers between face-to-face meetings

through structured online activities that enable

deeper reflection on their practice (Berger et al.,

2008). Blended learning is recognized as a

promising approach for working with in-service

teachers and other professionals because it can

extend the duration of the professional development,

allowing more time for teachers to try out new

practices in the classroom and receive immediate

feedback from the online community (Owston et al.,

2008b). Owston et al. have called for additional

studies that examine ways to better sustain teacher

participation in the online component and determine

the impact of blended programs on student learning.

3 TRAINING DESCRIPTION

The goal of this project was to develop a training

(professional development) program for robotics

team mentors that would give them knowledge and

skills necessary to effectively guide students through

the process of participating in a competition.

Weinberg et al. (2007) found that students

participating in the Botball robotics program who

have positive perceptions regarding their mentor's

effectiveness tend to have increased positive

perceptions towards STEM careers. However, the

components of effective mentoring are not well

understood, including the role that mentors might

play in increasing student self-efficacy for STEM.

Through an experimental study, the project sought to

determine which mentoring practices would be most

effective in increasing students’ self-efficacy in

STEM as measured by three assessments.

Mentors were teachers or administrators in the

students’ school, but they were not necessarily the

students’ STEM classroom teacher. New teams of 8-

10 students were recruited from 45 schools that had

never participated in the Botball program. Mentors

had a range of years of classroom experience and

most had no prior experience with robotics teams.

The study assigned teams to a control group or one

of three experimental groups. One group of mentors

received mentor training using best practices, a

second group of mentors received training in

mentoring for self-efficacy, and a third group

received training that combined best practices and

self-efficacy. Mentors were then expected to apply

these mentoring practices with their robotics teams

as they prepared for the Botball tournament. Surveys

administered to students before and after the

competition measured changes in students’ self-

efficacy and career-related choices.

CSEDU2015-7thInternationalConferenceonComputerSupportedEducation

332

The professional development for the best

practices group focused on team-building theory

(Tuckman, 1965) and practical issues such as team

member roles, goal-setting, creating cohesion,

managing interpersonal dynamics, and celebrating

success. The mentoring for self-efficacy professional

development focused on the theory of self-efficacy

and its four components: mastery experience,

vicarious experience, social persuasion, and

physiological reaction (Bandura, 1977, 1997).

Mentors in the third experimental group received

training that combined best practices and self-

efficacy. Mentors in the self-efficacy and combined

groups were given practical suggestions on how to

build students’ sense of mastery of technical and

communication tasks, provide students opportunities

to observe and learn from role models, offer targeted

feedback, and teach students strategies for managing

physiological responses to stress. The self-efficacy

techniques were drawn from the literature on STEM

and self-efficacy (Rittmayer and Beier, 2008).

The training program was well-suited to blended

learning because it required that mentors a) learn

theories with which they might be unfamiliar, b)

reflect on their own previous mentoring experiences,

c) practice team-based skills, and d) monitor and

discuss their mentoring practice throughout the

competition season. Mentors came from two

different geographic regions of the U.S., and they

were all based at different schools. The mentors’

widely varying and busy schedules meant that it

would have been difficult to schedule multiple face-

to-face workshops during a 10-week time span. Self-

paced Web-based modules gave mentors flexibility

to learn new content at a time and place convenient

for them. The tutorial modules also facilitated

personalization of learning by prompting mentors to

connect their learning to prior experience through

responses to open-ended questions. The

complementary face-to-face session promoted group

cohesion, which is critical to the development of a

community of practice, whether in-person or online

(Garrison and Arbaugh, 2007).

The training proceeded in a logical sequence that

began with an introduction to mentoring theory and

practice through the Web-based modules and

webinars. Prior to the face-to-face workshop,

mentors attended two facilitated webinars where

they were introduced to other mentors and provided

time to share and discuss their open-ended responses

to the tutorial questions. During the fourth week of

the training, mentors attended an on-site workshop

that included small-group activities and role-playing

to reinforce targeted concepts. Each mentor also

developed a written mentoring plan to help them

structure their team’s activities through the

competition season. Following the face-to-face

workshop, the mentors attended two webinars at 6

weeks prior and 3 weeks prior to the Botball

tournament. These webinars reinforced mentoring

concepts and addressed questions or problems

concerning the mentoring implementation. Total

mentor training contact hours were 17 hours for the

best practices and self-efficacy groups and 20 hours

for the combined group. Mentors in the control

group attended a technical workshop led by the

Botball organization, but did not receive any

additional training. Mentors in the experimental

groups also attended the technical workshop.

4 RESULTS

To examine the impact of the mentor training,

students in the four treatment groups (control, best

practices, self-efficacy, and combined) completed

three assessments pre- and post-competition:

Efficacy for Science-Related Jobs, STEM

Achievement-Related Choices, and STEM Self-

Efficacy. In addition, students evaluated their

mentor’s performance and reported on their mentor

practices, and mentors self-reported on their

mentoring practices. The reports on mentor practices

were intended as a check on implementation, since

the project facilitators were not present during team

practice sessions. Student reports of mentor practices

were considered alongside the mentor self-reports.

336 student and 41 mentors completed pre- and

post-surveys. Students who reported attending less

than 25% of their team’s meetings were excluded

from the statistical analysis because the treatment

was considered insufficient. It was expected that

mentor training that included self-efficacy would

result in higher student STEM self-efficacy and

STEM achievement-related choices compared to

best practices and the control. If that were the case,

then the study would provide evidence of the

benefits of including practical strategies that support

students’ self-efficacy in mentoring for STEM

competitions.

The proposed statistical approach was a mixed

analysis of variance (ANOVA) to test the

hypothesis. However, the implementation check

(student reports and mentor self-reports of practices)

suggested that students in the different treatment

groups did not receive distinctly different treatments.

With this confound, we were unable to explicitly

BlendedLearningTrainingforMentorsofSTEMTeamCompetitions

333

analyse the effect of mentor training on student

outcomes.

To further investigate the possible relationship

between effective mentoring practices and student

outcomes, the statistical team next conducted a post-

hoc analysis of covariance (ANCOVA) using the

student reports of mentoring practices. The student-

reported practices were categorized by student-

perceived amount of mentoring strategies applied

(low, mid-low, mid-high, and high groups). The

overall group effect of student-reported mentor

activities was significantly related to STEM Self-

Efficacy and to STEM Achievement-Related

Choices. The overall group effect was not

significantly related to Efficacy for Science-Related

Jobs. However, not using mentoring practices (low

group) had significantly lower outcomes than the

mid-low, mid-high, and high groups combined.

Thus, if only student-reported mentor practice

groups are considered, there was a significant

relationship for all three dependent variables.

Mentor practices do have a significant impact on

student STEM self-efficacy and achievement-related

choices.

In the next section we posit some of the possible

reasons for the lack of distinction between treatment

groups and provide recommendations for refining

the training using our qualitative evidence and

findings reported in the blended learning and teacher

professional development research bases.

5 DISCUSSION

The training program included many of the design

elements important for successful blended

professional development, including attention to

building a community of practice (Wenger et al.,

2002; Garrison and Vaughan, 2008); inclusion of

activities before, during, after, and in preparation for

face-to-face workshops (Berger et al., 2008;

Garrison and Arbaugh, 2007); flexibility (Owston et

al., 2008); and opportunity for extended discussion

(Bonk and Graham, 2006). During implementation,

however, mentor practices did not significantly

differ across treatment groups. Despite receiving

training in specific strategies targeted to best

practices and/or self-efficacy, similar mentor

practices were reported for all groups, including the

control. We hypothesize that a combination of

factors influenced these results, and that further

refinement of the training program could improve

the impact of training on student outcomes.

Important factors appear to be time constraints,

balancing group mentoring with technical tasks,

malleability of teacher practices, cohesiveness of the

community of practice, and the need for additional

structure in online activities.

Time and Scheduling: Webinar discussions

revealed that mentors faced significant challenges in

finding meeting times that would work for all team

members, in resolving conflicts with other student

activities such as music and athletics, and in moving

the team through the robotics design challenges

according to the Botball schedule. The relative short

competition season (7 weeks) meant that some

mentors struggled to simply keep the team on

schedule, and this may have prevented them from

implementing specific strategies discussed during

the mentor training sessions. Inconsistent student

attendance exacerbated some mentors’ efforts to

apply what they had learned.

Balancing Mentoring and Technical Guidance:

For mentors with limited exposure to robotics, and

those outside the fields of STEM, the need to help

students with the technical aspects of building a

robot was sometimes overwhelming and superseded

the mentoring implementation goals. One mentor, an

early-career English teacher, expressed frustration

with her inexperience with computer programming.

Although the Botball program is designed to

carefully scaffold the process of designing and

building a robot, mentors’ negative emotions might

interfere with their implementation of group

mentoring techniques. Allotting more time for

mentors to prepare for the technical aspects of the

program might build their own confidence levels,

enabling them to refocus attention to mentoring

strategies.

Malleability of Teacher Practices: When faced

with time limitations, teachers would be more likely

to revert to coaching and guiding the team in ways

that are familiar to them from classroom or other

experiences. For example, mentoring for self-

efficacy should include attention to all four sources

of self-efficacy (mastery, vicarious experiences,

social persuasion, and physiological response), but

some mentors were unable to schedule visits by role

models who could interact with students, a strategy

associated with increased self-efficacy.

Alternatively, experienced teachers might

unconsciously or even consciously resist changing

their practice. In one case in our study, a mentor

appeared to be resistant to trying new approaches,

stating that with “many years of teaching

experience” they were confident they knew the best

ways to work with their team.

CSEDU2015-7thInternationalConferenceonComputerSupportedEducation

334

Building the Community of Practice: Zhao and

Rop (2001) emphasized the importance of

meaningful discourse in virtual communities, so that

teachers participating in professional development

feel they have a reason to stay connected to each

other. Using this principle, the interactive webinars

in this project provided opportunities for mentors to

reflect on their team’s progress, receive feedback on

specific problems, and experience emotional support

when faced with challenges beyond their control.

Although all webinars were scheduled well in

advance, as the program progressed many mentors

had trouble attending during scheduled times and

had to reschedule in smaller groups or in one-on-one

sessions with facilitators. This negatively impacted

the cohesion of the mentors within a treatment group

and reduced opportunities for reflective group

discourse. Owston et al. (2008) also found a decline

in participation rates for online biweekly reflective

tasks in a blended learning professional development

program for science and math teachers.

Structure of Online Activities: Owston et al.

(2008) suggested that shorter online tasks and skilled

online facilitators may help prevent weak

participation rates in blended learning environments.

For example, they found that teachers were more

motivated to post reflective online journals if they

received helpful feedback from facilitators. While

the mentoring webinars were an opportunity for real-

time feedback, the facilitators’ questioning was

generally open-ended. Our experiences and the

findings of Owston et al. suggest that the mentor

training might be improved by adding opportunities

to post online reflections and receive feedback more

frequently. For example, an online discussion board

would provide space for mentors to post the

implementation plans they developed during the

face-to-face workshop, revisit and revise them

periodically, and receive feedback from an online

facilitator and peers. The more frequent contact

would serve as a reminder to mentors to continually

integrate group mentoring principles into their

team’s activities.

6 SUMMARY AND

CONCLUSIONS

Team-based robotics competitions provide students

with the opportunity to learn technical skills such as

engineering design and computer programming,

while also enhancing problem solving, team work,

and communication—21

Century skills that are

valued by employers. Mentors for youth robotics

teams and other academic competitions may be

school teachers, university faculty and students, or

STEM professionals who have varied experiences

and skill levels in working with youth and guiding

teams. Despite the critical role these mentors play in

creating a positive experience, there are limited

resources available to mentors to help them

effectively guide their teams. To fill this gap, we

developed a training program for group mentoring

that is grounded in theory and empirical evidence of

mentoring best practices and sources of self-

efficacy. The training program was tested in a

robotics educational program, but the mentoring

concepts and strategies are applicable in any

learning environment where an adult mentor is

guiding a group of students through a team project.

The training used a blended learning approach

because mentors were not co-located and because a

long-term project goal is to make the training

available to a wide audience. The program

integrated Web-based tutorial modules on

mentoring, a face-to-face workshop, and interactive

webinars that reinforced the content and provided

time for group reflection and discussion.

The project tested three approaches to mentor

training using four groupings: control, best practices,

self-efficacy, and a combination of best practices

and self-efficacy. No differences were found in

student self-efficacy outcomes among the groups;

however, student and mentor reports of

implementation suggest that mentors in the four

groups did not differ significantly in their approach

to mentoring, despite the training. This finding is not

inconsistent with the teacher professional

development literature, which has noted the

difficulty of designing professional development that

leads to a sustained change in teacher practice

(Borko, 2004; Loucks-Horsley et al., 2009). We

suggest that both external factors (time and

scheduling) and training design features (the need

for additional structured online activities) influenced

the outcomes. Also, given the short duration of the

robotics competition season, it may be that

mentoring training is most effective with mentors

who have had at least one year of experience with

the robotics program, or who are sufficiently

familiar with the technology that they are able to

remain focused on evidence-based mentoring

strategies. In the next phase of this work we plan

interviews with a select group of mentors to

determine ways in which the online portions of the

training could be enhanced.

BlendedLearningTrainingforMentorsofSTEMTeamCompetitions

335

ACKNOWLEDGEMENTS

This material is based upon work supported by the

U.S. National Science Foundation under Grant No.

1139400. Any opinions, findings, and conclusions or

recommendations expressed in this material are

those of the authors and do not necessarily reflect

the views of the National Science Foundation.

REFERENCES

Alvarez, C. A., Edwards, D., and Harris, B., 2010. STEM

specialty programs: A pathway for under-represented

students into STEM fields. NCSSSMST Journal, 16(1),

pp.27-29.

Ausburn, L.J., 2004. Course design elements most valued

by adult learners in a blended online learning

environment: An American perspective. Educational

Media International, 41(4), pp.327-337.

Bandura, A., 1977. Self-efficacy: Toward a unifying

theory of behavioural change. Psychological Review,

84(2), pp.191-215.

Bandura, A., 1997. Self-Efficacy: The Exercise of Control.

Freeman. New York.

Berger, H., Eylon, B.S., and Bagno, E., 2008. Professional

development of physics teachers in an evidence-based

blended learning program. Journal of Science

Education and Technology, 17(4), pp.399-409.

Bliuc, A.-M., Goodyear, P. and Ellis, R.A., 2007.

Research focus and methodological choices in studies

into students’ experiences of blended learning in

higher education. Internet and Higher Education, 10,

pp.231-244.

Bonk, C.J., and Graham, C.R., 2006. The Handbook of

Blended Learning. San Francisco, CA: Pfeiffer.

Borko, H. (2004). Professional development and teacher

learning: Mapping the terrain. Educational

Researcher, 33(8), pp.3-15.

Dabney, K.P., Tai, R.H., Almarode, J.T., Miller-

Friedmann, J.L., Sonnert, G., Sadler, P.M., and Hazari,

Z., 2012. Out-of-school time science activities and

their association with career interest in STEM.

International Journal of Science Education, Part B,

2(1), pp.63-79.

Dziuban, C., Hartman, J., Juge, F., Moskal, P., and Sorg,

S., 2006. Blended learning enters the mainstream. In

C.J. Bonk and C. Graham (Eds.) The Handbook of

Blended Learning: Global Perspectives, Local

Designs, pp.195-206.

Eshtehardi, R., 2014. Pro-ELT; A teacher training blended

approach. Advances in Language and Literary Studies,

5(5), pp.106-110.

Garrison, D.R. and Kanuka, H., 2004. Blended learning:

Uncovering its transformative potential in higher

education. Internet and Higher Education, 7(2), pp.95-

105.

Garrison, D.R. and Arbaugh, J.B., 2007. Researching the

community of inquiry framework: Review, issues, and

future directions. The Internet and Higher Education,

10(3), pp.157-172.

Garrison, D.R. and Vaughan, N.D., 2008. Blended

Learning in Higher Education: Framework,

Principles, and Guidelines. John Wiley and Sons.

Ho, V.T., Nakamori, Y., Ho, T.B., and Lim, C.P., 2014.

Blended learning model on hands-on approach for in-

service secondary school teachers: Combination of E-

learning and face-to-face discussion. Education and

Information Technologies, pp.1-24.

Laurillard, D., 2014. Thinking about blended learning: A

paper for the Thinkers in Residence programme. Royal

Flemish Academy of Belgium for Science and the Arts

Available at http://ethicalforum2013.fuus.be/sites/

default/files/DP_BlendedLearning_Thinking-

about_0.pdf.

Loucks-Horsley, S., Stiles, K.E., Mundry, S., Love, N.,

and Hewson, P.W., 2009. Designing Professional

Development for Teachers of Science and

Mathematics. Corwin Press.

Matzat, U., 2013. Do blended virtual learning

communities enhance teachers' professional

development more than purely virtual ones? A large

scale empirical comparison. Computers and

Education, 60(1), pp.40-51.

Owston, R. D., Garrison, D. R., and Cook, K., 2006.

Blended learning at Canadian universities: Issues and

practices. In C.J. Bonk and C. Graham (Eds.) The

Handbook of Blended Learning: Global Perspectives,

Local Designs, pp.338-350.

Owston, R., Sinclair, M., and Wideman, H., 2008.

Blended learning for professional development: An

evaluation of a program for middle school

mathematics and science teachers. The Teachers

College Record, 110(5), pp.1033-1064.

Owston, R., Wideman, H., Murphy, J., and Lupshenyuk,

D., 2008. Blended teacher professional development:

A synthesis of three program evaluations. The Internet

and Higher Education, 11(3), pp.201-210.

Rittmayer, A.D. and Beier, M.E., 2008. Overview: Self-

efficacy in STEM, SWE-AWE CASEE Overviews,

pp.1-12.

Sahin, A., 2013. STEM clubs and science fair

competitions: Effects on post-secondary matriculation.

Journal of STEM Education: Innovations and

Research, 14(1), pp.7-13.

Tuckman, B.W., 1965. Developmental sequence in small

groups. Psychological Bulletin, 63, pp.384-399.

Weinberg, J.B, Pettibone, J.C., Thomas, S.L., Stephen,

M.L. and Stein, C., `2007. The impact of robot

projects on girls’ attitudes toward science and

engineering. Robotics Science and Systems (RSS)

Workshop on Research in Robots for Education,

Georgia Institute of Technology, Atlanta, June 30,

2007. Available at http://www.roboteducation.org/rss-

2007/

Wenger, E., McDermott, R.A., and Snyder, W., 2002.

Cultivating Communities of Practice: A Guide to

Managing Knowledge. Harvard Business Press.

CSEDU2015-7thInternationalConferenceonComputerSupportedEducation

336

Zhao, Y. and Rop, S., 2001. A critical review of the

literature on electronic networks as reflective

discourse communities for inservice teachers.

Education and Information Technologies, 6(2), pp.81-

94.

Ziegler, M., Paulus, T., and Woodside, M., 2006. Creating

a climate of engagement in a blended learning

environment. Journal of Interactive Learning

Research, 17(3), pp.295-318.

BlendedLearningTrainingforMentorsofSTEMTeamCompetitions

337