The Utilization of the Valid Rubric to Improve High School Students'

Real-world Problem-solving Skills in STEM Education

Andinisa Rahmaniar

Department of Technology, Illinois State University

Keywords: STEM education, rubrics, problem-solving

Abstract: Nowadays, students confront challenges with high-level technology. Thus, they need to learn how they

can appropriately use technology. Students learn every subject, such as science, technology, engineering, and

mathematics at school separately. Then, when they must find a solution that requires to combine two different

subject matters, such as engineering and mathematics, they have difficulties to solve the problem because

they do not know how to combine two subjects as one discipline. If a school provides Science Technology

Engineering and Mathematics (STEM) education in their curriculum, students will learn how to apply their

knowledge to solve the problem. Besides, teachers still use the conventional method to teach students and

separate every subject as one independent subject. To analyze students' problem- solving skills, teachers use

a rubric to assess students' ability in problem-solving. Even though the teachers use a rubric, the rubric is

probably not valid because the rubric cannot analyze students' performance continuously. This research

explains the usage of a valid rubric to improve students' problem -solving skills in STEM education. The

researcher does a comprehensive literature review as a research method by using more than fifteen sources

about STEM education, problem-solving, and a valid rubric of problem- solving skills in the results of this

study indicate that students can improve their real -world problem-

solving skills in STEM education after

they getting feedback based on the proposed rubric.

1 INTRODUCTION

High school students learn every subject, such as

science, technology, engineering, and mathematics at

school separately. When they must find a solution to

a problem by applying two different disciplines, such

as engineering and mathematics, they are asked to

solve the problem by integrating two different

subjects. Winarni et al. (in Widya et al., 2019)

examine that STEM education is an integrated

learning of science, technology, engineering, and

mathematics to improve students’ problem-solving

skill. STEM education has three principles; STEM

education should advance the learning, provide a

logical and authentic connection between and across

the individual STEM discipline and serve as a bridge

to STEM careers (Wilson, 2019). One of the STEM

education principles encourage students to have

problem-solving skills, which is providing a logical

and authentic connection. The goal of learning in the

K-12 curriculum is to gain students' knowledge and

ability to unravel problems by using an integrated

concept. Moreover, some researchers use a rubric to

analyze students' problem-solving skills. Doctor et al.

(2016) said that rubric need a proof of validity

because it should be able to give consistent value

when used by anyone and at any time. Therefore, this

research will explain how to create a valid rubric that

the teacher can use in STEM learning to increase

students' real-world problem-solving skills.

2 METHODOLOGY

This paper uses the database from google scholar and

Milner Library (EBSCO) at Illinois State University

and the target is high school teachers. The researcher

divides the literature into three groups, such as STEM

education, real-world problem-solving skills, and

problem-solving rubrics. The keywords that the

researcher uses are "STEM education," "problem-

solving," "problem-solving rubrics”, and “valid

rubric.” There are five studies about STEM

education, one resource describes the STEM

education principle, and three types of research of

implementation of STEM education, three studies

Rahmaniar, A.

The Utilization of the Valid Rubric to Improve High School Students’ Real-world Problem-solving Skills in STEM Education.

DOI: 10.5220/0010357203790385

In Proceedings of the 2nd International Conference on Applied Economics and Social Science (ICAESS 2020) - Shaping a Better Future Through Sustainable Technology, pages 379-385

ISBN: 978-989-758-517-3

379

explain the problem-solving skills. Eight kinds of

research discuss rubric for assessing problem-solving

skills. Those groups of papers are combined to make

one conclusion of the literature review, which is the

criteria of rubrics that teachers can use for assessing

students' real-world problem- solving skills.

3 BACKGROUND

Williams (2011) reviewed the STEM education

research and produced a literature review about

reasons why STEM education has not been

implemented yet at school. For example, the teachers

already teach all subjects in one class, so integrated

STEM education is not essential. The article revealed

that STEM education developed from a non-

educational rationale. Besides that, elaborating

science, mathematics, and technology appears not to

be taken seriously. The author mentions that

integrating science and mathematics will reduce

technology use rather than engineering. In other

perspectives, STEM education is not successful

because teachers do not understand their content and

learning process design, and students cannot transfer

one subject to other subjects. Therefore, the teacher

should believe that interaction between disciplines

will improve students' learning. Other reasons why

STEM education is needed today is mentioned by

Bell (2016), he examined the need for teachers who

has high qualification professional people in STEM is

increased. The article explains that an integrated

STEM curriculum is needed because students become

aware of real-world relations. The author uses

phenomenography as a methodology in his research,

and it is a method by learning the phenomena. When

students learn STEM education, they should

understand how STEM is interconnected in the

implementation of different STEM disciplines to

solve the problems (Ejiwale et al., 2013).

Furthermore, there are three STEM education

principles described by Wilson (2019); STEM

education should develop learning, provide a logical

and reliable connection between and across the

individual STEM disciplines, and serve as a bridge to

STEM careers. According to those principles, STEM

education should provide a logical and reliable

connection, which means the technology can engage

creative thinking and real-world problem- solving. In

addition, integrated STEM education can guide

students to get knowledge for a suitable solution for

human needs. The researcher recommended some

actions, such as starting the STEM education for

elementary grades, engage parents' perception of

STEM education, implementing project-based or

problem-based learning, and encourage students with

STEM-focus, in and outside of school practical

learning occasions. Those principles can help

teachers to develop the learning process, which

improves students' real- world problem-solving skills.

For instance, English (2016) said that STEM

education should encourage students to use

knowledge and skills for multiple disciplines. The

authors explain STEM integration correlates with

Common Core State Standards for Mathematics and

Next Generation Science Standards. Besides that,

STEM education can be integrated from

multidisciplinary to interdisciplinary. The authors

describe an excellent development of STEM that can

make students get twenty- century skills, such as the

inquiry process, problem-solving, critical thinking,

creativity, and innovation. Therefore, students are

encouraged to create new and productive connections

in more than two disciplines. The researcher proves

that real- world problem-solving skills can be

advanced through STEM education based on its

principles.

Meyrick (2011) described STEM education gain

students' knowledge, make students learn the topics

more in-depth, and increase the skill to solve the

problem so that students will be more creative and

innovative. Zhang (2017) explained that students

could not solve real-life problems routinely, so by

involving a problem-based learning process, students

can study to solve open-ended problems. In addition,

Basu et al. (2015) explained that undertaking the real-

world problem can encourage students' problem-

solving skills and make them an independent and

competent learner. Moreover, Ferreira and Trudel

(2012) explained problem-based learning gives an

impact on problem-solving skills and a sense of

community in the classroom. The authors mentioned

that problem-based learning increases students'

analytical and reasoning skills to develop a solution

to the problem. Euefeuno (2019) described the four

steps to do problem-based learning in STEM

education, such as identify the problem, design,

evaluate, and communicate the solution. It helps

students to follow the learning process.

The problem-solving skills can be measured by

using a rubric of problem- solving. Docktor et al.

(2016) presented an assessment for students'

problem-solving skills in Physics. They said that

physics is a foundation of STEM education to teach

problem-solving. If students want to be an expert on

solving the problem, they must integrate their

knowledge with the problem- solving framework.

Assessing problem- solving skills is needed as a

ICAESS 2020 - The International Conference on Applied Economics and Social Science

380

rubric because it can be used for measuring different

scopes of performance and standards of achievement

for each of the ranges, and it has consistent scoring

rates. The researcher used Minnesota Assessment of

Problem Solving (MAPS), which is applied in the

classroom activities, such as written problem

solutions. There are requirements to make a rubric,

such as easy to use, usable in the authentic situation,

have evidence for its validity, reliability, and utility.

The rubric in their article has five criteria, namely

useful description, physics approach, specific

application of physics, mathematical procedures, and

logical progression. Moreover, Medina, et al. (2013)

described the rubric of problem-solving is designed to

evaluate four areas that is answer selection, answer

prioritization and defense, organization of the

respond, and evidence. The results showed that

students can improve their problem-solving skills

after they get feedback from the teacher based on the

problem-solving rubric.

Docktor et al. (2016) used a valid rubric to assess

students' problem-solving skills, and it is different

from Hull et al. (2013) who studied the development

of problem-solving assessment and rubrics which is

included physical situation and mathematical

description. The article mentions that they continue

the global rubric and modifying it by adding

mathematical procedures to the rubric. Another

possibility is adapting the comprehensive rubric to

build a rubric by engaging the conceptual and formal

mathematical reasoning. Therefore, this article

described that guiding students to blend both

theoretical and mathematical reasoning can improve

students' problem-solving skills. The problem-

solving rubrics can be useful for researchers or

teachers to teach students about the problem- solving

and how their works are assessed. In addition, the

researcher explained the problem-solving strategies,

such as visualize the problem, describe a problem,

plan a solution, execute the plan, and check and

evaluate.

In term of rubric validation, Mustapha, et al.

(2016) mentioned that a rubric should present

flexibility and consistencies among teachers or raters.

Eseryel et al. (2013) designed an article about the

validation of assessment for ill-structured problem

solving in interdisciplinary STEM education on

ninth-grade students. The researcher said that valid

assessment for problem-solving skills could improve

the progress of learning outcomes. They used two

methods for their research, such as problem-solving

rubrics and HIMATT (High Integrated Model

Assessment Technology and Tools). As a result, both

the assessment methods gave significantly difference

to student's learning outcomes after the students got

the treatment. HIMATT provided the indicator of the

quality of ill- structured problem-solving in complex

domains, such as STEM education. This article

prefers using HIMATT because it is more valid than

a problem- solving rubric. Memarian and Mccahan

(2018) explained that making the problem-solving

VALUE rubrics which presents high achieving

descriptor for the indicators through two steps, such

as creating the descriptor and mapping the descriptor

topics. Six faculty members validate the rubrics, and

they made a focus group discussion about revising the

description and indicators. The author found that a

rubric should have incorporate representation on the

problem and solution level, to be more specific to a

particular issue and increasing the level of the

descriptor.

Dahm (2014) created rubrics to analyze students'

outcomes, which has four indicators, and each

indicator has four different levels (Figure 1) in STEM

education. There was a proof that this rubric has

significant consistency for assessing students by

different evaluators. Moreover, Gray et al. (2017)

developed Quantitative Literacy VALUE Rubric

from the Association of American Colleges and

Universities to analyze students' works sample in a

STEM course. This rubric contains six skills, such as

interpretation, representation, calculation,

application, assumptions, and communication which

has four-point Likert scale each skill. Gray et al.

(2017) discussed that there is an issue to assess

students' performance because of the possible rater

bias and the differences score among rater; thus, there

should be a validation for the rubric. The result of

their study is the Quantitative Literacy VALUE

rubrics gives consistency score in analyzing students’

performances.

Figure 1. Sample rubric for the outcome “students will

demonstrate an ability to apply knowledge of mathematics,

science, and engineering (Dahm,2014).”

The Utilization of the Valid Rubric to Improve High School Students’ Real-world Problem-solving Skills in STEM Education

381

Nevertheless, there are threats to validate a rubric,

such as subjective and illogical coding rubrics

(Arffman et al., 2015). The distinction between the

criteria is uncertain but relied on subtle, subjective

difference, illogical, counteractive, and inconsistent

standards make biased grading of students'

performance. Then, Ge et al. (2011) who said that

rubric is needed to investigate the critical thing in the

process of problem-solving skills because students

have different abilities to solve real-life problems. In

the article, the author designed the problem-solving

rubric and validated through nine versions of rubrics

samples scoring and continuously testing. The rubrics

have two or three scoring criteria for two aspects

problem representation and generating the solution.

The scoring was done by two professors in the

same field and two raters from a different area. When

the assessors have different total scoring, which is

more than one point, they discussed it and made final

scoring. Another research from Lertyosbordin et al.

(2019) created a problem-solving rubric for

computing science. The author validated the rubrics

through five professors in computing science and five

raters. The results of grading are discussed and

summarized to be some standards, such as best,

average, and miserable.

4 DISCUSSION

The previous research describes the improvement of

students' problem-solving skills in STEM education,

but they did not explain what kind of rubric they used

to assess students and how to develop and validate the

rubric in different fields. Therefore, this literature

review only discusses the connection between the

utilization of a valid rubric and the problem - solving

skill in STEM education. From this literature review,

the reasons why STEM education is not implemented

yet at schools is because some teachers do not

understand the importance of STEM education and

the needed for professional people with high

qualifications in the STEM fields. Therefore, it is not

only teachers who need to understand STEM

education, but also the school administration and

students because students will gain their knowledge

from integrated subject. Based on the principles of

STEM education, which is STEM education logically

connect every subject, teachers can advance the

learning process in STEM education. Therefore,

teachers can improve students' problem-solving skills

through the learning process, which is developed by

the teacher using appropriate rubrics.

In addition, the literature review explains that

teachers should integrate their methodology of

teaching students in STEM education. The teachers

also should know how to approach students to make

students interested in learning and understand the

STEM field. The methodology can be problem-based

learning. As Ferreira and Trudel use problem- based

learning to teach STEM, they use a rubric to know the

development of students' problem-solving skills. The

result shows significant differences in students who

learn with problem-based learning. Therefore, before

teachers start to teach STEM education, the teacher

should make a rubric to analyze the students'

problem-solving skills. Thus, the teacher will know

the ability of their students and can increase students'

real-world problem- solving skills.

Moreover, the rubric should contain identification

of the problem situation, defining problem,

evaluation alternative plan, implementation of the

solution, communication of the plan. For instance,

STEM education uses the VALUE rubric to assess

students' performance. VALUE rubric has six skills

to be analyzed, which is the same with problem-

solving skills, such as interpretation, representation,

calculation, application, assumptions, and

communication. This rubric is valid because after the

rubric is tested for the validation, the results showed

that the VALUE rubric gives a consistency score for

students.

Furthermore, the rubric should be flexible and

consistent because the raters have a different

perspective, and it will impact the result of students'

performance. According to the literature review, there

are some steps to validate the rubrics, such as teachers

need more than two experts from the same and

different field. After the teacher gets the result of

revision from the experts, they test the rubric to know

the validity and reliability. If the results say that the

rubric is reliable and valid, teachers can use it to

analyze students' problem-solving skills.

Teachers can assess students' performance based

on the component of their rubric. First, teachers could

know how students identify the problem. This

component analyzes students' critical thinking skills.

Second, the creativity of students could be assessed

by how students design the solution for the problem.

Then, students evaluate the solution to ensure that the

solution can be used to solve the problem.

Communication skills can be assessed when

students communicate the solution that they find to

solve the problem. In the end, teachers can be more

precise in analyzing students' performance and can

improve students' problem-solving skills in real-

world issues in STEM education.

ICAESS 2020 - The International Conference on Applied Economics and Social Science

382

5 CONCLUSIONS

Students cannot solve real-life problems correctly

because they do not know how to integrate multiple

disciplines. STEM education can increase students'

problem- solving skills by applying integrated multi-

discipline concepts. This skill could be perceived by

using an assessment that are provided by a rubric.

Therefore, the teacher needs to have a valid rubric to

improve students' real-life problem-solving skills in

STEM education. Thus, the teacher can advance their

learning methodology to enhance students' problem-

solving skills.

A rubric should contain four components, such as

identifying the problem, creating a plan of solution

for solving the problem, evaluating the solution, and

communicating the solution. The steps of rubric

validation are a revision by more than two experts in

the same and different fields and testing the rubric.

Teachers can use the rubric after the rubric is proved

that it is valid and reliable to analyze students' real-

world problem-solving skills in STEM education.

In contrast, there is a lack of information about

how significant the impact of using a valid rubric in

STEM education to improve students' problem-

solving. Therefore, there could be future research to

measure the improvement of students' real- life

problem-solving skills in STEM education. It could

be research about the students' interest in STEM

education based on the results of the assessment by

using a valid rubric. The teacher can analyze whether

students can solve the real-world problem step by step

based on the correct rubric. After that, teachers can

give feedback to the students; then, students will

know their ability and increase their skills in problem-

solving. Without the rubric, the teacher could not

analyze students' problem-solving skills accurately

because there is no indicator of problem-solving

skills.

REFERENCES

Arffman, I. (2015). Threats to validity when using open-

ended items in international achievement studies:

Coding responses to the PISA 2012 problem-solving

test in Finland. Scandinavian Journal of Educational

Research, 60(6), 609–625. doi:

10.1080/00313831.2015.1066429

Basu, S., Kinnebrew, J. S., Shekhar, S., Caglar, F., Rafi, T.

H., Biswas, G., & Gokhale, A. (2015). Collaborative

Problem Solving using a Cloud-based Infrastructure to

Support High School STEM Education. Proceedings of

the ASEE Annual Conference & Exposition, 1–21.

Retrieved from https://search-

ebscohostcom.libproxy.lib.ilstu.edu/login.aspx?direct=

true&db =a9 h&AN=116025167&site=ehost-

live&scope=site

Bell, D. (2016). The reality of STEM education, design, and

technology teachers' perceptions: A Phenomenographic

study. International Journal of Technology & Design

Education, 26(1), 61–79. https://doi-

org.libproxy.lib.ilstu.edu/10.1007/s10798-015-9300-9

Dahm, K. (2014). Combining the Tasks of Grading

Individual Assignments and Assessing Student

Outcomes in Project-Based Courses. Journal of STEM

Education: Innovations & Research, 15(1), 20–31.

Retrieved from https://search-

ebscohostcom.libproxy.lib.ilstu.edu/login.aspx?direct=

true&db=a9h&AN=96381826&site=edslive&scope=si

Docktor, J. L., Dornfeld, J., Frodermann, E., Heller, K.,

Hsu, L., Jackson, K. A., … Yang, J. (2016). Assessing

student written problem solutions: A problem-solving

rubric with application to introductory physics.

Physical Review Physics Education Research, 12(1).

doi: 10.1103/physrevphyseducres.12.010130

Ejiwale, J. A. (2013). Barriers to Successful

Implementation of STEM Education. Journal of

Education and Learning (EduLearn), 7(2), 63. doi:

10.11591/edulearn. v7i2.220

English, L. D. (2016). STEM education K-12: perspectives

on integration. International Journal of STEM

Education, 3(1). DOI: 10.1186/s40594-016-0036-1

Eseryel, D., Ifenthaler, D., & Ge, X. (2013). Validation

study of a method for assessing complex ill-structured

problem solving by using causal representations.

Educational Technology Research and Development,

61(3), 443–463. doi: 10.1007/s11423-013-9297-2

Euefueno, W. D. (2019). Project-/problem-based learning

in STEM: impacts on student learning. Technology &

Engineering Teacher, 78(8), 8–12. Retrieved from

https://search-ebscohost-

com.libproxy.lib.ilstu.edu/login.aspx?direct=true&db=

iih &AN=136066910&site=eds-live&scope=site

Ferreira, M, and Trudel A. R. (2012). The Impact of

Problem-Based Learning (PBL) on Student Attitudes

Toward Science, Problem-Solving Skills, and Sense of

Community in the Classroom. Journal of Classroom

Interaction, 47(1), 23–30. Retrieved from

https://search-

ebscohostcom.libproxy.lib.ilstu.edu/login.aspx?direct=

true&db=e ft&AN=83525495&site=eds-

live&scope=site

Ge, X., Planas, L. G., & Eseryel, D. (2011). Developing

Valid Assessment Methods and Scoring Rubrics to

Measure Ill-structured Problem-Solving Performance.

American Educational Research Association.

Gray, J. S., Brown, M. A., & Connolly, J. P. (2017).

Examining Construct Validity of the Quantitative

Literacy VALUE Rubric in College-Level STEM

Assignments. Research & Practice in Assessment, 12,

20–31. Retrieved from https://search-ebscohost-

com.libproxy.lib.ilstu.edu/login.aspx?direct=true&db=

e ric&AN=EJ1149590&site=eds-live&scope=site

The Utilization of the Valid Rubric to Improve High School Students’ Real-world Problem-solving Skills in STEM Education

383

Hull, Michael M., et al. (2013). Problem-Solving Rubrics

Revisited: Attending to the Blending of Informal

Conceptual and Formal Mathematical Reasoning.

Physical Review Special Topics - Physics Education

Research, vol. 9, no.1,

doi:10.1103/physrevstper.9.010105.

Lertyosbordin, C., Maneewan, S., Yampinij, S., &

Thamwipat, K. (2019). Scoring Rubric of Problem-

Solving on Computing Science Learning. International

Education Studies, 12(8), 26. doi:10.5539/ies.

v12n8p26

Mustapha, A., Samsudin, N. A., Arbaiy, N., Mohamed, R.,

& Hamid, I. R. (2016). Generic Assessment Rubrics for

Computer Programming Courses. The Turkis Online

Journal of Educational Technology, 15(1), 53–68.

Retrieved from www.tojet.net/articles/v15i1/1516.pdf

Medina, M. S., Conway, S. E., Davis-Maxwell, T. S.,

&Webb, R. (2013). The Impact of Problem-Solving

Feedback on Team-Based Learning Case Responses.

American Journal of Pharmaceutical Education, 77(9),

189. doi: 10.5688/ajpe779189

Memarian, B., & Mccahan, S. (2018). Development and

Validation of Descriptors for Universal Problem-

Analysis Rubric. Proceedings of the Canadian

Engineering Education Association (CEEA). doi:

10.24908/pceea. v0i0.9687

Meyrick, K. (2011). How STEM Education Improves

Student Learning. Meridian K-12 School Computer

Technologies,14(1). Retrieved from

https://meridian.ced.ncsu.edu/archive/summer2011/m

ey rick/index.htm

Widya, Rifandi, R., & Rahmi, Y. L. (2019). STEM

education to fulfil the 21st century demand: A literature

review. Journal of Physics: Conference Series, 1317,

012208. doi:10.1088/1742-6596/1317/1/012208

Williams, P. J. (2011). STEM Education: Proceed with

caution. Design & Technology Education, 16(1), 26–

35. Retrieved from

https://search.ebscohost.com/login.aspx?direct=true&

db=vth&AN=83260111&site=eds live&scope=site

Wilson, H.W. (2019). STEM4: The power of collaboration

for change. Technology & Engineering Teacher, 78(6),

1–7. Retrieved from

https://search.ebscohost.com/login.aspx?direct=true&

db=eft&AN=134708005&site=eds-live&scope=site

Zhang, L, et. al. (2017). Can Students Identify the Relevant

Information to Solve a Problem? Journal of Educational

Technology & Society, 20(4), 288–299. Retrieved from

https://search-

ebscohost.com.libproxy.lib.ilstu.edu/login.aspx?direct

=true&db=eft&AN=125829920&sit eeds

live&scope=siteconcer

ICAESS 2020 - The International Conference on Applied Economics and Social Science

384