Engineering Education: Measuring the Relationship between

Knowledge and Confidence to the Student Performance

Noek Sulandari

, Cindrawaty Lesmana

and Cindy Maria Setyana

Department of Civil Engineering, Universitas Kristen Maranatha, Jl. Prof. drg. Surya Sumantri,

M.P.H. No. 65, Bandung, Jawa Barat, Indonesia

Department of Psychology, Universitas Kristen Maranatha, Jl. Prof. drg. Surya Sumantri,

M.P.H. No. 65, Bandung, Jawa Barat, Indonesia

Keywords: Engineering Education, Confident, Learning, Knowledge.

Abstract: The students can perform well in the study as well as in the practices if they learn to solve engineering

problems. However, the gaps between the obtaining knowledge in school and skills in practices were still

large. It is a challenge to find improvement in the education model so that the students can perform well

both in school and future practices. This study aimed at investigating the relationship between knowledge

and confidence to the engineering students’ achievement through confidence-based testing. The survey was

distributed to the students who were taking structural analysis course. The tests were scored and run using

Partial Least Square Structural Equation Modelling to obtain the inter-relationship between all action. The

results showed the confidence did not directly affect the students’ achievement, but the knowledge did. The

highly confidence can make students to gain highly or lower achievement. The highly confidence with the

true knowledge developed potential from the students to get higher achievement. The findings showed

highly achievement can be increased if the trainer can continuously make the students realize their false

knowledge in their highly confidence. The highly confidence with the correct knowledge will impact in how

the engineers behave in solving the engineering’s problems.

https://orcid.org/0000-0002-3716-3140

https://orcid.org/0000-0003-2466-850X

https://orcid.org/0000-0001-9984-0904

1 INTRODUCTION

The current practices of engineering education are

students require solving many engineering problems.

By solving those problems, the students can perform

well in the study as well as in the practices.

However, the gaps between the obtaining knowledge

in school and skills in practices are still large

(Adams & W., 2009; Salehi & Sadighi, 2015). It is a

challenge to find improvement in education model

so that the students can perform well both in school

and future practices.

The learning should be delivered in effective

ways so that the students can gain an understanding

of their knowledge. One of the struggles in

education institution is to ensure the students have

an appropriate level of knowledge acquisition, skills,

and competency to perform in practices. The current

practices of assessment, such as: traditional exam or

quiz, can assess the obtaining knowledge of the

students from taking the lecture and to motivate

students to get good grade. However, the typical

grading system tends to encourage the students to

put some answers in hopes that they will get partial

credit, which does not encourage a deeper

understanding of their missing knowledge (Gardner-

Medwin, 1995). Grading can interrupt understanding

but the assessment is needed to measure the learning

process of the students. Therefore, the assessment

should be developed to become a productive way to

help students succeed.

The most common assessments in engineering

are essay-type written tests and oral examinations,

but these methods are believed too subjective and

unreliable (Salehi & Sadighi, 2015). Multiple

Choice Question (MCQ) is also a common format

Sulandari, N., Lesmana, C. and Setyana, C.

Engineering Education: Measuring the Relationship between Knowledge and Conﬁdence to the Student Performance.

DOI: 10.5220/0010748400003113

In Proceedings of the 1st International Conference on Emerging Issues in Technology, Engineering and Science (ICE-TES 2021), pages 227-232

ISBN: 978-989-758-601-9

227

for assessing knowledge and performance (Ben-

Simon, Budescu, & Nevo, 1997), but this method

sometimes leads to a bias whether the students make

a decision because of the right answer based on their

knowledge or whether they have a lucky guess to the

right answer without having an exact knowledge

(Kurz, 1999; Salehi & Sadighi, 2015). The

confidence-based method has started with

Information Reference Testing (Bruno, 1993) that

introduced two metrics where mastery as a result of

high confidence and knowledgeable. It has also

developed with multiple levels of confidence, such

as: three levels of confidence with a correct or

wrong answer (Gardner-Medwin, 2006; Gardner-

Medwin & Gahan, 2003), five levels of confidence

(Khan, Davies, & Gupta, 2001), and simplification

of the method (Reed & Reed, 2015). Although the

method is not new, it needs to test for the different

field of education such as engineering.

Table 1: Weighting for CBT.

Absent

don’t

know

Partially

Confident

Wrong Right Wrong Right

Knowledge 0 1 1 3 1 4

Confident 0 1 2 1 3 4

The linkage of confidence and knowledge in the

learning process is not new. It The students who

confident in learning new material will retain the

knowledge better (Hunt, 2003). The confident

students can develop their own solutions to any

engineering problems. Students who have high self-

confident to learn and put greater effort to overcome

difficult situations and in contrary students with low-

confident tend to give up hope easily (Busse &

Walter, 2013; Stiggins, 2005). Confident and

knowledge are the two important qualities for

successful engineering education and practices.

Knowledge as a product of reason can remain at rest

if the students have no certain it is right. It is

important that students can recognize how confident

they are in solving problems (Bruno, 1993; Nelson

& Webster, 2015; Reed & Reed, 2015). However, a

confidence that driven by ignorance can lead to

failure because students are being certain of own

abilities without any supporting knowledge.

Students are expected to success in school, practices,

and life so it is a challenge for every education

institution to help the students develop to be

independent to solve the problems with their

knowledge.

This paper presents a confidence testing method

through multiple choice and open-ended questions.

The study examines the correlation between

knowledge and confidence in performing the exam

through the former method confidence-based testing

that modified from Bruno (1993).

METHODS

2.1 Descriptions

This study used a survey questionnaire that

integrated two types of questions: multiple choice

and open-ended questions. A concept of the two

axes of the knowledge and the confidence level that

similar with the concept presented as Information

Reference Testing (Bruno, 1993), Confidence-

Based Assessment (Gardner-Medwin, 2006), and

Confidence-Based Scoring (Reed & Reed, 2015)

was adopted, but Confidence-based testing (CBT)

was adding more condition if the students skip the

class to the assessment of the students’ performance.

From the method, confidence and knowledge are

both variable that believe will affect the students’

performance. As the students give their response on

knowledge and confidence for each question, the

answers lead to the true or false knowledge in the

certain portion of confidence level. CBT was used

for the knowledge from the answering questions and

the selection of “confident” or “partially confident”

or “no confident”. The knowledge was measured by

“right” and “wrong” answer to the questions. The

summary of the weighting answers can be seen in

Table 1. The best scenario was having right answer

with confident. If students had the right answer but

lacking confidence, they cannot get perfect score.

The “no confident” answer was given if the students

mark “I don’t know” because it was assumed that

the students had no knowledge on the material as

well as no confident to try to answer the question. If

the students had wrong answer in partially confident,

the weighting knowledge was still low, but they

assumed slightly confident than simply saying “I

don’t know”. If the students’ absent, they do not

have any knowledge or confident to answer the

questions so zero points are assigned. There three

points that arise in this weighting system, the

students with wrong answer were always having the

lowest score of knowledge, knowing they do not

know but they still try to answer the question were

sort of good efforts in confident, and having right

answer but partially confident was assumed similar

with no confident because the students were not sure

what they had known.

ICE-TES 2021 - International Conference on Emerging Issues in Technology, Engineering, and Science

228

2.2 Participants

The participants of this study included 49 first

semester undergraduate students majoring in civil

engineering. The survey was distributed to the

students who were taking a basic structural analysis

course. To eliminate some bias, the valid data were

taken only for students that took the class for the

first time. The total of the respondents were 64

students, but 15 data were eliminated because too

many data were missing, or the respondents had

taken the same course before. The participants

consisted of 31 males and 18 females. The following

data included a total of 5 problems from 10 quizzes.

The CBT method was integrated in every quiz, but

not on any exams. The exam was the measurement

of the outcomes from the learning process since it

gave the grade for the students passed the course.

The students’ performance is divided into three

measurements before they lead to the final grade

(FG), “pass” or “not pass”. The three measurements

are Mid Exam (ME), Final Exam (FE), and practices

(Pr). There are two quizzes for Knowledge (K) and

respectively confidence (C) in Mid Exam (ME) and

eight quizzes for Knowledge (K) and respectively

Confidence (C) in Final Exam (FE). The practice

value is believed a result from knowledge and

respectively confidence of the ten quizzes.

2.3 Analysis Methodology

After the collected data is coded and scored, the

Partial Least Squares Structural Equation Modeling

(PLS-SEM) is adopted to explain the variables.

Structural equation modeling (SEM) is a statistical

technique that very powerful in its ability to model

latent variables, to consider various forms of

measurement error, and to test entire theories. Partial

least squares (PLS) is a path modeling that fully

developed and widely used in the general system

(Hair, Sarstedt, Ringle, & Mena, 2012; McDonald,

1996).

A number of researchers indicated that PLS is

the most appropriate and powerful tool to avoid

small sample size problems, such as: the

assumptions of multivariate normality and interval

scaled data that cannot be made, or the researcher’s

priority is the prediction of dependent variable

(Birkinshaw, Morrison, & Hulland, 1995; Henseler,

Ringle, & Sinkovics, 2009). The PLS path modeling

avoids small sample size problems and even can be

applied for situations that other statistical methods

cannot, this method can be helpful for the

investigation of the relationship between knowledge,

confidence, and outcomes. From the statistic

correlation, hypothesis is investigated to find the

effect of knowledge, confidence, and outcomes.

PLS algorithm was conducted using SmartPLS

(Ringle, Wende, & Will, 2005) to test reliability and

validity of the research constructs. This test is

demonstrated by an accumulation of evidence that

should demonstrate using content analysis,

correlation coefficients, factor analysis, to make sure

the data is valid. Rule of thumbs for these three

general indicators to evaluate the construct reliability

and validity is ‘greater than 0.50’ for Average

Variables Extracted – AVE (Fornell & Larcker,

1981; Götz, Liehr-Gobbers, & Krafft, 2010), and

‘greater than 0.70’ for both Composite

Table 2: Correlation Matrix for Relationship.

AVE C1 C2 K1 K2 Pr FG FE ME

C1 0.545 0.738

C2 0.661 0.377 0.813

K1 0.637 0.641 0.430 0.798

K2 0.533 0.363 0.784 0.358 0.730

Pr 1.000 0.337 0.515 0.394 0.727 1.000

FG 1.000 0.466 0.561 0.436 0.762 0.873 1.000

FE 1.000 0.446 0.615 0.391 0.741 0.731 0.948 1.000

ME 1.000 0.453 0.344 0.424 0.613 0.807 0.898 0.732 1.000

Note: The diagonal values are

√

𝐴𝑉𝐸, the rest are R

Engineering Education: Measuring the Relationship between Knowledge and Conﬁdence to the Student Performance

229

Reliability – CR (Werts, Linn, & Jöreskog, 1974)

and Cronbach’s alpha (Nunnally, 1978). The

correlations among variables should show

acceptable validity results.

SmartPLS can generate t-statistics for

significance testing of both the inner and outer

model, using a procedure called bootstrapping (Hair,

Sarstedt, Ringle, & Gudergan, 2017). The procedure

includes a large number of subsamples that taken

from the original sample with replacement to give

bootstrap standard errors. In results, the analysis turn

gives approximate the normality of data of t-values

for significance testing of the structural path. PLS

algorithm and bootstrapping was used for testing the

research hypotheses. There are three stages of the

basic PLS Algorithm: (1) iterative estimation of

latent variable scores, (2) estimation of outer

weights/loading and path coefficients, and (3)

Estimation of location parameters. All path

relationships required to indicate t-values higher

than significance level of 1.96.

Figure 1: SmartPLS Model and Bootstrapping Results.

RESULTS AND DISCUSSION

Majority of the factor loading results were above the

most common acceptance level of 0.6. After deleting

the items of both constructs due to a low factor

loading, the results show all acceptable results for

Average Variables Extracted (AVE), Composite

Reliability (CR), and Cronbach’s alpha. Fornell and

Laker method were followed to confirm the

discriminant validity as the square root of AVE in

each latent variable was used to establish

discriminant validity if this value is larger than other

correlation values among the latent variables

(Fornell & Larcker, 1981). This validity test shows

how much variance in the indicators that are able to

explain variance in the construct (Afthanorhan,

2013). Numerical indications for correlation matrix

are shown in Table 2, which the square root of AVE

value is manually calculated and written in bold on

the diagonal of the table while the correlations

between the latent variables are copied from the

“Latent Variable Correlation” between the

respective constructs.

The correlation matrix in Table 2 shows that

after some treatment of several items, the R value is

lower than

√

𝐴𝑉𝐸, which indicates the model has

already acceptable since the discriminant validity is

achieved when a diagonal value bold is higher than

the value in its row and column. The SmartPLS

framework model for path analysis in the first order

analysis is shown in Figure 1. The detailed results

are shown in Table 3 and Table 4.

Table 3: PLS-SEM Results for Relationship.

Path coefficients SE t-value

C1 → Pr

0.002 0.137 0.058

C1 → ME

0.324*** 0.086 3.514

C2 → Pr

-0.225 0.141 1.459

C2 → FE

0.102 0.117 0.814

K1 → Pr

0.198 0.113 1.873

K1 → ME

0.229* 0.104 2.198

K2 → Pr

0.839*** 0.105 7.600

K2 → FE

0.662*** 0.107 6.674

Pr → FG

0.197*** 0.012 17.350

FE → FG

0.566*** 0.018 31.806

ME → FG

0.325*** 0.010 37.924

Note: significant at: * >1.96, ** >2.58, and *** >3.52

levels

Table 4: Overall Fit Assessment for Relationship.

Dependent

variables

R-square Redundancy

ME 0.257 0.119

Pr 0.626 0.037

FE 0.546 0.069

FG 1.000 0.306

From the relationship among K1, C1, K2, and C2

as the independent variables and ME, Pr, FE, and

FG as the dependent variables, the coefficient of

determination (R-square) is 25.7% for ME from K1

and C1, 62.6% for Pr from K1, C1, K2 and C2,

54.6% for FE from K2 and C2, and 100% for FG

from ME, Pr, and FE as shown in Table 4. Since all

ICE-TES 2021 - International Conference on Emerging Issues in Technology, Engineering, and Science

230

the values of R-square are more than 10%, then all

the dependent variables explain the variance in the

independent variables

From path relationships, some of the paths

indicate t-values lower than significance level of

1.96. The confidents, C1 and C2, are statically not

significant to the Practical (Pr), while the confident,

C2, is also statically not significant to the Final

Exam (FE). Knowledge variables (K1 and K2) are

statically significant to the exam, but confident

variables tend not directly significant to the exam.

Confident affects knowledge and knowledge affects

confident as well. However, knowledge is statically

significant to the performance of the students, but

the confident is not statically significant to the

students’ performance. The confidence directs the

students to gain more knowledge which it is an asset

for doing the exam. However, in overall the

interaction of knowledge and confident are

positively significant associated with the final

performance of the students (FG).

The purpose of the simulation is to demonstrate

how the confidence-based testing can improve the

students’ performance by understanding the

relationships between knowledge and confidence to

the exam. The Partial Least Square Structural

Equation Modeling (PLS-SEM) was used to study

the parameters. Since the T-statistic values that

greater than 1.96 show that the relationship both

knowledge and confidence in the first set of quizzes

to the mid exam is statically significant, so both

knowledge (K1) and confidence (C2) are affecting

the achievement of the Mid Exam (ME). However,

the similar relationship cannot be found in the Final

Exam (FE). The relationship of the confidence (C2)

is not statically significant to the final exam, but the

relationship of the knowledge (K2) is statically

significant to the achievement of the final exam and

practice (Pr) as well. Most of the relationship of the

confidence are not statistically significant to the

students’ achievement so knowledge really affects

the students’ performance. Does the confidence

really affect the students’ performance?

Although the scoring of the CBT has

accommodated the weight of the confidence and

knowledge, the confidence is not really affecting the

performance. Knowledge is acquired when the

students learn. The exam is one of the methods to

test how deep the knowledge of the students to the

subject. The confidence does not directly affect to

the students’ performance, but it affects to how the

students gain knowledge.

In the first term, the confidence seems has

influenced to the students’ performance but after

several quizzes it seems no impact. The CBT

method believed that confidence tends to motivate

the students to gain more knowledge whether the

knowledge is true or false. In the structural analysis

course, the subject is in sequence from the session 1

to the final session as it is common in most of the

engineering courses. As the students tend to learn

false knowledge, the confidence led them to learn

false knowledge, so in the beginning the knowledge

and confidence are significant to the students’

performance. However, as the students tend to learn

more and more false knowledge with highly

confidence, the confidence is not as significant as in

the beginning of the students’ achievement, since the

confidence only leads to the false knowledge.

From the test of the relationship by the PLS-

SEM, it can be observed that the combination of

both confidence and knowledge will create an

unstoppable force of human potential, but it can be

destructive as well if it goes to the false direction.

Knowledge is directly affecting the students’

achievement and confidence seems to be a booster

for the students to have higher achievement as well

as lower achievement.

Although the CBT method can stimulate the

reflection learning, the students’ self-awareness

along with the learning process, the deeper learning

with the fairness of the assessment method only can

be obtained if the trainer gives a continuously

feedback to the students.

CONCLUSIONS

The unique confidence-based testing has been used

by some scholars. The CBT method can increase the

competencies of the MCQ exams to become an

effective examination. The CBT stimulate the

reflection for deeper learning among the students

through the students ask to select the right answer as

well as the level of confidence. The statistic results

show that the knowledge is more significant to the

student’s achievement rather than the confidence

level of the certain knowledge. The highly

achievement of the exam can be increased by

implementing the CBT if only the trainer can

continuously make the students realize their false

knowledge in their highly confidence.

ACKNOWLEDGEMENTS

We thank the undergraduate students at Department

Engineering Education: Measuring the Relationship between Knowledge and Conﬁdence to the Student Performance

231

of Civil Engineering, Universitas Kristen Maranatha

who participated in the Structural Analysis class and

provided data for this study.

REFERENCES

Adams, T. M., & W., E. G. (2009). The importance of

confidence in improving educational outcomes. In

Proceedings of the 25th annual conference on distance

teaching and learning (pp. 1–15).

Afthanorhan, W. M. A. B. W. (2013). A Comparison Of

Partial Least Square Structural Equation Modeling

(PLS-SEM) and Covariance Based Structural Equation

Modeling (CB-SEM) for Confirmatory Factor Analysis.

International Journal of Engineering Science and

Innovative Technology (IJESIT), 2(5), 198–205.

Ben-Simon, A., Budescu, D. V, & Nevo, B. (1997). A

Comparative Study of Measures of Partial Knowledge

in Multiple-Choice Tests. Applied Psychological

Measurement, 21(1), 65–88. https://doi.org/10.1177/014

6621697211006

Birkinshaw, J., Morrison, A., & Hulland, J. (1995).

Structural and competitive determinants of a global

integration strategy. Strategic Management Journal,

16(8), 637–655. https://doi.org/10.1002/smj.425016

0805

Bruno, J. (1993). Using testing to provide feedback to

support instruction: A reexamination of the role of

assessment organization. In D. L. & J. Burno (Ed.), Item

bank: Interactive testing and self assesment (NATO

ASI S, pp. 190–209). Berlin: Springer Verlag.

Busse, V., & Walter, C. (2013). Foreign Language Learning

Motivation in Higher Education: A Longitudinal Study

of Motivational Changes and Their Causes. The Modern

Language Journal, 97(2), 435–456. https://doi.org/

10.1111/j.1540- 4781.2013.12004.x

Fornell, C., & Larcker, D. F. (1981). Evaluating Structural

Equation Models with Unobservable Variables and

Measurement Error. Journal of Marketing Research,

18(1), 39–50. https://doi.org/10.2307/3151312

Gardner-Medwin, A. R. (1995). Confidence assessment in

the teaching of basic science. Research in Learning

Technology, 3(1), 80–85.

Gardner-Medwin, A. R. (2006). Confidence-based marking:

Towards deeper learning and better exams. Innovative

Assessment in Higher Education, Routledge.

Gardner-Medwin, A. R., & Gahan, M. (2003). Formative

and summative confidence-based assessment. In

Proceedings of the 7th International Computer-Aided

Assessment Conference (pp. 147–155).

Götz, O., Liehr-Gobbers, K., & Krafft, M. (2010).

Evaluation of Structural Equation Models Using the

Partial Least Squares (PLS) Approach. In V. Esposito

Vinzi, W. W. Chin, J. Henseler, & H. Wang (Eds.),

Handbook of Partial Least Squares: Concepts, Methods

and Applications (pp. 691–711). Berlin, Heidelberg:

Springer Berlin Heidelberg. https://doi.org/10.1007/978-

3-540-32827-8_30

Hair, J. F., Sarstedt, M., Ringle, C. M., & Gudergan, S. P.

(2017). Advanced Issues in Partial Least Squares

Structural Equation Modeling: SAGE Publications.

Hair, J. F., Sarstedt, M., Ringle, C. M., & Mena, J. A.

(2012). An assessment of the use of partial least squares

structural equation modeling in marketing research.

Journal of the Academy of Marketing Science, 40(3),

414–433. https://doi.org/10.1007/s11747-011-0261-6

Henseler, J., Ringle, C. M., & Sinkovics, R. R. (2009).

The use of partial least squares path modeling in

international marketing. In New Challenges to

International Marketing (Vol. 20, pp. 277–319).

Emerald Group Publishing Limited. https://doi.org/

doi:10.1108/S1474- 7979(2009)0000020014

Hunt, D. P. (2003). The concept of knowledge and how to

measure it. Journal of Intellectual Capital, 4(1), 100–

113. https://doi.org/10.1108/14691930310455414

Khan, K. S., Davies, D. A., & Gupta, J. K. (2001).

Formative self-assessment using multiple true-false

questions on the Internet: feedback according to

confidence about correct knowledge. Medical Teacher,

23(2), 158–163.

Kurz, T. B. (1999). A review of scoring algorithms for

multiple-choice tests. San Antonio, TX: Annual

Meeting of the Southwest Educational Research

Association.

McDonald, R. P. (1996). Path Analysis with Composite

Variables. Multivariate Behavioral Research, 31(2),

239–270. https://doi.org/10.1207/s15327906mbr3102_5

Nelson, L., & Webster, S. (2015). How Confident Are

You- Using Knowledge Confidence to Learn Anatomy.

The FASEB Journal, 29(1 Supplement). Retrieved from

http://www.fasebj.org/content/29/1_Supplement/689.10.

abstract

Nunnally, J. C. (1978). Psychometric theory. McGraw- Hill.

Retrieved from https://books.google.co.id/books?id=

WE59AAAAM AAJ

Reed, G. Y., & Reed, K. B. (2015). Engineering Student

Self-Assessment Through ConfidenceBased Scoring.

Advances in Engineering Education, 4(4), 1–23.

Retrieved from http://advances.asee.org/category/

summer-2015/ Ringle, C. M., Wende, S., & Will, A.

(2005). SmartPLS v. 2.0M3. Retrieved from

http://www.smartpls.de/

Salehi, M., & Sadighi, F. (2015). Comparing Confidence-

based and Conventional Scoring Methods: The Case of

an English Grammar Class. The Journal of Teaching

Language Skills, 6(4), 123–152. Retrieved from

http://jtls.shirazu.ac.ir/article_3105_f12428f95f7587c99

8d08fcd1170f24f.pdf

Stiggins, R. (2005). From Formative Assessment to

Assessment for Learning: A Path to Success in

Standards-Based Schools. Phi Delta Kappan, 87(4)

Werts, C. E., Linn, R. L., & Joreskog, K. G. (1974).

Interclass reliability estimates: Testing structural

assumptions. Education and Psychological Measure-

ment, 34, 25-33. doi:10.1177/001316447403400104

ICE-TES 2021 - International Conference on Emerging Issues in Technology, Engineering, and Science

232