Validation of Teacher Competency Questionnaire (TCQ) in Indian

Context

Sourav Choudhury and Vijay Kumar Chechi

Lovely Professional University, Phagwara

Keywords: Teacher, Competence, Validation, Confirmatory Factor Analysis, India.

Abstract: Educators perform a cardinal part in the impartation of quality education. Teacher competence and its

enhancement has humongous significance in forming the new terrain of the twenty-first century which

demands holistic development. The investigators have validated the Teacher Competence Questionnaire

(TCQ) by Meicky Shoreamanis Panggabean and Karel Karsten Himawan (2016) in Indian context, which was

developed to measure teacher competence. It follows a Likert-scale rating format with Student Assessments

of Teachers (SETs) method. The tool has 42 items across five dimensions- Professional Knowledge (5 items),

Professional Skills (17 items), Personal Characteristics (8 items), Personal Ethical Standards and Values (7

items), and Professional Development and Lifelong Learning (5 items). Based on a sample of 350 senior

secondary students (56% males; 44% females) across four districts in the state of West Bengal, India, analysis

was conducted through descriptive statistics such as Mean, Median and Test of normality, Kaiser-Meyer-

Olkin (KMO) Test, Bartlett’s Test of Sphericity (BTS), Confirmatory Factor Analysis (CFA) and Cronbach’s

Alpha using AMOS v26 and SPSS v23. The empirical evidence determined an excellent fit of the retained 23

items in final version with a Cronbach’s alpha coefficient of 0.661, and can be utilized in the Indian context

to measure competence of the teacher.

1 INTRODUCTION

Education is expected to increase the knowledge,

competency and skills of a child without the curb of

time and place (Bhardwaj, A, 2016). Researchers and

educators have widely investigated for decades on

which school aspect has the most impact on the

students’ academic performance. As policymakers

have grown to be more implicated in school reforms,

this question takes on new importance since many

endeavors depend on assumed relationships between

multiple teaching-related factors and learning

outcomes (Álvarez, J., and Patrinos, H. A, 2007;

Kyriakides, L, 2013). Despite the little difference

conventional methods that schools input make in

student learning, a budding body of research in India

and worldwide conclude that schools hold a

significant impact on the students learning with the

teacher-related aspects playing the major role (Singh

and Sankar, 2015; Dev, M, 2016). Black and William

(1998) compiled evidence from more than 250 studies

and the outcome was an unmistakable and distinct

message: that student achievement can be raised by

initiatives designed to improve the competency of

teachers to promote learning outcomes. Smith’s

(2003) assertion that "Teacher quality matters" as

cited by Black Williams (1998) and Greenstein

(2010), also lends credence to this connection

between teacher practice and student success, and

concluded it to be the most significant school-related

factor that affects student performance.

The changing education paradigms also demands

character development alongside moral and social

growth, social efficiency and spiritualism (Sootipon,

2010). Recognizing this humongous potential that

education and educators hold for both the child and

society, nations have commitment towards

universalization of elementary education with the

overt aim of providing ‘Quality Education for All’

(DKE Lipsky and Gartner,1989). A vital need to

improve the overall quality of education in India has

also been recognized by an Act of Parliament of the

National Council for Teacher Education, 1993. This

brought about a major shift from the previous view

and laid down dedication, execution and competence

as foremost standards for teacher’s education (Rajput

and Walia, 2001). Consequently, educators need to

also foster their competencies in practice as per this

926

Choudhury, S. and Chechi, V.

Validation of Teacher Competency Questionnaire (TCQ) in Indian Context.

DOI: 10.5220/0012529500003792

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 1st Pamir Transboundary Conference for Sustainable Societies (PAMIR 2023), pages 926-931

ISBN: 978-989-758-687-3

educational paradigm shift (Surasak, 2013; Khalid, et

a., 2021). Numerous experts in the field have

undertaken the pursuit of defining competence based

on one’s personal characteristics. Blömeke et al.

(2015) view “competence as a continuum from traits

to observed behaviors in real-world situations”.

Jinga and Istrate (2001) define it according to their

“cognitive, affective, motivational and managerial

skills, which interacts with their personality traits”.

Tigelaar et al. (2004) define teaching competency as

“an amalgamated set of personal characteristics,

knowledge, skills, attitudes aimed to achieve effective

performance”.

According to the Planning Commission 2012–

2017, Government of India, teacher development is

therefore regarded as essential for the quality

education in India going forward. The skill of a

person can be recognized through the work conduct

and that will be a triumph pointer for the association

as opposed to only his/her educational level (Peklaj,

C, 2015; Nessipbayeva, O, 2012) Thus, the current

human resource improvement ought to underscore on

the ability of the educators and not just their degrees.

Thus, globally, there is interest in assessing their skill

incited by interest for guaranteeing quality and for

more noticeable affirmation of this calling (Vermunt,

J. D. and Verloop, N. (1999). Numerous

measurement tools have been formed to quantify the

competence of educators at various stages in their

careers such as enrollment, experience and

professional development (Dwyer, 1998). Academic

institutions as a rule utilize Student Assessments of

Teachers (SETs) in eliciting remarks and criticism

about the teaching process, also as a matter of fact,

many schools rely upon SETs to settle choices

associated with workforce maintenance, promotion

and advancement, tenure of contract and salaries

(Jacobs, 2004). SETs continue being the primary

choice of execution indication of showing viability

and quality with outstanding effort facilitated at

rendering SETs trustworthy and significant (Marsh

and Roche, 1997; Toland and De Ayala 2005). Most

SETs are straightforwardly drawn from perceptions

of educators and don't actually quantify critical work

parts. This may not feature portions of their teaching

that are colossal from students perspective, or that

mirror their qualities. There brews the necessity for

student incorporation at the primary stage, without

restricting them to assessing the properties or

practices proposed by educators.

2 LITERATURE REVIEW

A review of existing scales and their properties was

undertaken across various nations worldwide on the

target group of respondents for assessing the

educators competence. While most of them utilized a

Likert-scale format across various dimensions

elicited on the basis of competence, teachers remain

to be the sole respondents. Eg: Gresham and Elliott,

1990, in their scale Social Skills Rating Scale (SSRS)

consisting of 53 items, collected data from the

teachers themselves. Likewise, Plake, Impara and

Fager’s (1993), Raju (1994), Dr. Jayaramanna

(Kammati, 2001), Silva and Martorell (2001),

D’Agostino, J. V., and VanWinkle, W. H. (2007),

Spanierman et al., (2011), Raimundo and colleagues

(2012), Tom, K (2012), Murugan and Rajammal

(2018), Devrim E (2020), Debabrata Bhattacharjee

and Remith George Carri (2020) and, Gumus MM

and Volkan K (2022) had teachers has the

respondents. This format can lead to tremendous bias.

Scales that elicited data from the students were

handful such as by Tang F, et al., (2005), Nurdock

T.B, et al., (2001) based on Wentzel’s (1997)

qualitative data, Aaron MT (2019) and Kamila

Ludwikowska (2019). On the parallel, Gilbert F.

Shearron (1978) states that choosing a rating scale

over questionnaire, product or interviews is the most

apt method to measure competence. The Teacher

Competence Questionnaire (TCQ) by Meicky

Shoreamanis Panggabean and Karel Karsten

Himawan (2016) is one such scale that not only

employs a SET but also a Likert type format and

hence, proved an apt scale to measure teacher

competence.

3 OBJECTIVE

To validate the Teacher Competence Questionnaire

(TCQ) by Meicky Shoreamanis Panggabean and

Karel Karsten Himawan (2016) in the Indian context.

3.1 Methodology &Scale Specifications

3.1.1 Study Period and Area

This research survey was conducted in 2022 among

senior secondary school students from four districts

of West Bengal, India- Howrah, North 24

paraganas, Malda and Uttar Dinajpur.

Validation of Teacher Competency Questionnaire (TCQ) in Indian Context

927

3.1.2 Respondents, Sampling Technique and

Data Collection

The survey was conducted among 350 senior

secondary students of class 11

. The questionnaire

consisting of demographic details and the scale was

developed using Google forms and was shared to

students of the researcher’s contacts and networks in

different schools across the four districts in the state

of West Bengal. A random sampling strategy of

deciding the maximum sample size was adopted. The

students were contacted after receiving the google

form and were explained on the purpose and

procedure of the survey. They were also notified that

this was voluntary and did not hold any stakes on their

studies if withdrawn. As the samples were drawn

from different genders, castes, domains (Arts, science

and commerce) and socio-economic background, it is

well inclusive and diversified to be accepted as being

well representative. Thus, conclusions concerning the

given topic can be generalized.

3.1.3 Scale

The scale has a total 42 statements which are to be

rated on a five-point continuum: (AD) Absolutely

Disagree, (D) Disagree, (N) Neutral, (A) Agree, (AA)

Absolutely Agree. The domains are Professional

Knowledge (5 items), Professional Skills (17 items),

Personal Characteristics (8 items), Personal Ethical

Standards and Values (7 items), and Professional

Development and Lifelong Learning (5 items). The

minimum score is 1 and maximum score 5. The

domains mentioned in the scale are common for the

eleven south Asian countries and are:1) Teaching

style competence, 2) Individual competence, 3)

Societal competence, and 4) Skilled competence.

3.1.4 Data analysis

The data was fed into AMOS v26 and SPSS v23, and

examined for inappropriate values, outliers, and

completeness (Black, et al. 2010). The data analysis

began with demographic details revealing 56% males

and 44% females. Right after that, inorder to

determine the adequacy of the sample size Kaiser-

Meyer- Olkin (KMO) Test and Bartlett’s Test of

Sphericity (BTS) as an initial requirement to perform

CFA was determined. Kline (2011) and Joseph et al.

(2012) detailed that the rationale of CFA is to

examine the standing theory or model, or authenticate

the factor structure of a group of prevalent variables

which is the five-factor structured Likert type scale in

this case. As the scale was developed with priori

theory, CFA alone proves to be well sufficient to be

carried out. Also, since the fitness estimates of the

model is considered here, CFA is the apt analysis to

be undertaken in this fairly new scale to measure the

validity and reliability in Indian context (Hurley, A.E,

et al, 1997).

4 RESULTS

The test results are elaborated based on descriptive

statistics, test of normality, Confirmatory Factor

Analysis and Reliability score for each factor. The

descriptive statistics of the samples which describe

and summarize the overall features of the data as well

as its distribution was assessed. As the minimum

score given is 1 and maximum is 5 as per the

questionnaire’s rating, we noticed in the descriptive

statistics results that the lowest score (Mean) is 2 and

the maximum is 4.5.

Following this, Kolmogorov-Smirnov

Goodness

of fit test was run to check the distribution of the data

and assumption of normality. It compares the

probability distribution of a hypothetical data and the

data fed into the system (Kolmogorov–Smirnov

distribution, 2021). The p value was 0.00 which is

less than 0.05 after Kolmogorov-Smirnov Z indices

were analyzed, and shows that the distribution is

normal. Likewise, Shapiro-Wilk test was also run

inorder to determine if normality assumptions of the

study data was met (Lilliefors, H. W., 1967). The p=

0.000 i.e., < 0.05 and proves normality.

Next, the sample size adequacy using KMO Test and

BTS as a requirement to implement Confirmatory

Factor Analysis (CFA) was executed as the

subsequent step. The KMO value retrieved is 0.781

and thus, is greater than 0.6 and even close to 1.0

(highly adequate). The value of significance of BTS

for homogeneity of variance χ2=4936.069; p 0.000

i.e., <0.05 proves variance to be homogenous and

acting as an approval for CFA analysis to be

conducted (Hair et al., 2010; Tabachnick and Fidell,

2007).

4.1 CFA- Confirmatory Factor

Analysis

CFA was employed to investigate the validity of the

hypothesized dimensions and to improve the model

fit. In this study, the five subscales are highly

correlated with the 42 items and was tested in a

sample size of 350 students. The initial model was run

and the results of few individual loading factors fell

below the cross loading standard value of 0.5. The

goodness of fit estimates was poor and led to the

PAMIR 2023 - The First Pamir Transboundary Conference for Sustainable Societies- | PAMIR

928

deletion of the following 18 items: PK 5, PSPED 5

and 6, PSCM 2 and 3, PSLA 1, 3 and 5, PC 2,3,5,6

and 8, ECV 2,3 and 6, and PDLL 1. Thus, second

model fit estimate was again carried out after

removing the 17 items and 1 item i.e., ESV 5 had an

individual loading factor of less than 0.5 and led to its

deletion. Following this, third fitness estimate was

run and met the model estimates (Figure 1).

Table 1: KMO and Bartlett’s test

Kaiser-

Meyer-Olkin

(KMO) Measure of

Sampling

Adequacy

0.781

Bartlett's Test

of Sphericity

Approx.

Chi-Square

Sig.

4936.069

820

0.000

The factor models tested and accompanying fit

indices are shown in Table 2. The good estimate for

all the fit indices retrieved like GFI (Goodness of-Fit

Index) 0.914, AGFI (Adjusted Goodness-of-Fit

Index) 0.886, CFI (Comparative Fit Index) 0.956,

RMSEA (Root Mean Square Error of Approximation)

0.047, PCFI (Parsimony Comparative Fit Index)

0.789, IFI (Incremental Fit Index) 0.956 and CFI

(comparative fit index) 0.956 are mentioned here. It

shows good overall estimates with their values

according to the desired benchmarks, thus making the

model satisfactory (Ding and Ng, 2008).

Figure 1. The fitness estimates of the model.

A scale is not valued as credible or valid lest its

reliability be furnished (Tavakol and Dennick, 2011).

In this present survey, the updated scale version’s

reliability was calculated using Cronbach’s alpha,

which if 0.70 or above is considered meeting the

benchmark value to conclude if the scale as a whole

and its subscales is/are reliable or not. The factor wise

reliability coefficient of the seven subscales showed

are 0.713 for professional knowledge, 0.842 for

Pedagogies, 0.802 for classroom management, 0.751

for learners assessment, 0.951 for personal

characteristics, 0.574 for ethical standards and values

and 0.816 for professional development and lifelong

learning. The range of these Cronbach's alpha

coefficients, nevertheless, attests to the good

reliability value in line with the that suggested by Hair

et al., (2010). Additionally, the overall Cronbach’s

alpha coefficient of the final scale is 0.661, signifying

that it is very reliable. The finalized scale with 23

items has high validity and reliability and can be used

in Indian settings to assess the teacher’s competence.

Table 2 A.

Measures P CMIN/

RMR RMSEA GFI

Result 0.00 1.704 0.026 0.047 0.914

Benchmark <0.05 <3 <0.08 <0.1 >0.90

Table 2 B.

Measures AGFI PCFI IFI CFI

Result 0.886 0.789 0.956 0.956

Benchmark 0 -1 >0.8 >0.90 >0.95

5 CONCLUSION

The purpose of present investigation was to validate

Teacher competence Questionnaire (TCQ) by

Meicky Shoreamanis Panggabean and Karel Karsten

Himawan (2016) in Indian context. It is composed of

a clear five-factor structure and originally consists of

42 competency items. It is also precise and

emphasizes only on those aspects that are under the

power of educators. The scale, likewise, does not

confound assessment of teaching with the assessment

of the process or workload/assignments. The result of

Confirmatory Factor Analysis of the questionnaire

has given approvable results, the psychometrics are

highly reliable and valid to assess teacher’s

competencies in Indian context. After the inspection

of the scale, variances and modification indices, 19

statements were deleted. Empirical evidence

demonstrates an excellent score having a final set of

23 items. The reliability of the scale is 0.661 and

Validation of Teacher Competency Questionnaire (TCQ) in Indian Context

929

according to George and Mallery (2003) there exists

good degree internally.

REFERENCES

Álvarez, J., Moreno, V. G., and Patrinos, H. A. (2007).

Institutional effects as determinants of learning

outcomes: Exploring state variations in Mexico (Vol.

4286). World Bank Publications.

Bhattacharjee, D. and Carri, R. G. (2020). An investigation

into the teaching competency of english teachers of

Tripura board of secondary education. Rupkatha

Journal on Interdisciplinary Studies in Humanities.

12(1): 1-1. Bhardwaj, A. (2016). Importance of

education in human life: A holistic approach.

International Journal of Science and Consciousness.

2(2): 23-28.

Black, P. and Wiliam, D. (1998). Assessment and

classroom learning. Assessment in Education:

Principles, Policy & Practice. 5(1): 7–74.

Churchill, G.A. (1979),A paradigm for developing better

measures of marketing constructs. Journal of Marketing

Research. Vol. 16 No. 1, pp. 64–73

Cochran-Smith, M. (2003). Teaching quality matters.

Journal of Teacher Education. 54(2): 95-98.

Cronbach, L. J. (1951). Coefficient alpha and the internal

structure of tests. Psychometrika, 16(3), 297–334.

Dev, M. (2016). Factors Affecting the Academic

Achievement: A Study of Elementary School Students

of NCR Delhi, India. Journal of Education and Practice.

7(4): 70-74.

Ding, Z. and Ng, F. (2008). A new way of developing

semantic differential scales with personal construct

theory. Construction Management and Economics.

26(11): 1213–1226.

D’Agostino, J. V. and VanWinkle, W. H. (2007).

Identifying prepared and competent teachers with

professional knowledge tests. Journal of Personnel

Evaluation in Education. 20(1): 65-84.

George, D. and Mallery, P. (2003). SPSS for Windows step

by step: A simple guide and reference. 11.0 update (4th

ed.). Boston: Allyn & Bacon.

Greenstein, L. (2010). What teachers really need to know

about formative assessment. ASCD.

Hair, J. F., Gabriel, M. and Patel, V. (2014). AMOS

covariance-based structural equation modeling (CB-

SEM): Guidelines on its application as a marketing

research tool. Brazilian Journal of Marketing. 13(2).

Himawan, M. S. P. K. K. (2016). The development of

Indonesian teacher competence questionnaire. Journal

of Educational, Health and Community Psychology.

5(2): 1-15.

Hurley, A. E., Scandura, T. A., Schriesheim, C. A.,

Brannick, M. T., Seers, A., Vandenberg, R. J. and

Williams, L. J. (1997). Exploratory and confirmatory

factor analysis: Guidelines, issues, and alternatives.

Journal of organizational behavior. 667-683.

Kline, R. B. (2015). Principles and practice of structural

equation modeling. Guilford publications.

Lilliefors, H. W. (1967). On the Kolmogorov-Smirnov test

for normality with mean and variance unknown.

Journal of the American statistical Association.

62(318): 399-402.

Khalid, M., Hashmi, A., Javed, Z., and Javed, I. (2021).

Effect of Teachers’ Advance Knowledge and Pedagogy

Skills on Students’ Academic Performance. Ilkogretim

Online. 20(4): 2009-2014.

Kyriakides, L., Christoforou, C. and Charalambous, C. Y.

(2013). What matters for student learning outcomes: A

meta-analysis of studies exploring factors of effective

teaching. Teaching and teacher education. 36:143-152.

Lipsky, D. K. E. and Gartner, A. E. (1989). Beyond separate

education: Quality education for all. Paul H. Brookes

Publishing.

Ludwikowska, K. (2019), "Teacher competence inventory:

An empirical study on future-oriented competences of

the teaching profession in higher education in India",

Education + Training. Vol. 61 No. 9: 1123-1137.

Marsh, H. W. and Roche, L. A. (1997). Making students'

evaluations of teaching effectiveness effective: The

critical issues of validity, bias, and utility. American

psychologist. 52(11): 1187.

Nessipbayeva, O. (2012). The Competencies of the Modern

Teacher. Bulgarian Comparative Education Society.

Peklaj, C. (2015). Teacher competencies through the prism

of educational research. Center for Educational Policy

Studies Journal. 5(3): 183-204.

Plake, B. S., Impara, J. C. and Fager, J. J. (1993).

Assessment competencies of teachers: A national

survey. Educational Measurement: Issues and Practice.

12(4): 10-12.

Rajput, J. S. and Walia, K. (2001). Reforms in teacher

education in India. Journal of educational change.

2(3):239-256.

Raimundo, R., Carapito, E., Pereira, A. I., Pinto, A. M.,

Lima, M. L. and Ribeiro, M. T. (2012). School social

behavior scales: an adaptation study of the Portuguese

version of the social competence scale from SSBS-2.

The Spanish Journal of Psychology. 15(3):1473-1484.

Shearron, G. F. (1978). Developing and Improving

Instruments for Measuring Competence. Journal of

Teacher Education. 29(2): 18–20.

Singh, R. and Sarkar, S. (2015). Does teaching quality

matter? Students learning outcome related to teaching

quality in public and private primary schools in India.

International Journal of Educational Development.

41:153-163.

Sootipon, J. (2010). The changes of learning world in the

21st Century and development for professional

teachers. World wide web.

Spanierman, L. B., Oh, E., Heppner, P. P., Neville, H. A.,

Mobley, M., Wright, C. V.... and Navarro, R. (2011).

The multicultural teaching competency scale:

Development and initial validation. Urban Education.

46(3): 440-464.

PAMIR 2023 - The First Pamir Transboundary Conference for Sustainable Societies- | PAMIR

930

Tabachnick, B. G., Fidell, L. S. and Ullman, J. B. (2007).

Using multivariate statistics (Vol. 5). Boston, MA:

pearson. 481-498

Tang, F. I., Chou, S. M. and Chiang, H. H. (2005). Students'

perceptions of effective and ineffective clinical

instructors. Journal of nursing education. 44(4):187-

192.

Tavakol, M. and Dennick, R. (2011). Making sense of

Cronbach's alpha. International journal of medical

education. 2: 53.

Toland, M. D. and De Ayala, R. J. (2005). A multilevel

factor analysis of students’ evaluations of teaching.

Educational and Psychological Measurement. 65(2):

272-296.

Vermunt, J. D. and Verloop, N. (1999). Congruence and

friction between learning and teaching. Learning and

instruction. 9(3): 257-280.

Wentzel, K. R. (1997). Student motivation in middle

school: The role of perceived pedagogical caring.

Journal of Educational Psychology. 89: 411–417

Validation of Teacher Competency Questionnaire (TCQ) in Indian Context

931