A TOOL FOR MEASURING INDIVIDUAL INFORMATION

COMPETENCY ON AN ENTERPRISE INFORMATION SYSTEM

Chui Young Yoon, In Sung Lee and Byung Chul Shin

School of Electrical, Electronic & Computer Engineering, Chungbuk National University

410 Sungbong-ro, Heungduk-gu, Cheongju city, Chungbuk, 361-763, South Korea

Keywords: Information competency, Measurement factor, Measurement tool, Measurement system.

Abstract: This study presents a tool that can efficiently measure individual information competency to execute the

given tasks on an enterprise information system. The measurement items are extracted from the major

components of a general competency. By factor analysis and reliability analysis, a 14-item tool is proposed

to totally measure individual information capability. The tool’s application and utilization are confirmed by

applying it to measuring the information competency of the individuals in an enterprise.

1 INDIVIDUAL INFORMATION

COMPETENCY

In this study, an individual is defined as a person

who directly interacts with his or her information

systems based on previous studies (Rockart and

Flannery, 1983; Martin, 1982). Competency is a

total set of knowledge, skills, and attitudes as the

action characteristics of an organizational member

that can do his or her tasks outstandingly and

efficiently in an organizational environment

(Mirable, 1997; Arthey and Orth, 1999; Rodriguez et

al., 2002). In general competency, individual

characteristics such as motives, traits, self-concepts

and knowledge lead to skills, and the action of a

person with skills has an effect on the performance

of his or her business in an organizational

environment (Spencer and Spencer, 1993). Namely,

information competency can be defined by

transforming a general competency into a type of

competency in an information perspective.

Hence, the individual information competency

(IIC) can be defined as a total set of knowledge,

technology, skills, and attitudes which function as

action characteristics of an organizational member

who can do his or her tasks outstandingly and

efficiently on an enterprise information system. With

these studies, we generated the first 24 items for

efficiently measuring an IIC on an enterprise

information system.

2 RESEARCH METHODS

In previous literature, most studies presented two

methods of model construct validation (Etezadi-

Amoli and Farhoodmand, 1996; Torkzadeh and Doll,

1999; and Torkzadeh and Lee, 2003): (1)

correlations between total scores and item scores

(Torkzadeh and Doll, 1999; Torkzadeh and Lee,

2003), and (2) factor analysis (Etezadi-Amoli and

Farhoodmand, 1996; Torkzadeh and Lee, 2003).

This study used factor analysis and reliability

analysis to verify the tool construct and to extract

adequate items for measuring an IIC. The

measurement questionnaire used a five-point Likert-

type scale from 1 (not at all) to 5 (very good). The

survey was gathered data from a variety of industries

and business departments. A sample of 258 usable

responses was identified in business departments as

follows: strategy planning (20.9%), development

and maintenance (26.8%), business application

(38.4%), and administration support (13.9%).

3 ANALYSIS AND DISCUSSION

After factor analysis and reliability analysis, the first

24 measurement items were reduced to 14 items,

with 10 items were deleted. The elimination was

considered sufficient to ensure that the retained

items were adequate measures of IIC. These

analyses were used to identify the underlying factors

or components that comprise the IIC construct. Each

289

Yoon C., Lee I. and Shin B. (2009).

A TOOL FOR MEASURING INDIVIDUAL INFORMATION COMPETENCY ON AN ENTERPRISE INFORMATION SYSTEM.

In Proceedings of the 11th International Conference on Enterprise Information Systems - Artiﬁcial Intelligence and Decision Support Systems, pages

289-292

DOI: 10.5220/0001855202890292

 SciTePress

of the 14 items had a factor loading > 0.635. The

reliability coefficients (Cronbach’s alpha) of four

potential factors had values > 0.792, above the

threshold recommended for exploratory research

(Rodriguez, Patel, Bright, and Gowing, 2002).

Table 1: Factor loadings obtained from factor analysis.

0.727V23

0.635V24

0.786V21

0.723V18

0.719V17

0.781V16

0.894V14

0.702V12

0.713V11

0.787V10

0.839V08

0.642V06

0.713V03

0.754V01

Factor 4Factor 3Factor 2Factor 1

Factor Loadings

Variable

0.727V23

0.635V24

0.786V21

0.723V18

0.719V17

0.781V16

0.894V14

0.702V12

0.713V11

0.787V10

0.839V08

0.642V06

0.713V03

0.754V01

Factor 4Factor 3Factor 2Factor 1

Factor Loadings

Variable

* Significant at P ≤ 0.01

The descriptions and loadings for the 14 items are

presented in Table 1 and Table 2.

Table 2: Corrected item-total correlations and coefficient

alpha for each factor.

Coefficient alpha for the above 3 items as a composite measure of Factor =0.798

0.5930.624V24

0.7380.712V23

0.7240.692V21

Coefficient alpha for the above 4 items as a composite measure of Factor =0.901

0.7980.738V18

0.8360.634V17

0.7780.826V16

0.8520.743V14

Coefficient alpha for the above 4 items as a composite measure of Factor =0.884

0.7230.629V12

0.8210.817V11

0.8480.714V10

0.8470.781V08

Coefficient alpha for the above 3 items as a composite measure of Factor =0.792

0.6270.678V06

0.7240.689V03

0.8120.728V01

Alpha if item deleted

Corrected item-total

correlation

Variable

Coefficient alpha for the above 3 items as a composite measure of Factor =0.798

0.5930.624V24

0.7380.712V23

0.7240.692V21

Coefficient alpha for the above 4 items as a composite measure of Factor =0.901

0.7980.738V18

0.8360.634V17

0.7780.826V16

0.8520.743V14

Coefficient alpha for the above 4 items as a composite measure of Factor =0.884

0.7230.629V12

0.8210.817V11

0.8480.714V10

0.8470.781V08

Coefficient alpha for the above 3 items as a composite measure of Factor =0.792

0.6270.678V06

0.7240.689V03

0.8120.728V01

Alpha if item deleted

Corrected item-total

correlation

Variable

In order to research the reliability and validity of the

measures, we calculated the corrected item-total

correlations between each variable and its

corresponding factor. This also shows the alpha

coefficients for the measurement of factors if a

measure was deleted from the scale. These

coefficients indicate the relative contribution of a

measure to the construction of a scale for measuring

a particular factor. They are all in the acceptable

range. Most corrected item-total correlations were

greater than 0.600, showing that the individual

measures are good indicators of their corresponding

factors. Each of the 14 items had a corrected item-

total correlation > 0.624. The correlation for each

item was positive and significant (p = 0.01 or below).

Hence, the measurement items with a validity and

reliability were extracted by carrying two analyses

as shown in Table 1 and Table 2. However, efforts

to present additional evidence of this tool’s validity,

internal consistency, and stability are encouraged.

4 MEASUREMENT TOOL

The extracted 14 items were classified as 4 factor

groups. The 4 factor groups indicate the potential

factors that can measure the IIC. With researching

the measurement items of each factor, we identified

the 4 potential factors as shown in Figure 1.

Information Understanding (IU)

IU01 Do you understand the Internet and information society ?

IU03 Do you know IT progress trends in IT leading countries ?

IU06 Do you consider an etiquette in using your information systems ?

Information Knowledge (IK)

IK08 Do you know hardware, software, networks, and database for your information systems ?

IK10 Do you have solution knowledge related to ERP, SCM, CRM, and e-Commerce ?

IK11 Do you know how to use B2E, B2C, and B2B on your information systems ?

IK12 Do you know how to establish security measures on your information systems ?

Information Application (IA)

IA14 Can you use word processing, spreadsheets, and presentation in on your information systems ?

IA16 Can you use the solutions such as ERP, SCM, CRM, and E-Commerce ?

IA17 Can you apply your information systems to B2E, B2C, and B2B ?

IA18 Can you establish and manage information security measures on your information systems ?

Information Potential (IP)

IP21 How long did you work at IT departments ?

IP23 How many did you participate in overseas or domestic educations and trainings related to IT ?

IP24 How many did you present your articles and ideas for a task improvement on your enterprise’s

webpage ?

Information Understanding (IU)

IU01 Do you understand the Internet and information society ?

IU03 Do you know IT progress trends in IT leading countries ?

IU06 Do you consider an etiquette in using your information systems ?

Information Knowledge (IK)

IK08 Do you know hardware, software, networks, and database for your information systems ?

IK10 Do you have solution knowledge related to ERP, SCM, CRM, and e-Commerce ?

IK11 Do you know how to use B2E, B2C, and B2B on your information systems ?

IK12 Do you know how to establish security measures on your information systems ?

Information Application (IA)

IA14 Can you use word processing, spreadsheets, and presentation in on your information systems ?

IA16 Can you use the solutions such as ERP, SCM, CRM, and E-Commerce ?

IA17 Can you apply your information systems to B2E, B2C, and B2B ?

IA18 Can you establish and manage information security measures on your information systems ?

Information Potential (IP)

IP21 How long did you work at IT departments ?

IP23 How many did you participate in overseas or domestic educations and trainings related to IT ?

IP24 How many did you present your articles and ideas for a task improvement on your enterprise’s

webpage ?

Individual

Information

Competency

(IIC)

Information

Understanding

(IU)

Information

Application

(IA)

Information

Knowledge

(IK)

Information

Potential

(IP)

V01 V03 V06 V08 V11 V12 V14 V17 V18 V21 V23 V24

V10 V16

Figure 1: Structure of the developed measurement tool.

These are considered as the major measurement

factors of the tool construct. Figure 1 shows the

structure of the measurement tool with the 4

potential factors and 14 measurement items. Each

factor has three or four measurement items, and each

item is composed of two or three measurement

problems from the measurement problem database.

As presented in Figure 1, the information

understanding is the realm where measures concepts,

attitude, and adaptability on information. The

ICEIS 2009 - International Conference on Enterprise Information Systems

290

information knowledge indicates the knowledge that

an individual has to know to efficiently apply

information solutions and systems to his or her tasks.

The information application means the ability that an

individual effectively apply information knowledge,

solutions, and systems to his or her tasks. The

information potential refers to the potential

development probability of the IIC by job

experience, participation of domestic and overseas

educations and trainings, and presentation of articles

and ideas for a task improvement on the enterprise

website.

Hence, this tool with 4 factors and 14 items is an

important theoretical construct to measure an

individual’s total information ability.

5 MEASUREMENT SYSTEM

The measurement system, Figure 2, is comprised of

two main processes including the measurement stage

and interpretation stage.

Start

Measurement Tool

(Information understanding,

Knowledge, Application,

and Information Potential)

Measurement

Problem

Database

Performing

Measurement or Checking

Results ?

Extraction of Measurement Problems

(4 Factors/14 Items)

Measurement Execution

(Questionnaire, Written,

and Application Test)

Measurement System

(ID/PW)

Personal

Information

Database

Analysis of Measurement Results

(Extraction of Measurement Index)

Presentation of Measurement Results

(Problems and Improvement Methods)

Weight Values

Database

(4 Factors)

End

Measurement

Results

Database

Check

Perform

Figure 2: Framework of the measurement system.

The former extracts the problems based on each

measurement factor and its items from the problem

database. The problems have three kinds of problem

forms such as a questionnaire test, a written test, and

an application test based on the peculiarity of each

factor. After generating the measurement problems,

the tool examines an individual by the extracted

problems. The results are analyzed by extracting the

measurement values of each factor, and by applying

each weight value to the values of each factor. The

latter explained the measurement results based on

each measurement index extracted from each factor.

The interpretation presents the present states and

problems of the IIC, and the directions and methods

to efficiently improve the IIC.

5.1 Measurement Method

This study used the weight values for each factor in

order to develop an objective and efficient tool

considered the relative importance of each factor in

measuring the IIC as shown in Table 3.

Table 3: Weight value of each measurement factor.

The measurement index (MI) means the value

extracted by multiplying the weight value by the

measurement value of each factor. The sum of the

measurement indices of each factor becomes the

total MI of the individual. In this way, this tool

presents the measurement results of the IIC based on

the total MI and the MI of each factor.

6 CASE STUDY

AND DISCUSSION

This case study applied the developed tool to 163

workers working in “B” enterprise, South Korea.

The business departments of respondents were

identified as follows: the strategy plan department:

23.1%; development and maintenance department:

21.3%; business application department: 37.4% and

administration support department: 18.2%.

First, we present the measurement results of each

business department of the overall organization as

shown in Figure 3. Total MI of the organization was

61.58. The business application department (BAD)

were 65.78. The MI of the BAD was higher than

those of the other departments. This is due to the

ability to effectively accomplish their tasks by

frequently applying information knowledge,

solutions, and systems to their tasks on an enterprise

information system.

0.20Information Potential

0.33Information Application

0.25Information Knowledge

0.22Information Understanding

Weight ValueMeasurement Factor

0.20Information Potential

0.33Information Application

0.25Information Knowledge

0.22Information Understanding

Weight ValueMeasurement Factor

A TOOL FOR MEASURING INDIVIDUAL INFORMATION COMPETENCY ON AN ENTERPRISE INFORMATION

SYSTEM

291

Total

Measurement

Index

Strategy

Plan

Department

Development &

Maintenance

Department

Business

Application

Department

Administration

Support

Department

61.58

62.12

60.34

65.78

58.09

Figure 3: Measurement indices of each business

department and overall organization.

Second, the measurement results of an individual

working in the administration support department

(ASD) were presented as a sample. The MI of each

factor was generated by multiplying each weight

value by the measurement value of each factor. The

total MI of an individual is the sum of the

measurement indices of each factor as shown in

Table 4.

Table 4: Extraction process of the total measurement index

for an individual.

63.0911.2921.2415.0315.53

Calculation of Total

Measurement Index

1.000.200.330.250.22

Weight Values of

Each Factor

-56.4664.3760.1261.48

Measurement Indices

of Each Factor

Total

Measurement

Index

Information

Potential

Information

Application

Information

Knowledge

Information

Understanding

Division

63.0911.2921.2415.0315.53

Calculation of Total

Measurement Index

1.000.200.330.250.22

Weight Values of

Each Factor

-56.4664.3760.1261.48

Measurement Indices

of Each Factor

Total

Measurement

Index

Information

Potential

Information

Application

Information

Knowledge

Information

Understanding

Division

The individual MI was 63.09. The MI of the

information application was a little high. This means

the outstanding application ability that the individual

can apply the information knowledge, solutions, and

systems to his or her given tasks on an enterprise

information system. But the information potential

was very low.

63.09

61.48

60.12

64.37

56.46

Total

Measurement

Index

Information

Understanding

Information

Knowledge

Information

Application

Information

Potential

Figure 4: Measurement indices of an individual in the

ASD.

Hence, this individual should endeavour after IT

educations and trainings, job experience, and

presentation of articles on the firm’s website in order

to efficiently raise his or her total information ability.

7 CONCLUSIONS

This tool can be used in measuring an IIC to perform

the given tasks on an enterprise information system.

This presents the concrete items, process, and

method to measure the IIC. Hence, this 14-item tool

maybe provides a new direction and foundation for

developing the efficient measures for an IIC.

REFERENCES

Arthey, Timothy R., and Orth, Michael S, 1999. Emerging

Competency Methods for The Future, Human

Resource Management, Vol. 38, No. 3, pp.215-226.

Etezadi-Amoli, J., and Farhoomand, Ali F., 1996. A

structural model of end-user computing satisfaction

and user performance, Information and Management,

30, pp.65-73.

Martin, J., 1982. Application development without

programmers, Prentice-Hall, Eaglewoods, Cal.

McCoy, Randall W., 2001. Computer Competencies for

The 21st Century Information Systems Educator,

Information Technology, Learning, and Performance

Journal, Vol. 19, No. 2.

Mirable, R., 1997. Everything You wanted to know about

Competency Modeling, Training and Development, pp.

73-77

Munro, Malcolm C., Huff, Sid L., Marcolin, Barbara L.,

and Compeau, Deborah R., 1997. Understanding and

Measuring User Competence, Information &

Management Vol. 33, pp.45-57.

Rifkin, Kenneth I., Fineman, Michal and Ruhnke, Clara H.,

1999. Developing Technical Managers – First You

Need a Competency Model, Research & Technology

Management, pp.53-57.

Rockart, J. and Flannery, L., 1983. The management of

end user computing, Communication of the ACM 26,

10, pp.776-784.

Rodriguez, D., Patel, R., Bright, A., Gregory, D., and

Gowing, M. K., 2002. Developing Competency

Models to Promote Integrated Human Resource

Practices, Human Resource Management, Vol. 41, No.

3, pp.309-324.

Torkzadeh, G. & Doll, W. J., 1999. The Development of a

Tool for Measuring the Perceived Impact of

Information Technology on Work, Omega,

International Journal of Measurement Science, Vol.

27, pp.327-339.

Torkzadeh, Gholamreza and Lee, Jungwoo, 2003.

Measures of Perceived End-user’s Computing Skills,

Information & Management, 2018, pp.1-9.

ICEIS 2009 - International Conference on Enterprise Information Systems

292