CONSTRUCTING A COMPUTER SIMULATED EXPERIMENT
TESTING SYSTEM
Ben Yang, Shardrom Johnson* and Daniel Hsu
School of Computer Engineering and Science, Shanghai University, Shanghai 200072, P. R. China
*Information Centre of Shanghai Municipal Education Commission, Shanghai 200433, P. R. China
Keywords: Computer Simulated Experiment, CAT, CSET.
Abstract: This paper describes applications of Computer-Assisted theory to testing; reviews related research on
Computer-Assisted Testing, Computer-Assisted Instruction and Computer-Based Education; and introduces
the analysis, design, and development of the Computer-Simulated Experiment Testing System (CSET). The
simulation of electrochemistry experiment is being described with the CSET theory. With the intelligent
processing mechanism and web-based multimedia platform, the CSET theory can be the important part of
Computer-Based Education.
1 INTRODUCTION
With the further development of instructional
content and course system, the Computer-Based
Education (CBE) has become the principal direction
and important method (Xia, 2000). Intelligent CBE
system based on multi-media and hyper-media
comes into existence as the situation requires.
Meanwhile, the Computer Assisted System
subdivides to Computer Assisted Instruction and
Computer Management Instruction in the process of
instruction activities. We can see a good application
in U.S. patent. The invention assesses the students’
progress, and selects appropriate lessons for the
student at the time (Seifert, 1999).
Based on the specific content, there are two
major phases in the process of instructional activity.
One is implementation of instruction, namely how to
help educates to learn. Another is assessment of
instruction, namely how to evaluate knowledge
mastery. This is usually called the process of
experiment testing (Xia et al., 2003).
Assessment is an essential aspect of all
instruction and Interest in Computer Assisted
Assessment is growing rapidly. It is increasingly
accepted that assessment is the engine that drives a
great deal of students' learning. With the wave to
widen participation in higher and further education,
combined with reductions in the actual amount of
resource per student, it is impossible to extend
traditional assessment processes, practices and
instruments to meet demand (Brown et al., 1999).
What and how well students have learned and so
do students themselves. Assessments can take the
form of a quiz or examination to test students’
learning achievements or a questionnaire to
investigate students' attitudes and reactions to new
instructional courseware (Chou, 2000).
A key subject of experimental testing is to make
full use of the simulation function and management
function, which is highly valued by education with
electrical audiovisual aids and experiment instructor.
2 COMPUTER SIMULATED
EXPERIMENT TESTING
2.1 Development of CSET System
The experiment testing as the important parts of
instructional activity emphasizes examining the
manipulation of equipments and operating
specifications. On the basic requirements, the
purpose of developing CSET system is listed below.
2.1.1 Maintain Impartiality and Objectivity
of Scoring
Because of strict environment of experiment testing,
it is difficult to give the accurate scoring. There are
two kinds of factors. One is human factor. Usually
271
Yang B., Johnson S. and Hsu D. (2009).
CONSTRUCTING A COMPUTER SIMULATED EXPERIMENT TESTING SYSTEM.
In Proceedings of the International Conference on Knowledge Management and Information Sharing, pages 271-275
DOI: 10.5220/0002289702710275
Copyright
c
 SciTePress

several examinees are invigilated by one instructor
through their whole process of experiment. Only
those errors being observed are effective and can be
scored. So the experiment testing varies different
evaluation standard from different instructor and
different time. Another is non-human factor. It
mainly reflects the mismatch between equipments
and examinees. There are two generalized solutions,
to supply the randomness of testing, to maintain the
consistency of testing, in which examinees are
divided into groups.
Considering the above factors, examinee can be
put on the same platform and at the same starting
point by using networking computer to assist
experiment testing. All those make sure the
impartiality and objectivity of testing.
2.1.2 Extend Choosing Range and Reduce
Equipment Cost
Considering the limited environment, there're few
experiments can be chosen because of slow reaction
process and complicated manipulation. Moreover,
the cost also affects on the range that the experiment
to be chosen. The equipments are used frequently
and repeatedly in a short time so that it not becomes
less accurate and shorter operational life span, but
leads to environmental problems. Therefore it is
safer, more efficient and less-wasted to interact with
examinees by using real-time simulation to score
and have error-analysis process (Xia et al., 2003).
2.2 Rationale of CSET Theory
Computer Simulated Experiment Testing is a new
kind of computer-based education theory (Xia,
2000). It's on the basis of specific experiment testing
process (as Figure 1 shows) to raise the pattern of
computer-involved testing. The details can be read
for reference (Xia et al., 2002).
Figure 1: Basic Flow of Experiment Testing.
A good test will not only help the instructor to
evaluate students’ learning status, but also facilitate
the diagnosis of the problems embedded in the
students' learning process (Hwang et al., 2005).
2.2.1 Preparation and Generating of Testing
This phase mainly uses existing management pattern
of item pool for reference, which is divided into
three parts (as Figure 2 shows). First, build the
model of experiment testing. Second, keep the
maintenance of item pool. Third, generate the
contents.
Figure 2: Preparation and Generating of Testing Content.
2.2.2 Implementation of Experiment Testing
This is the phase with which the users directly
interact to the system. It mainly includes
process-simulated management and real-time
monitoring management (as Figure 3 shows). The
process-simulated management reflects all segments
and running status of testing to simulate the whole
process. The real-time monitoring management is
mainly to record the data path of programs and to
record the contents of assisted-services.
Reference (Shavelson, 1992) further suggests
using computer simulations for hands-on
performance assessment. In their project “Electric
Mysteries,” students were required to replicate
electric circuits by manipulating icons of batteries,
bulbs, and wires.
Figure 3: Implementation of Experiment Testing.
2.2.3 Analysis of Testing Results
After testing, the management reflects two aspects.
One is to be judged by comparing with the standard
library. Another is to get information and conclude
the records to maintain and modify the testing
model. This method can reduce the error rate and
improve the reliability.
Over the past several decades, a wider range of
assessment strategies has gained prominence in
KMIS 2009 - International Conference on Knowledge Management and Information Sharing
272

classrooms, including complex assessment items
such as individual or group projects, student journals
and other creative writing tasks, graphic/artistic
representations of knowledge, clinical interviews,
student presentations and performances, and so on
(Ramakishnan and Ramadoss
, 2009).
Based on above discussion, the CSET theory
shows the special advantage of computer-assisted
system by compromising the merits of simulation
and dynamic strategy control.
2.3 CAI, CAT and CSET
All kinds of courses can be provided by the CAI
system, which is based on knowledge-imparting.
Comparatively speaking, the CSET system is limited
in some specific field. But it still need resources and
material to computerize those experimental testing.
In the field of CAI, the computer-assisted testing
theory is the first one to be introduced into
educational testing. The CAT theory has a complete
set of summary on guiding testing process, and it has
been proved to have powerful testing functions. But
it is limited to objectivity testing. The questionnaire
is usually composed of choice question, true-false
question and matching question, which makes it
ineffective and inefficient to test manipulation and
experiment. And, the experiment testing requires not
only omnibearing environment and process
simulated functions but also the functions of
collecting effective data and manipulations. We can't
get all of these from the CAT system.
The CSET system uses three kinds of
instructional pattern: one is the manipulation and
exercise; the other is the tutorial instruction; another
is the simulation.
According to the process of manipulation and
exercise pattern, users get real-time results of the
adaptive questions by using networking computers.
This pattern is not just for imparting knowledge, but
for strengthening the learnt knowledge by the large
amounts of questions or exercises.
The CGI programming will be used to control
presentation by the adaptive questionnaires. The
adaptive questioning uses the answers to some
certain questions to determine the next series of
questions and skip unrelated questions (Chou, 2000).
Although the experiment testing belongs to the
field of instructional assessment, there are lots of
commons in the references of those CAI patterns,
such as the command of experiment by the pattern of
manipulation and exercise, the hands-on ability by
the pattern of simulation. So the three instructional
patterns of the CAI theory have definite referential
effect on experiment testing.
3 THE CSET CASE
The CSET system is being designed to test students’
learning achievements and evaluate course under the
networking learning environment. Both in concept
and construction, the CSET project is intended to
integrate three major components: system,
evaluation, and interface in its analysis, design, and
development phases.
3.1 Analysis Phase
To show the advantage of CSET theory, we use the
software engineering to develop the specific CSET
system which is based on electrochemistry
experiment "Measurement of Electrode Potential".
This experiment is mainly used to measure an
unknown side of electrode. The basic steps are listed
below. First, use saturated KQ solution as
electrolytic bridge to make the battery under test.
Second, connect test circuit according to
compensation method. Finally, adjust the
potentiometer to read scale and we get the
electromotive force of battery under test.
This experiment requires examinees to prepare
in three phases.
1. The phase of preparation. It emphasizes the
usage and principle of compensation method. The
examinees are required to be capable of adjusting
electromotive force by using potentiometer and
formula. (t as ℃)
2. The phase of manipulation. It emphasizes
operating-specifications, note-taking of experimental
data and the capability of using galvanometer.
3. The phase of follow-up. It emphasizes on
using the function to calculate the standard
electromotive force and find the difference with
referential data.
According to the basic steps, we have
requirement of practical experiment testing. First,
understand the conventional manipulation well and
check the equipments before experiment. Second, be
familiar with the specific steps and all the
equipments listed. Third, be familiar with the
method of data process and evaluate based on
experiment value and provided value.
To resolve the problem of flexibility between
structural programming and system flow, the scheme
which sets down the sequence of range and ignores
the flow sequence in the range has been proposed.
1. The fundamental analysis. Based on the
requirement of different knowledge, the system is
divided into four levels (as Figure 4 shows).
2. The assignment analysis. Based on the
different content in the assignment, the system is
CONSTRUCTING A COMPUTER SIMULATED EXPERIMENT TESTING SYSTEM
273

divided into the design of flow, the simulation of
manipulation, data process, score process etc (as
Figure 5 shows).
3. The flow analysis. The fundamental range
sequence is determined by the basic flow of system.
But the specific manipulation flow inside the range
is concealed. The Figure 6 shows the basic flow.
Figure 4: Fundamental Analysis of CSET System.
Figure 5: Assignment Analysis of CSET System.
Figure 6: Flow Analysis of CSET System.
3.2 Design Phase
The system contains two parts, one is the
administration module and the other is the
simulation module. In the design of administration,
the flow is divided into five steps, namely the
preparation of experiment, the connection of
equipment, the manipulation, the data process and
the finishing off. So it's easy to be monitored in
sections during the runtime. In the design of
simulation, these specific equipments used in the
experiment are displayed in the same interface (we
call the virtual equipment), such as potentiometer,
galvanometer, battery under test, working power
supply, standard cell etc. Those virtual equipments
can describe the dynamic variation.
Because of the fixed range sequence and ignored
inner-range flow, the evaluation adopts the method
of setting flags indirectly, recording when meeting
flags and scoring after experiment. Fixed range
sequence means that the CSET system will check
precondition to move into next phase. For example,
in the phase of preparation only if the examinee gets
the positive and negative charges correct, then they
can move to the phase of pre-check. Meanwhile,
when they get all equipments checked, then the
phase of preparation is finished. Otherwise, the
system will deduct corresponding points. Ignored
inner-range flow means that the manipulation is not
restricted with the sequence (except the experiment
requires). When the examinee finishes all the
required manipulation, then he can move to next
phase. Take the example mentioned above. As to
variable resistance and switch, their checking
manipulation would not affect other equipments. So
we can ignore the specific sequence within the range
and only pay attention to the complete of flags. In
the phase of evaluation, we can get the score
according to those required flags and backtrack the
error easily.
The subject of test administration will discuss
obtaining examinee responses scoring them,
recording them for later use, reporting and
interpreting the results and giving prescriptive
advice (Bunderson et al., 1988).
3.3 Development Phase
The main frame is the specific platform for
examinee to get testing. The development of CSET
involved the specification which is to fulfil the
production requirements identified in the analysis
and design phase.
1. The design of menu and confirmation key.
Adopt one-level menu and hotkey to realize the
control of five phases. It helps examinee to know
which phase they are. In that phase, we’ve set a
Confirm flag. This flag is not only good for
manipulation but also good for system control.
2. The design of standard cell, battery under test
and external battery. In this experiment, these
equipments are only referring to the process of
connection and power-off.
3. The design of connection frame. We set a
pop-up window to guide examinee to manipulate
with the given information.
4. The design of galvanometer. We put three
buttons on it. They are 'OFF', 'ON', and 'RESET'.
KMIS 2009 - International Conference on Knowledge Management and Information Sharing
274

There is a visual window of this rectangle equipment
as well, namely cursor observing window. The range
and the way cursor moves is based on the runtime
experimental data.
5. The design of potentiometer. In the practical
experiment, the potentiometer includes many
functions, such as the calibrating input of standard
cell, the protection key of galvanometer, the
adjusting of variable resistance, and the replacement
of switches. According to the testing requirements,
these functions must be kept in the environment of
simulation.
a. The design of external terminal. On the side
close to connecting device, the ports of the
galvanometer, standard cell, battery under test and
external battery are listed in proper order.
b. The design of knob of potentiometer. Because
it's difficult to manipulate with knob in the
simulation, we replace it with button '+' and '-'.
Meanwhile, we can read the runtime value from the
display.
c. The design of electromotive force of standard
cell. Provide the box to input the temperature
calibrating value and determine the value to the
second decimal place.
d. The design of variable resistance. Put a
'RESET' button to have zero check and put 'coarse
adjustment', 'fine adjustment', and 'trimming'. We
can use direction key to switch the level and read the
value from the display.
e. The design of switch. Based on the practical
requirements, we put 'OFF', 'Standard' and 'Under
test'.
f. The design of protection key of galvanometer.
We put 'short circuit', 'small end', 'large end' and
provide a time parameter to maintain galvanometer.
4 CONCLUSIONS
Computer-Based Education (CBE) has positive
effects on students’ attitude. Results indicate that
computer-assisted and managed instruction and CBE
used with disadvantaged students was generally
effective (Bangert-Drowns et al., 1985). Also the
application of CSET theory solves the common
problem which other patterns are facing in the field
of Computer-based education. The CSET system
realizes the objective to test hands-on ability and
simulate all kinds of practical experiments.
With the improvement of decision-supported
system, expert system and knowledge base, the
guiding of CSET theory will be strengthened,
especially in the process of judging and evaluation.
It will improve the stability and validity of the
system when using the multi-database system. In the
near future, as the network technique gets more
applications, multimedia technique develops further,
the web application of CBE will improve quickly.
And we can see, the CSET system will be the
important part of computer-based education with the
intelligent processing mechanism and web-based
multimedia platform.
REFERENCES
Bangert-Drowns, R. L., Kulik, J. A. and Kulik, C.-L. C.,
1985. Effectiveness of computer-based education in
secondary schools. Journal of Computer-Based
Instruction. 12(3):59-68.
Brown, S., Race, P. and Bull J., 1999. Computer-assisted
assessment in higher education. Kogan Page, London.
Bunderson, C. V., Inouye, D. K., and Olsen, J. B., 1988.
The four generations of computerized educational
measurement. In Educational Measurement.
Macmillan. New York. 3
rd
edition.
Chou, C. 2000. Constructing a computer-assisted testing
and evaluation system on the World Wide Web-the
CATES experience. IEEE Transactions on Education.
43(3):266-272.
Hwang, G.-J., B.M.T., L., Tseng, H.-H. and Lin, T.-L.,
2005. On the development of a computer-assisted
testing system with genetic test sheet-generating
approach. IEEE transactions on systems, man and
cybernetics. Part C, Applications and reviews.
35(4):590-594.
Ramakishnan, S. B. and Ramadoss, B., 2009. Assessment
using multi-criteria decision approach for "higher
order skills" learning domains. International Journal
on E-Learning. 8(2):241-262.
Seifert, D. M., 1999. Automated lesson selection and
examination in computer-assisted education. United
States Patent 5904485. http://www.wikipatents.com
/5904485.html.
Shavelson, R. J., Baxter, G. P. and Pine, J., 1992.
Performance assessments: political rhetoric and
measurement reality. Educational Researcher.
21(4):22-27.
Xia, J.-X., 2000. Computer assisted experiment testing
system. Computer Applications and Software.
17(10):57-64.
Xia, J.-X., Lu, J.-K. and Shi, Z.-X., 2002. Application of
CAT, CAI and CSET in experimental tests. Computer
Applications and Software. 19(1):29-34.
Xia, J.-X., Xu, J., and Wu, G.-F., 2003. The application
and research of CSET. Computer Applications and
Software. 20(5):66-68.
CONSTRUCTING A COMPUTER SIMULATED EXPERIMENT TESTING SYSTEM
275