A PROPOSED METHOD FOR GENERATING QUESTION TESTS
BASED ON PRESCRIPTION DRUG NAME SIMILARITY
Keita Nabeta, Hirotsugu Ishida, Masaomi Kimura, Michiko Ohkura
Shibaura Institute of Technology, Toyosu, 3-7-5 Koto-ku, Tokyo, Japan
Fumito Tsuchiya
International University of Health and Welfare, Kitakanemaru 2600-1, Ohtawara City, Tochigi, Japan
Keywords:
Medical safety, Medical education, Drug information, Prescription drug name.
Abstract:
An educational program aimed at orienting medical staff on proper prescription drug use needs to be im-
plemented to avoid medical errors. Presently, pharmacists are guided by information provided in package
inserts. However, these inserts are not suitable educational materials because their descriptions are usually
very complex. A huge effort is needed to create educational materials for each of the 20,000 prescription
drugs currently used in Japan. Therefore, it is necessary to develop a learning support system with functions
that can generate educational materials automatically from a drug information database. Here, we propose a
method for generating multiple-choice tests that allows students to associate brand and generic drug names
based on similarity.
1 INTRODUCTION
The Japan Council for Quality Health Care has gath-
ered many near miss cases which have been reported
by pharmacists, including confusing prescription drug
amount, standard unit, dosage format, as well as
other mix-ups (Japan Council for Quality Health
Care, 2010). Studies indicate that many pharmacists
consider these errors to arise from a lack of medi-
cal knowledge, technique, and education. Further-
more, these errors are caused by medical staff lacking
the necessary medical knowledge, and include phar-
maceutical students, new pharmacists, and returning
pharmacists. Thus, proper education is necessary to
ensure medical safety. The Ministry of Health, Labor,
and Welfare has set up a specific agenda regarding the
practical training of pharmaceutical students (Health,
Labour and Welfare Ministry, 2007). In this report,
students were required to have knowledge and proper
prescription drug technique.
Prescription drug information is commonly de-
tailed in package inserts, and includes information
related to dosage, efficacy, and cautions. By law,
pharmaceutical companies are obligated to append
package inserts on all drugs. While pharmacists ob-
tain information about a drug from package inserts,
these documents are not suitable educational mate-
rials since descriptions are generally complex. Yet,
generating educational materials for each drug be-
comes an enormous task as there are over 20,000
drugs presently used in Japan. The number of drugs is
expected to increase as the number of recommended
generic drugs increases. Therefore, it is necessary to
develop a system whereby educational materials are
generated automatically. Figure 1 shows the config-
uration of proposed system. This system generates
the educational materials based on drug information,
learning records and near-miss cases. Learners gain
the knowledge concerning drugs from them.
Figure 1: Learning supporting system for pharmacists.
449
Nabeta K., Ishida H., Kimura M., Ohkura M. and Tsuchiya F..
A PROPOSED METHOD FOR GENERATING QUESTION TESTS BASED ON PRESCRIPTION DRUG NAME SIMILARITY.
DOI: 10.5220/0003401404490452
In Proceedings of the 3rd International Conference on Computer Supported Education (CSEDU-2011), pages 449-452
ISBN: 978-989-8425-49-2
Copyright
c
2011 SCITEPRESS (Science and Technology Publications, Lda.)
As part of the educational materials used for this
method, we focused on brand and generic name as-
sociations, with generic names categorized by phar-
maceutical company and brand names by active in-
gredient. Although package inserts typically go by
brand name, pharmacy students learn drug informa-
tion by generic name. Therefore, to associate a brand
name with a particular piece of information, pharma-
cists must first learn to associate brand names with
generic names. In this report, we propose a method
for generating multiple-choice tests which facilitate
brand and generic name associations.
With regard to multiple-choice tests, question dif-
ficulty often depends on answer choice similarities.
Likewise, the method proposed here focuses on pre-
scription drug name similarity. Given that such simi-
larity creates confusion, students often struggle to find
the correct answer in multiple-choice tests. Further-
more, some brand names are derived from generic
names. These drug names, however, are easily iden-
tified if presented to students in a question format.
Thus, question difficulty can be controlled by using
prescription drug name similarity. To assess the re-
lationship between question difficulty and drug name
similarity, we experimented with questions generated
automatically by a computer, and evaluated the pro-
posed method by analyzing input logs obtained from
each learning activity.
2 GENERATION OF
PRESCRIPTION DRUG NAME
TEST
In this report, we used data previously presented
by Tsuchiya (F. Tsuchiya, 2008). Target prescrip-
tion drugs included 15,014 brand names and 1,402
generic names, all consisting of a single active in-
gredient. We notice that the medicine names are pre-
sented by Japanese characters although we described
the medicine names by alphabet in this paper.
A multiple-choice test was generated as shown in
Figure 2. The test consisted of a computer generated
question and brand names were provided as answer
choices. Only one brand name corresponded to the
active ingredient (i.e., generic name) in the question.
Thus, each question had only one possible answer,
and students were permitted to move on to the next
question if they could not answer it.
Figure 2: Computer generated sample question.
2.1 Selection of Prescription Drug
Names based on Similarity
Question difficulty in multiple-choice tests is usually
based on answer choice similarities. Likewise, for
the method devised here, question difficulty was con-
trolled by brand and generic name similarity. The
similarities between four prescription drug names are
shown in Figure 3.
Student’s level of understanding can be gauged
with questions that rely on drug name similarity. In
the study presented here, if knowledge regarding a
drug’s name was lacking, students tended to choose
the wrong answer given its similarity to the correct
answer choice.
Figure 3: Brand and generic name similarities.
Incorrect Choice based on Brand Name Similar-
ity (S
1
)
Students tend to select incorrect answers particu-
larly if the choices are similar to each other. With
this in mind, we devised a question-based method
using name similarity between correct and incor-
rect answer choices.
Correct Choice based on Brand and Generic
Name Similarity (S
2
)
Some brand names are derived from a generic
name. For instance, the brand name, ‘CLOZARIL
Tablet 25 mg, is derived from its generic name,
‘Clozapine. With these types of drug names, stu-
dents had no difficulty associating the brand name
with the generic name. In other words, test ques-
CSEDU 2011 - 3rd International Conference on Computer Supported Education
450
tions were easily answered if the prescription drug
had its brand name derivedfrom the generic name.
Questions based on such similarities were easier
for students compared to questions based on other
similarities.
Incorrect Choice based on Brand and Generic
Name Similarity (S
3
)
In this study, there were fewer cases where the
generic name of a particular drug was similar to
the brand name of another drug. With such ques-
tion types, students tended to select the incorrect
brand name. Thus, the more similar a generic
name was to the brand name of another drug, the
harder the question. In this report, questions based
on this type of similarity were effective and pro-
vided the next level of complexity for questions
exemplified by S
2
.
Incorrect Choice based on Generic Name Simi-
larity (S
4
)
Answers were difficult to identify if the cor-
responding generic names were similar to the
generic name presented in the question. There-
fore, students tended to select the wrong answer
because the generic name and the generic name
they associated with a particular brand name were
similar.
2.2 Method for Measuring Name
Similarity
Brand names consist of three parts: stem, dosage for-
mat, and standard unit. For example, Amaryl 1 mg
tablet’: the stem corresponds to Amaryl’, ‘1mg’ to
the standard unit which expresses the active ingredi-
ent content, and ‘tablet’ denotes the dosage format.
Similarity of brand name parts was considered be-
cause pharmacists typically use them to identify a pre-
scription drug. In this method, brand name stems
were used to measure similarity. To generate brand
name stems, standard unit and dosage format was re-
moved according to the method proposed by Kimura
et al (M. Kimura, K. Nabeta, F. Tsuchiya, 2010).
In addition, an edit distance algorithm, commonly
used to compute character sequence similarities, was
employed to measure drug name similarity. Similar-
ity was defined as the number of times required to
change characters by insertion and deletion. Values
were normalized and subtracted from 1. If the value
was equal to one, the string comparison was said to
be identical; however, if the value was closer to 0, the
string comparison was unrelated.
2.3 Process Generation
The process used to generate questions based on drug
name similarity was as follows: 1) instructors input
eight parameters that are regarded as maximum and
minimum values for each similarity (i.e., S
1
, S
2
, S
3
,
and S
4
); 2) the computer randomly selects a prescrip-
tion drug name from a database that best matches
a condition, and the drug name is not equal to the
generic name and min
2
< S
2
< max
2
; 3) the com-
puter randomly selects three drugs that best match a
condition, and the brand names are not equal to each
other and min
1
< S
1
< max
1
, min
3
< S
3
< max
3
, and
min
4
< S
4
< max
4
; and 4) the computer generates a
question based on the generic name and template in-
put, creates brand name choices, and outputs the ques-
tion in an HTML file format.
3 EXPERIMENT
Association between question difficulty and name
similarity was assessed in order to evaluate the va-
lidity of the proposed method. Generated questions
were used in an experiment where the ratio between a
correct answer and answer time was determined. To
evaluate the method properly, an experiment was con-
ducted on 12 students in their twenties who lacked
pharmaceutical knowledge and attended the Depart-
ment of Engineering.
Eight similarity parameters were used in the ex-
periment and three cases were generated as shown in
Table 1. In Case 1, brand names in the choices are
similar to each other (S
1
). In Case 2, the generic name
is similar to the brand name of the answer choice (S
2
).
In Case 3, the generic name is similar to the brand
name of an incorrect answer choice (S
3
). Similarities
corresponding to S
4
, which rely on proper knowledge
of generic names, were not included in this experi-
ment given the participant’s background. Finally, five
questions were prepared for each of the three cases
presented.
Table 1: Question threshold patterns.
S
1
S
2
S
3
S
4
Case 1 0.7-1.0 0.0-0.5 0.0-0.5 0.0-0.5
Case 2 0.0-0.5 0.7-1.0 0.0-0.5 0.0-0.5
Case 3 0.0-0.5 0.0-0.5 0.7-1.0 0.0-0.5
During the experimental process, participants
were shown a table of generic and brand names and
given 150 seconds to familiarize themselves with
the content. They were then asked to answer 15
computer-generated questions. Answer time, defined
A PROPOSED METHOD FOR GENERATING QUESTION TESTS BASED ON PRESCRIPTION DRUG NAME
SIMILARITY
451
as the response time from question presentation to an-
swer selection, was measured.
4 RESULTS AND DISCUSSION
4.1 Ratios of Correct Answers and
Answer Times
Table 2 showsthe average ratio of correct answers and
average answer times determined for each of the three
cases in the experiment. Results indicate that Case 2
had the highest ratio of correct answers with the short-
est answer times. As expected, Case 2 represented the
easiest question type.
Table 2: Experimental results.
Average ratio of Average answer
correct answers (%) time (sec)
Case 1 70.0 10.6
Case 2 85.1 7.13
Case 3 58.0 8.72
4.2 Participant Perspectives
A similarity score of 0.83 was obtained between
the generic name, ‘Bendazac’, and the brand name
stem, ‘Zibensak’. This score corresponded to ques-
tion nine, which was answered correctly by all par-
ticipants in the experiment. Moreover, this was the
highest score obtained for all questions represented by
Case 2. In contrast, question 15, which presented the
generic drug Sofalcone’, was missed by 10 partici-
pants. In this question, six participants answered ‘So-
farin Tablet 25 mg’, three answered ‘Phardine Tablet
200 mg’, and one answered ‘Falken Tape 20 mg’. The
similarity score obtained between the incorrect brand
and generic name was 0.73. Participants tended to
select choices based on dosage format and standard
unit, which were not taken into consideration while
computing similarity. In addition, most participants
did not select ‘Falken tape 20 mg’, since ‘tape’ is an
uncommon dosage format. This result suggests that
participants memorized certain brand names by fo-
cusing on the dosage format or standard unit of the
brand name drug. In a real situation, however, phar-
macists do not rely on dosage format and standard
unit only, but check brand name as well. Given the
importance of associating generic names with brand
name stems, the method presented here will facilitate
student learning.
5 CONCLUSIONS
In this report, we proposea multiple-choicetest which
facilitates generic and brand name associations. The
test can be implemented as a prototypic learning sup-
port system targeting pharmacists. In this study, ques-
tions were generated based on prescription drug name
similarities. Correct answers were easily identified
for questions relating to generic names with simi-
lar brand names. However, participants tended to
miss the question if answer choices had similar brand
names or if the generic name was similar to an incor-
rect answer choice.
To evaluate the method presented here, we con-
ducted a question-based experiment on students lack-
ing pharmaceutical knowledge, and determined ratios
of correct answers and answer times. The highest
ratios of correct answers and shortest answer times
were obtained for questions having similar generic
and brand names. However, in questions with an-
swer choices having similar brand names or where
a generic name was similar to an incorrect answer
choice, the ratios of correct answers were low and
answer times were long. In conclusion, our results
suggest that question difficulty can be controlled by
focusing on prescription drug name similarities.
Although drug name similarity was the primary
focus of the method presented here, additional fea-
tures regarding semantic drug information (e.g., ef-
ficacy) need to be incorporated in order to generate
prescription drug name tests. Furthermore, additional
methods are yet to be established which focus on con-
traindication, efficacy, and dosage. Thus, it is im-
portant to develop a general method which facilitates
adequate prescription drug information and promotes
student learning. Finally, effective learning and med-
ical safety must be evaluated.
REFERENCES
F. Tsuchiya (2008). The item and standard of drug dictio-
nary (ICHM5). Report of Health and Labour Sciences
Research.
Health, Labour and Welfare Ministry (2007). Process of
practical training for pharmaceutical students.
Japan Council for Quality Health Care (2010). Analysis of
near-miss cases in pharmacies 2010 annual report.
M. Kimura, K. Nabeta, F. Tsuchiya (2010). The standard-
ization of medicine name structures suitable for a pre-
scription entry system. In Proceedings of AHFEI2010.
CSEDU 2011 - 3rd International Conference on Computer Supported Education
452