A PROPOSED METHOD FOR GENERATING QUESTION TESTS

BASED ON PRESCRIPTION DRUG NAME SIMILARITY

Keita Nabeta, Hirotsugu Ishida, Masaomi Kimura, Michiko Ohkura

Shibaura Institute of Technology, Toyosu, 3-7-5 Koto-ku, Tokyo, Japan

Fumito Tsuchiya

International University of Health and Welfare, Kitakanemaru 2600-1, Ohtawara City, Tochigi, Japan

Keywords:

Medical safety, Medical education, Drug information, Prescription drug name.

Abstract:

An educational program aimed at orienting medical staff on proper prescription drug use needs to be im-

plemented to avoid medical errors. Presently, pharmacists are guided by information provided in package

inserts. However, these inserts are not suitable educational materials because their descriptions are usually

very complex. A huge effort is needed to create educational materials for each of the 20,000 prescription

drugs currently used in Japan. Therefore, it is necessary to develop a learning support system with functions

that can generate educational materials automatically from a drug information database. Here, we propose a

method for generating multiple-choice tests that allows students to associate brand and generic drug names

based on similarity.

1 INTRODUCTION

The Japan Council for Quality Health Care has gath-

ered many near miss cases which have been reported

by pharmacists, including confusing prescription drug

amount, standard unit, dosage format, as well as

other mix-ups (Japan Council for Quality Health

Care, 2010). Studies indicate that many pharmacists

consider these errors to arise from a lack of medi-

cal knowledge, technique, and education. Further-

more, these errors are caused by medical staff lacking

the necessary medical knowledge, and include phar-

maceutical students, new pharmacists, and returning

pharmacists. Thus, proper education is necessary to

ensure medical safety. The Ministry of Health, Labor,

and Welfare has set up a speciﬁc agenda regarding the

practical training of pharmaceutical students (Health,

Labour and Welfare Ministry, 2007). In this report,

students were required to have knowledge and proper

prescription drug technique.

Prescription drug information is commonly de-

tailed in package inserts, and includes information

related to dosage, efﬁcacy, and cautions. By law,

pharmaceutical companies are obligated to append

package inserts on all drugs. While pharmacists ob-

tain information about a drug from package inserts,

these documents are not suitable educational mate-

rials since descriptions are generally complex. Yet,

generating educational materials for each drug be-

comes an enormous task as there are over 20,000

drugs presently used in Japan. The number of drugs is

expected to increase as the number of recommended

generic drugs increases. Therefore, it is necessary to

develop a system whereby educational materials are

generated automatically. Figure 1 shows the conﬁg-

uration of proposed system. This system generates

the educational materials based on drug information,

learning records and near-miss cases. Learners gain

the knowledge concerning drugs from them.

Figure 1: Learning supporting system for pharmacists.

449

Nabeta K., Ishida H., Kimura M., Ohkura M. and Tsuchiya F..

A PROPOSED METHOD FOR GENERATING QUESTION TESTS BASED ON PRESCRIPTION DRUG NAME SIMILARITY.

DOI: 10.5220/0003401404490452

In Proceedings of the 3rd International Conference on Computer Supported Education (CSEDU-2011), pages 449-452

ISBN: 978-989-8425-49-2

 2011 SCITEPRESS (Science and Technology Publications, Lda.)

As part of the educational materials used for this

method, we focused on brand and generic name as-

sociations, with generic names categorized by phar-

maceutical company and brand names by active in-

gredient. Although package inserts typically go by

brand name, pharmacy students learn drug informa-

tion by generic name. Therefore, to associate a brand

name with a particular piece of information, pharma-

cists must ﬁrst learn to associate brand names with

generic names. In this report, we propose a method

for generating multiple-choice tests which facilitate

brand and generic name associations.

With regard to multiple-choice tests, question dif-

ﬁculty often depends on answer choice similarities.

Likewise, the method proposed here focuses on pre-

scription drug name similarity. Given that such simi-

larity creates confusion, students often struggle to ﬁnd

the correct answer in multiple-choice tests. Further-

more, some brand names are derived from generic

names. These drug names, however, are easily iden-

tiﬁed if presented to students in a question format.

Thus, question difﬁculty can be controlled by using

prescription drug name similarity. To assess the re-

lationship between question difﬁculty and drug name

similarity, we experimented with questions generated

automatically by a computer, and evaluated the pro-

posed method by analyzing input logs obtained from

each learning activity.

2 GENERATION OF

PRESCRIPTION DRUG NAME

TEST

In this report, we used data previously presented

by Tsuchiya (F. Tsuchiya, 2008). Target prescrip-

tion drugs included 15,014 brand names and 1,402

generic names, all consisting of a single active in-

gredient. We notice that the medicine names are pre-

sented by Japanese characters although we described

the medicine names by alphabet in this paper.

A multiple-choice test was generated as shown in

Figure 2. The test consisted of a computer generated

question and brand names were provided as answer

choices. Only one brand name corresponded to the

active ingredient (i.e., generic name) in the question.

Thus, each question had only one possible answer,

and students were permitted to move on to the next

question if they could not answer it.

Figure 2: Computer generated sample question.

2.1 Selection of Prescription Drug

Names based on Similarity

Question difﬁculty in multiple-choice tests is usually

based on answer choice similarities. Likewise, for

the method devised here, question difﬁculty was con-

trolled by brand and generic name similarity. The

similarities between four prescription drug names are

shown in Figure 3.

Student’s level of understanding can be gauged

with questions that rely on drug name similarity. In

the study presented here, if knowledge regarding a

drug’s name was lacking, students tended to choose

the wrong answer given its similarity to the correct

answer choice.

Figure 3: Brand and generic name similarities.

• Incorrect Choice based on Brand Name Similar-

ity (S

)

Students tend to select incorrect answers particu-

larly if the choices are similar to each other. With

this in mind, we devised a question-based method

using name similarity between correct and incor-

rect answer choices.

• Correct Choice based on Brand and Generic

Name Similarity (S

)

Some brand names are derived from a generic

name. For instance, the brand name, ‘CLOZARIL

Tablet 25 mg,’ is derived from its generic name,

‘Clozapine.’ With these types of drug names, stu-

dents had no difﬁculty associating the brand name

with the generic name. In other words, test ques-

CSEDU 2011 - 3rd International Conference on Computer Supported Education

450

tions were easily answered if the prescription drug

had its brand name derivedfrom the generic name.

Questions based on such similarities were easier

for students compared to questions based on other

similarities.

• Incorrect Choice based on Brand and Generic

Name Similarity (S

)

In this study, there were fewer cases where the

generic name of a particular drug was similar to

the brand name of another drug. With such ques-

tion types, students tended to select the incorrect

brand name. Thus, the more similar a generic

name was to the brand name of another drug, the

harder the question. In this report, questions based

on this type of similarity were effective and pro-

vided the next level of complexity for questions

exempliﬁed by S

• Incorrect Choice based on Generic Name Simi-

larity (S

)

Answers were difﬁcult to identify if the cor-

responding generic names were similar to the

generic name presented in the question. There-

fore, students tended to select the wrong answer

because the generic name and the generic name

they associated with a particular brand name were

similar.

2.2 Method for Measuring Name

Similarity

Brand names consist of three parts: stem, dosage for-

mat, and standard unit. For example, ‘Amaryl 1 mg

tablet’: the stem corresponds to ‘Amaryl’, ‘1mg’ to

the standard unit which expresses the active ingredi-

ent content, and ‘tablet’ denotes the dosage format.

Similarity of brand name parts was considered be-

cause pharmacists typically use them to identify a pre-

scription drug. In this method, brand name stems

were used to measure similarity. To generate brand

name stems, standard unit and dosage format was re-

moved according to the method proposed by Kimura

et al (M. Kimura, K. Nabeta, F. Tsuchiya, 2010).

In addition, an edit distance algorithm, commonly

used to compute character sequence similarities, was

employed to measure drug name similarity. Similar-

ity was deﬁned as the number of times required to

change characters by insertion and deletion. Values

were normalized and subtracted from 1. If the value

was equal to one, the string comparison was said to

be identical; however, if the value was closer to 0, the

string comparison was unrelated.

2.3 Process Generation

The process used to generate questions based on drug

name similarity was as follows: 1) instructors input

eight parameters that are regarded as maximum and

minimum values for each similarity (i.e., S

, S

and S

); 2) the computer randomly selects a prescrip-

tion drug name from a database that best matches

a condition, and the drug name is not equal to the

generic name and min

< S

< max

; 3) the com-

puter randomly selects three drugs that best match a

condition, and the brand names are not equal to each

other and min

< S

< max

, min

< S

< max

, and

min

< S

< max

; and 4) the computer generates a

question based on the generic name and template in-

put, creates brand name choices, and outputs the ques-

tion in an HTML ﬁle format.

3 EXPERIMENT

Association between question difﬁculty and name

similarity was assessed in order to evaluate the va-

lidity of the proposed method. Generated questions

were used in an experiment where the ratio between a

correct answer and answer time was determined. To

evaluate the method properly, an experiment was con-

ducted on 12 students in their twenties who lacked

pharmaceutical knowledge and attended the Depart-

ment of Engineering.

Eight similarity parameters were used in the ex-

periment and three cases were generated as shown in

Table 1. In Case 1, brand names in the choices are

similar to each other (S

). In Case 2, the generic name

is similar to the brand name of the answer choice (S

In Case 3, the generic name is similar to the brand

name of an incorrect answer choice (S

). Similarities

corresponding to S

, which rely on proper knowledge

of generic names, were not included in this experi-

ment given the participant’s background. Finally, ﬁve

questions were prepared for each of the three cases

presented.

Table 1: Question threshold patterns.

Case 1 0.7-1.0 0.0-0.5 0.0-0.5 0.0-0.5

Case 2 0.0-0.5 0.7-1.0 0.0-0.5 0.0-0.5

Case 3 0.0-0.5 0.0-0.5 0.7-1.0 0.0-0.5

During the experimental process, participants

were shown a table of generic and brand names and

given 150 seconds to familiarize themselves with

the content. They were then asked to answer 15

computer-generated questions. Answer time, deﬁned

A PROPOSED METHOD FOR GENERATING QUESTION TESTS BASED ON PRESCRIPTION DRUG NAME

SIMILARITY

451

as the response time from question presentation to an-

swer selection, was measured.

4 RESULTS AND DISCUSSION

4.1 Ratios of Correct Answers and

Answer Times

Table 2 showsthe average ratio of correct answers and

average answer times determined for each of the three

cases in the experiment. Results indicate that Case 2

had the highest ratio of correct answers with the short-

est answer times. As expected, Case 2 represented the

easiest question type.

Table 2: Experimental results.

Average ratio of Average answer

correct answers (%) time (sec)

Case 1 70.0 10.6

Case 2 85.1 7.13

Case 3 58.0 8.72

4.2 Participant Perspectives

A similarity score of 0.83 was obtained between

the generic name, ‘Bendazac’, and the brand name

stem, ‘Zibensak’. This score corresponded to ques-

tion nine, which was answered correctly by all par-

ticipants in the experiment. Moreover, this was the

highest score obtained for all questions represented by

Case 2. In contrast, question 15, which presented the

generic drug ‘Sofalcone’, was missed by 10 partici-

pants. In this question, six participants answered ‘So-

farin Tablet 25 mg’, three answered ‘Phardine Tablet

200 mg’, and one answered ‘Falken Tape 20 mg’. The

similarity score obtained between the incorrect brand

and generic name was 0.73. Participants tended to

select choices based on dosage format and standard

unit, which were not taken into consideration while

computing similarity. In addition, most participants

did not select ‘Falken tape 20 mg’, since ‘tape’ is an

uncommon dosage format. This result suggests that

participants memorized certain brand names by fo-

cusing on the dosage format or standard unit of the

brand name drug. In a real situation, however, phar-

macists do not rely on dosage format and standard

unit only, but check brand name as well. Given the

importance of associating generic names with brand

name stems, the method presented here will facilitate

student learning.

5 CONCLUSIONS

In this report, we proposea multiple-choicetest which

facilitates generic and brand name associations. The

test can be implemented as a prototypic learning sup-

port system targeting pharmacists. In this study, ques-

tions were generated based on prescription drug name

similarities. Correct answers were easily identiﬁed

for questions relating to generic names with simi-

lar brand names. However, participants tended to

miss the question if answer choices had similar brand

names or if the generic name was similar to an incor-

rect answer choice.

To evaluate the method presented here, we con-

ducted a question-based experiment on students lack-

ing pharmaceutical knowledge, and determined ratios

of correct answers and answer times. The highest

ratios of correct answers and shortest answer times

were obtained for questions having similar generic

and brand names. However, in questions with an-

swer choices having similar brand names or where

a generic name was similar to an incorrect answer

choice, the ratios of correct answers were low and

answer times were long. In conclusion, our results

suggest that question difﬁculty can be controlled by

focusing on prescription drug name similarities.

Although drug name similarity was the primary

focus of the method presented here, additional fea-

tures regarding semantic drug information (e.g., ef-

ﬁcacy) need to be incorporated in order to generate

prescription drug name tests. Furthermore, additional

methods are yet to be established which focus on con-

traindication, efﬁcacy, and dosage. Thus, it is im-

portant to develop a general method which facilitates

adequate prescription drug information and promotes

student learning. Finally, effective learning and med-

ical safety must be evaluated.

REFERENCES

F. Tsuchiya (2008). The item and standard of drug dictio-

nary (ICHM5). Report of Health and Labour Sciences

Research.

Health, Labour and Welfare Ministry (2007). Process of

practical training for pharmaceutical students.

Japan Council for Quality Health Care (2010). Analysis of

near-miss cases in pharmacies 2010 annual report.

M. Kimura, K. Nabeta, F. Tsuchiya (2010). The standard-

ization of medicine name structures suitable for a pre-

scription entry system. In Proceedings of AHFEI2010.

CSEDU 2011 - 3rd International Conference on Computer Supported Education

452