are syntactically identical and logically equivalent to
ideal JML statements.
The project used in Experiment 2 does not have
not only OCL statements but also JML statements.
We, however, need both data in order to perform an
Experiment 2. Therefore, firstly, we described JML
statements on the basis of the specification which is
read from the codes and comments of javadoc. Sec-
ondly, we checked the correctness of the JML state-
ments by running test cases with a runtime assertion
checker, JMLrac. Finally, we described OCL state-
ments that are syntactically identical and logically
equivalent to ideal JML statements.
In order to evaluate the quality of JML statements
generated by the translation tool, we measured three
items, coverage, recall ratio, and execution time of the
runtime assertion checker.
Here, firstly we provide a metric for the coverage.
We define the coverage by Expression (4), where C
all
and C
translatable
are the number of pre-conditions and
post-conditions, and the number of statements trans-
lated by the translation tool, respectively.
Coverage = C
translatable
/C
all
(4)
Secondly, we provide the recall ratio. We de-
fine the recall ratio by Expression (5), where C
equal
is
the number of statements which are translated by the
translation tool and are identical to the corresponding
ideal JML statement. Here, “corresponding” stands
for syntactical and logical equivalence.
recallratio = C
equal
/C
translatable
(5)
Finally, we compare execution times of runtime
assertion checker, JMLrac by applying source codes
with JML statements generated by the translation tool
and that with ideal JML statements. We use JML4c
(Sarcar and Cheon, 2010) and JML4rt (Sarcar and
Cheon, 2010) as a JML compiler and a runtime asser-
tion checker, respectively. As far as we know, JML4c
is the only compiler that supports enhanced for-loop
and generics. Also for a given executable file gener-
ated by JML4c, JML4rt checks consistency between
runtime values of variables and the constraints given
by the JML statements. When some of constraints of
JML statements violate to the runtime values of vari-
ables, JML4rt outputs the location of violated point
and the values of variables at the location.
4.2 Environment of Experiments
Evaluations have been performed on an Intel(R)
Core(TM)2 Duo E7300 2.66GHz 2.67GHz HP Com-
paq dx7500 Microtower with 4GB RAM running
Windows Vista 64bit.
4.3 Experiment 1
In Experiment 1, we evaluated the quality of the trans-
lation using the warehouse management program. It
consists of seven classes. Table 2 shows components
of the warehouse management program in details. It
(Owashi et al., 2008) has correct JML statements by
the past research. In Experiment 1, we add OCL state-
ments for 20 methods.
In Experiment1, as we described, JML statements
have already been fully annotated. We selected three
core classes and annotated OCL statements. For the
three classes, we can translate every OCL statements
into correct JML statements. Consequently, Cover-
age and recall ratio are both 100 %, which indicates
the power of the tool. Also, we observed execution
times of both cases. The results show that there is no
distinguishing execution time of manual JML state-
ments from that of JML statements by the translation
tool.
Execution time of translation from OCL state-
ments to JML statement is 20ms. Also, execution
time of translation from abstract tree of OCL to ab-
stract tree of JML is 1ms. Therefore, if we describe
OCL statements to all attributesand all methods of the
program, perhaps execution time of translation from
these OCL statements (about one hundred statements)
to JML statement will be about 2 sec.
4.4 Experiment 2
In Experiment 2, we evaluated the quality of transla-
tion tool by data of educational material of IT-Spiral
(Ministry of Education, Culture, Sports, Science and
Technology, 2010). The project is a syllabus manage-
ment system of Wakayama University. The system
consists of 200 classes, and we choose core 60 classes
and annotate 400 methods in the chosen classes. Ta-
ble 3 shows parts of OCL statements and JML state-
ments.
Table 4 shows the result of translation from OCL
statements of the syllabus management system to
Table 2: Components of warehouse management program.
Class Name # of methods # of lines
ContainerItem 12 224
Customer 10 156
Item 7 110
ReceptionDesk 8 162
Request 16 245
StockState 0 9
Storage 10 258
TOTAL 63 1164
ICEIS2012-14thInternationalConferenceonEnterpriseInformationSystems
392