4 PERFORMANCE
EVALUATION AND DESIGN
DECISION SUPPORT
To evaluate the performance of the nine ANN and
six GP models developed in this paper in terms of
their prediction ability in determining the combina-
tion of form elements for matching a given set of
image words, the six test samples identified in Table
2 are used. Rows 3 and 4 of Table 5 show the aver-
age image values (i.e. Q-E and R-E) of the six test
samples assessed by 26 participants, which are used
as a comparison base for the performance evalua-
tion. With the six test samples as the input, Table 5
shows the corresponding image values predicted by
using the ANN, GRA-1-ANN, GRA-2-ANN, GP,
GRA-1-GP, and GRA-2-GP models, respectively.
The last column of Table 5 shows the root mean
squared error (RMSE) of these models in compari-
son with the assessed image values. To evaluate the
performance of a model, the RMSE is commonly
used, given as:
RMSE =
x
i
x
0(i )
i1
n
2
n
(10)
where x
i
is the i-th output value predicted by the
model and x
0(i)
is the expected values assessed by the
participants in this paper. If there is no difference or
error between the predicted value and the assessed
value, the RMSE is 0.
The RMSE given in the last column of Table 5 is
normalized, with the values (the assessed values and
the predicted values) being transformed into a value
between 0 and 1. As indicated in Table 5, the RMSE
value of ANN-HN1 model for Q-E image is 0.12.
This result indicates that this model has an accuracy
rate of 88% (100%-12%) for predicting the value of
Q-E image about fragrance bottles. Table 5 shows
that the average RMSE value of GRA-1-ANN-HN1
model (0.14) is the smallest, followed by the ANN-
HN1, ANN-HN2, and GRA-1-ANN-HN3 models
(0.16). It indicates the GRA-1-ANN-HN1 model is
the best or most suitable for modeling the consum-
ers’ preferences about fragrance bottles. Moreover,
it is noteworthy that the RMSE of the ANN models
is smaller than the GP models. This implies that the
prediction performance of the ANN models is better
than the GP models. In other words, the ANN mod-
els are a more effective technique to formulate the
product design process for determining the optimal
combination of product form elements to best match
to desirable product images (Lai et al., 2005).
Table 5: RMSE of the ANN and GP models developed.
Image
Test sample
RMSE
T1 T2 T3 T4 T5 T6
Consumers
Q-E 3.77 4.73 3.69 3.42 4.38 3.77
R-E 3.12 5.12 4.42 4.46 5.12 3.12
ANN-HN1
Q-E 2.79 4.01 3.57 3.81 4.88 4.92 0.12
0.16
R-E 5.32 4.34 4.25 3.35 4.30 4.03 0.19
ANN-HN2
Q-E 3.69 3.72 4.25 4.12 4.71 4.93 0.12
0.16
R-E 3.82 4.23 3.44 3.29 5.22 5.38 0.20
ANN-HN3
Q-E 2.93 4.02 4.22 4.57 5.81 5.57 0.19
0.21
R-E 4.85 4.18 3.48 3.00 3.80 4.71 0.23
GRA-1-
ANN-HN1
Q-E 3.38 3.99 3.92 4.00 4.77 4.68 0.10
0.14*
R-E 3.67 4.26 3.19 3.03 4.76 4.84 0.19
GRA-1-
ANN-HN2
Q-E 2.50 4.12 3.76 4.07 4.98 5.10 0.14
0.17
R-E 3.60 5.09 3.60 3.14 5.23 5.38 0.19
GRA-1-
ANN-HN3
Q-E 2.80 3.89 4.06 4.13 4.99 5.19 0.15
0.16
R-E 4.69 4.28 3.68 3.09 4.90 4.14 0.18
GRA-2-
ANN-HN1
Q-E 3.47 4.76 4.54 4.33 6.00 5.35 0.18
0.17
R-E 4.06 5.03 3.53 3.12 4.84 4.49 0.16
GRA-2-
ANN-HN2
Q-E 2.95 4.86 4.13 3.73 5.43 5.06 0.13
0.17
R-E 3.98 4.67 3.26 2.54 4.37 4.99 0.22
GRA-2-
ANN-HN3
Q-E 3.43 4.83 5.03 4.29 5.68 5.94 0.21
0.20
R-E 4.44 3.76 3.26 2.75 4.23 3.34 0.20
GP
Q-E 1.60 2.43 1.98 2.82 3.57 1.59 0.25
0.28
R-E 1.58 2.20 1.84 2.43 4.16 1.15 0.30
GRA-1-GP
Q-E 1.79 2.68 2.23 3.04 3.76 1.88 0.22
0.26
R-E 1.60 2.22 1.86 2.44 4.18 1.17 0.30
GRA-2-GP
Q-E 0.52 0.92 0.39 1.33 0.61 0.70 0.47
0.34
R-E 2.00 3.07 3.94 2.24 4.30 2.60 0.20
Consequently, product designers can use the
GRA-1-ANN-HN1 model to build a fragrance bottle
design decision support database that can be gener-
ated by inputting each of all possible combinations
(1,440, 2×5×4×2×3×2×3) of product form elements
for generating the associated image values. In other
words, 1,440 design alternatives generated by the
GRA-1-ANN-HN1 model can be chosen to best
match specific consumers’ preferences. Product de-
signers can also specify a desirable image value for a
new fragrance bottle form design, and the design de-
cision support database can then work out the opti-
mal combination of form elements. For example, the
product designer can use a computer-aided design
(CAD) or a computer-aided manufacture (CAM)
system to facilitate the product design in the new
fragrance development process. As an illustration,
Figure 1 shows the new fragrance bottle form design
by CAD/CAM system with “rational” image, and
Table 6 shows its corresponding combination of
form elements (out of 1,440 design alternatives).
GreyRelationalAnalysisbasedArtificialNeuralNetworksforProductDesign:AComparativeStudy
657