by the summarization system. Finally, we compute the similarity between the each of
the obtained vectors and the vector corresponding to the “perfect” product using the co-
sine similarity measure
7
.We recommend the top 5 matching products. In order to better
understand the process of recommendation, we will consider an example and suppose
the user would like to buy a 4 Megapixel camera. There are around 250 available mod-
els on the market and for each model one can read an average of 10 customer reviews.
Instead of having to read 2500 reviews, employing the presented system, being given
the 5 best products, the user will only have to browse through 50 reviews, in case (s)he
is not confident in the system classification; when the user is confident, (s)he has to read
none.
The list of features for a 4 Megapixel camera is: (price, warranty, size, design, ap-
pearance, weight, quality, lens, viewfinder, optical zoom, digital zoom, focus, image
resolution, video resolution, memory, flash, battery, battery life, LCD size, LCD reso-
lution, accessories)
The vector associated to the “perfect” 4 Megapixel camera will have as indices the
features in the abovelist and all corresponding values 1: v perf(price)=1;v perf(warranty)
=1 and so on, in the order given by the list of features. After applying the customer re-
view summarization system on other 4 Megapixel cameras, we obtain among others the
vectors v1 and v2, correspondingto Camera1 4MP and Camera2 4MP. The values of v1
are: (0.7,0.5,0.6,0.2,0.3,0.6,0.5, 0.5,0.7,0.8,0.7,0.8, 0.4,0.3,0.3,0.7,0.6,0.3,0.8,0.4,0.4)
The values of v2 are: (0.8,1, 0.7,0.2,0.2,0.5, 0.4,0.4,0.8,0.8,0.8,0.8,0.7,0.7,0.3,0.8,0.6,
0.7,0.5,0.3,0.6) Calculating the cosine similarity between v1 and v perf and v2 and
v perf, respectively, we obtain 0.945 and 0.937. Therefore, we conclude that Cam-
era1 4MP is better than Camera2 4MP, because it is more similar to the “perfect” 4
Megapixel camera model.
9 Evaluation and Discussion
We performed a thorough evaluation the system for customer review summarization
and an informal evaluation of the recommender. For the first, we annotated a cor-
pus of 50 customer reviews for each language, collected from the sites containing the
“pros and cons”- style reviews: “newgegg.com”, “eopinions.com”, “shopping.com”,
“quesabesde.com” and “ciao.es”. The corpus was annotated at the level of feature at-
tributes, by the following scheme: <attribute>[name of attribute] <feature>[feature it de-
termines]</feature><value>[positive / negative]</value></attribute>.
It is difficult to evaluate the performance of such a system, since we must take into
consideration both the accuracy in extracting the features that reviews comment on, as
well as the correct assignation of identified feature attributes to the positive or nega-
tive category. Therefore, we introduced three formulas for computing the system per-
formance: Accuracy (Eq 1), Feature Identification Precision (FIP) (Eq. 2) and Feature
Identification Recall (FIR) (Eq. 3).
A =
P
n
i
(
#pos id features(i)
#pos f eatures(i)
+
#neg id f eatures(i)
#neg f eatures(i)
)
2n
(1)
7
www.dcs.shef.ac.uk/ sam/stringmetrics.html#cosine
39