![](bg6.png)
Table 3: Preferences of experienced and inexperienced eval-
uation groups. Bolded values are significantly better than
50% using a binomial test and alpha of 0.05.
Pool Structured Plain Text
Experienced 77.3% 22.7%
Inexperienced 60.6% 39.4%
tables are comprehended. One participant wrote, “Be-
cause the information is organized in a table or has il-
lustrations, you can determine more quickly how rele-
vant the info is.” However, one participant felt that ta-
bles added additional mental work as was mentioned
before, “tables require me to decipher another layout.
It’s slower than just reading the information without
extra images, lists, etc.”
Images received a similar reception. Some users
really liked the images, “Also, the embedded images
were useful in determining the content of the web
page.” Another user felt that images were a big dis-
traction, “...however I noticed that the instant I saw
an image my eye jumped immediately to the url with
the image, bypassing the one I was viewing. I needed
words to be sure it was what I was looking for.”
Overall users preferred summaries when possible
to include structure and multimedia. As one user put
it, the summaries preserving document structure were
“More intuitive and easier to read and understand.”
There will always be those that are familiar with the
current approach and would prefer no change take
place, but these users queried in our evaluation over-
all would benefit from the addition of structured text
and multimedia to search engine summaries.
6 CONCLUSIONS
In this study we introduced an extension to our cur-
rent search engine summary generation system, Re-
Close (Wenerstrom and Kantardzic, 2011b). This ex-
tension introduces the use of structured text and mul-
timedia in the form of HTML tables, lists, text fields,
buttons, hyperlinks and images. We compared these
additions experimentally to traditional summaries ob-
tained from Bing.
The results of our users evaluation suggest that
users will comprehend search results more quickly,
lose no accuracy and prefer the structured summaries
to plain text. From our users evaluation a signifi-
cant number (23 of 33 or 69.7%) of inexperienced
abstract thinkers were faster using structured sum-
maries. While the experienced pool was faster (13/22
or 59.1%) using traditional summaries. No signif-
icant difference existed between the accuracies of
user choices between traditional summaries and struc-
tured, multimedia summaries. Overall a significant
number (37 of 55 or 67.3%) of users preferred struc-
tured, multimedia summaries to the traditional, plain-
text summaries. We now plan to explore the use of
HTML entities as the basic unit of summary genera-
tion rather than text entities like sentences.
REFERENCES
D¨ork, M., Williamson, C., and Carpendale, S. (2009). To-
wards visual web search: Interactive query formula-
tion and search result visualization. In Workshop on
Web Search Result Summarization and Presentation.
Dziadosz, S. and Chandrasekar, R. (2002). Do thumbnail
previews help users make better relevance decisions
about web search results? In Proceedings of the 25th
Annual International ACM SIGIR Conference on Re-
search and Development in Information Retrieval, SI-
GIR ’02, pages 365–366, New York, NY, USA. ACM.
Joho, H. and Jose, J. M. (2008). Effectiveness of addi-
tional representations for the search result presenta-
tion on the web. Information Processing & Manage-
ment, 44(1):226 – 241. Evaluation of Interactive In-
formation Retrieval Systems.
Kelkar, S., John, A., and Seligmann, D. (2009). Visualizing
search results as web conversations. In Workshop on
Web Search Result Summarization and Presentation.
Li, Z., Shi, S., and Zhang, L. (2008). Improving relevance
judgment of web search results with image excerpts.
In Proceeding of the 17th international conference on
World Wide Web, WWW ’08, pages 21–30, New York,
NY, USA. ACM.
Wenerstrom, B. and Kantardzic, M. (2011a). Enhanced vi-
sualization for web-based summaries. In Very Large
Data Search (VLDS) 2011 First International Work-
shop on Searching and Integrating New Web Data
Sources, pages 38–43.
Wenerstrom, B. and Kantardzic, M. (2011b). ReClose:
Web page summarization combining summary tech-
niques. International Journal of Web Information Sys-
tems, 7:333–359.
White, R., Jose, J. M., and Ruthven, I. (2001). Query-biased
web page summarisation: a task-oriented evaluation.
In Proceedings of the 24th annual international ACM
SIGIR conference on Research and development in in-
formation retrieval, SIGIR ’01, pages 412–413, New
York, NY, USA. ACM.
Xue, X.-B., Zhou, Z.-H., and Zhang, Z. M. (2008). Improv-
ing web search using image snippets. ACM Transac-
tions on Internet Technology, 8:21:1–21:28.
KDIR2012-InternationalConferenceonKnowledgeDiscoveryandInformationRetrieval
186