Authors:
Po-Shen Lee
and
Bill Howe
Affiliation:
University of Washington, United States
Keyword(s):
Visualization, Multi-chart Figure, Chart Segmentation, Chart Recognition and Understanding, Scientific Literature Retrieval, Content-based Image Retrieval.
Related
Ontology
Subjects/Areas/Topics:
Applications
;
Computer Vision, Visualization and Computer Graphics
;
Image Understanding
;
Pattern Recognition
Abstract:
We are analyzing the visualizations in the scientific literature to enhance search services, detect plagiarism,
and study bibliometrics. An immediate problem is the ubiquitous use of multi-part figures: single images with
multiple embedded sub-visualizations. Such figures account for approximately 35% of the figures in the scientific
literature. Conventional image segmentation techniques and other existing approaches have been shown
to be ineffective for parsing visualizations. We propose an algorithm to automatically segment multi-chart visualizations
into a set of single-chart visualizations, thereby enabling downstream analysis. Our approach first
splits an image into fragments based on background color and layout patterns. An SVM-based binary classifier
then distinguishes complete charts from auxiliary fragments such as labels, ticks, and legends, achieving
an average 98.1% accuracy. Next, we recursively merge fragments to reconstruct complete visualizations,
choosing be
tween alternative merge trees using a novel scoring function. To evaluate our approach, we used
261 scientific multi-chart figures randomly selected from the Pubmed database. Our algorithm achieves 80%
recall and 85% precision of perfect extractions for the common case of eight or fewer sub-figures per figure.
Further, even imperfect extractions are shown to be sufficient for most chart classification and reasoning tasks
associated with bibliometrics and academic search applications.
(More)