![](bg2.png)
2 RELATED WORK
Although there are many methods aiming at the de-
tection of specularities, shadows, or occlusions, the
majority of those approaches deals with only one of
these phenomena.
An overview of specularity removal techniques
can be found in (Artusi et al., 2011). Methods for
specularity detection can be divided into two groups
based on whether they use a single or multiple images
as input. The technique in (Klinker et al., 1988) an-
alyzes the color space of a single image in order to
estimate the distribution of specular pixels. The work
in (Mallick et al., 2006) is based on a single image as
well, analyzes the spatial neighborhood of a pixel, and
uses partial differential equations to iteratively erode
the specular components. The method of (Yang et al.,
2013) separates specular and diffuse components in
the HSI color space, clusters pixels with similar char-
acteristics, and subsequently finds the optimal satura-
tion.
Multi-image methods use multiple input images
of the same scene but obtained from different points
of view. The work in (Lin et al., 2002) uses color
histogram differencing to retrieve specular pixels. It
is based on color changes of specular regions among
the different views which are estimated by computing
the distance of the corresponding color histograms.
The method for specularity removal of flat objects
proposed in (Biasotti et al., 2015) employs a pixel
value minimization across multi-view images which
are captured by a mobile phone. It requires known
camera orientation for each image, which is obtained
by the built-in inertial measurement unit. The refer-
ence image is selected to be approximately parallel to
the scanned surface.
A second category of approaches includes meth-
ods that require special equipment. The multi-
flash method in (Feris et al., 2004) demands a flash
system and uses an image sequence captured from
fixed viewpoints with different positions of flash-light
sources. The work in (Ma et al., 2007) is based on the
fact that specular components of the reflected light are
polarized. To separate these components, this method
needs suitable polarization filters to measure the re-
flected light.
The detection of shadows is an important pre-
processing step in many computer vision applications.
The corresponding methods can be coarsely divided
into pixel- and region-based approaches. An example
of the former is the method in (Murali and Govindan,
2013) which detects shadows of a single image in the
CIELab color space. The work in (Guo et al., 2011) is
a region-based approach that classifies regions of the
segmented image as shadow or non-shadow, accord-
ing to their relative illumination.
Specularities and shadows can also be handled
by deriving intrinsic images that decompose an im-
age into its reflectance and illumination component.
For example, the method in (Weiss, 2001) applies
two derivative filters to a sequence of images to re-
cover individual illumination images and a single re-
flectance map which is almost free of specularities
and shadows. Several multi-view approaches for out-
door scenes (Laffont et al., 2013; Duch
ˆ
ene et al.,
2015) compute a proxy geometry of the scene and uti-
lize it for illumination estimation.
Occlusions can be considered as one of the biggest
challenges in stereo vision. In (Zitnick and Kanade,
2000) the authors introduce a cooperative stereo algo-
rithm. It is based on an uniqueness assumption and
is jointly creating disparity maps as well as detect-
ing occlusions. In order to detect occlusions in two-
frame stereo, the iterative optimization algorithm in
(Sun et al., 2005) uses a visibility constraint which is
more general than the uniqueness constraint of (Zit-
nick and Kanade, 2000).
The technique proposed in this paper does not re-
quire any special equipment. Furthermore, two input
images from two different viewpoints are sufficient
while there are no specific constraints on the spatial
relation between the two cameras and the object. In
contrast to works such as (Biasotti et al., 2015), it
is not necessary to determine the orientation of the
cameras. Consequently, there is no predefined cri-
terion for the selection of an image as the reference
frame. The main contribution of the proposed ap-
proach is the creation of a corrected image, free of
specularities, shadows, as well as occlusions for a pla-
nar object, even if the images contain all these unde-
sired phenomena. The simplicity of the developed al-
gorithm makes it robust and applicable in many use
cases, i.e. it does not depend on specific image acqui-
sition circumstances, high-end cameras, or other spe-
cial equipment. Instead, images can be captured by
hand-held consumer or mobile phone cameras from
arbitrary viewpoints.
3 PROPOSED ALGORITHM
Specularities, shadows, and occlusions are effects that
most commonly and most strongly affect the visibil-
ity of objects in images. They become especially se-
vere in the case of images of planar and reflective
surfaces which were acquired in crowded scenarios,
e.g. posters in the interactive sessions of a conference.
That is why the proposed technique focuses on the
Specularity, Shadow, and Occlusion Removal for Planar Objects in Stereo Case
99