Unsupervised Learning for Color Constancy
Nikola Bani
´
c and Sven Lon
ˇ
cari
´
c
Image Processing Group, Department of Electronic Systems and Information Processing,
Faculty of Electrical Engineering and Computing, University of Zagreb, 10000 Zagreb, Croatia
Keywords:
Clustering, Color Constancy, Illumination Estimation, Unsupervised Learning, White Balancing.
Abstract:
Most digital camera pipelines use color constancy methods to reduce the influence of illumination and camera
sensor on the colors of scene objects. The highest accuracy of color correction is obtained with learning-based
color constancy methods, but they require a significant amount of calibrated training images with known
ground-truth illumination. Such calibration is time consuming, preferably done for each sensor individually,
and therefore a major bottleneck in acquiring high color constancy accuracy. Statistics-based methods do
not require calibrated training images, but they are less accurate. In this paper an unsupervised learning-based
method is proposed that learns its parameter values after approximating the unknown ground-truth illumination
of the training images, thus avoiding calibration. In terms of accuracy the proposed method outperforms all
statistics-based and many state-of-the-art learning-based methods. The results are presented and discussed.
1 INTRODUCTION
Beside other abilities the human visual system (HVS)
can recognize colors of scene objects even under var-
ious illumination. This ability is known as color con-
stancy (Ebner, 2007) and most digital cameras have
computational color constancy implemented in their
image processing pipelines (Kim et al., 2012). The
main task of computational color constancy is to per-
form an accurate illumination estimation, which is
then used to chromatically adapt the image in order
to remove the influence of the illumination on colors.
The most commonly used image f formation model
for this problem with included Lambertian assump-
tion is (Gijsenij et al., 2011)
f
c
(x) =
Z
ω
I(λ, x)R(λ, x)ρ
c
(λ)dλ (1)
where c {R, G, B} is a color channel, x is a given
image pixel, λ is the wavelength of the light, ω is
the visible spectrum, I(λ, x) is the spectral distribution
of the light source, R(λ, x) is the surface reflectance,
and ρ
c
(λ) is the camera sensitivity of color channel c.
To make the problem simpler, uniform illumination is
usually assumed and by removing x from I(λ, x), the
observed light source color is given as
e =
e
R
e
G
e
B
=
Z
ω
I(λ)ρ(λ)dλ. (2)
By knowing only the direction of e, an image can
be successfully chromatically adapted (Barnard et al.,
2002). With only image pixel values f given and
both I(λ) and ρ(λ) unknown, calculating e is an ill-
posed problem, which needs additional assumptions
to be solved. Many illumination estimation methods
with different assumptions have been proposed. In
the first of two main groups of illumination estima-
tion methods are low-level statistics-based methods
such as White-patch (Land, 1977; Funt and Shi, 2010)
and its improvements (Bani
´
c and Lon
ˇ
cari
´
c, 2013;
Bani
´
c and Lon
ˇ
cari
´
c, 2014a; Bani
´
c and Lon
ˇ
cari
´
c,
2014b), Gray-world (Buchsbaum, 1980), Shades-of-
Gray (Finlayson and Trezzi, 2004), Grey-Edge (1st
and 2nd order) (Van De Weijer et al., 2007a), us-
ing bright and dark colors (Cheng et al., 2014).
The second main group consists of learning-based
methods, all of which are supervised, like gamut
mapping (pixel, edge, and intersection based) (Fin-
layson et al., 2006), using high-level visual infor-
mation (Van De Weijer et al., 2007b), natural im-
age statistics (Gijsenij and Gevers, 2007), Bayesian
learning (Gehler et al., 2008), spatio-spectral learn-
ing (maximum likelihood estimate, and with gen.
prior) (Chakrabarti et al., 2012), simplifying the illu-
mination solution space (Bani
´
c and Lon
ˇ
cari
´
c, 2015a;
Bani
´
c and Lon
ˇ
cari
´
c, 2015b; Bani
´
c and Lon
ˇ
cari
´
c,
2015b), using color/edge moments (Finlayson, 2013),
using regression trees with simple features from color
Bani
´
c, N. and Lon
ˇ
cari
´
c, S.
Unsupervised Learning for Color Constancy.
DOI: 10.5220/0006621801810188
In Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018) - Volume 4: VISAPP, pages
181-188
ISBN: 978-989-758-290-5
Copyright © 2018 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
181
distribution statistics (Cheng et al., 2015), performing
various kinds of spatial localizations (Barron, 2015;
Barron and Tsai, 2017), using convolutional neural
networks (Bianco et al., 2015; Shi et al., 2016; Hu
et al., 2017). Statistics-based methods are character-
ized by a relatively high speed, simplicity, and usu-
ally lower accuracy, while with the learning-based
methods it is vice versa. However, several recently
proposed learning-based methods are not only highly
accurate, but also as fast as statistics-based meth-
ods (Bani
´
c and Lon
ˇ
cari
´
c, 2015b; Cheng et al., 2015).
Nevertheless, since all well-known learning-based
methods are supervised, a major obstacle for their ap-
plication is that for a given sensor, despite proposed
workarounds (Gao et al., 2016), supervised learning-
based methods have to be trained on calibrated images
taken by preferably the same sensor (Aytekin et al.,
2017). To calibrate the images, a calibration object
has to be placed in the scenes of these images and
later segmented to extract the ground-truth illumina-
tion. The amount of manual work required for such
calibration is the main bottleneck in enabling highly
accurate color constancy for a given sensor.
To try to avoid such calibration, in this paper an
unsupervised learning-based method is proposed that
learns its parameter values from non-calibrated im-
ages with unknown ground-truth illumination. Such
learning is possible by clustering the approximated
ground-truth illuminations of images from the train-
ing set and then extracting useful information. The
method is fast, hardware-friendly, and it outperforms
most state-of-the-art methods in terms of accuracy. To
the best of the authors’ knowledge this is the first suc-
cessful unsupervised learning-based color constancy
method evaluated on benchmark datasets and thus
also a contribution to the color constancy philosophy.
The paper is structured as follows: Section 2 lays
out the motivation for the proposed method, Section 3
describes the method, in Section 4 the experimental
results are presented and discussed, and finally Sec-
tion 5 concludes the paper.
Figure 1: The rb-chromaticities of the ground-truth illumi-
nations and Gray-world illumination estimations for images
of the Samsung benchmark dataset (Cheng et al., 2014).
2 MOTIVATION
Ground-truth illumination of training images for su-
pervised learning-based methods is extracted from
calibration objects placed in the image scenes. As
explained in the introduction, obtaining the ground-
truth illumination is time consuming, but it enables
supervised learning and high illumination estimation
accuracy. To speed things up significantly, usage of
calibration objects has to be dropped out. Then in
place of the real ground-truth illumination, some kind
of its approximation has to be used instead, e.g. illu-
mination estimations obtained by means of statistics-
based methods that require no previous learning. But
since they are usually less accurate than learning-
based methods, using their estimations as the ground-
truth illumination may be counterproductive. How-
ever, instead of only image-based illumination esti-
mation, there are other kinds of information that such
methods provide. Namely, even illumination estima-
tions of the simplest statistics-based methods appear
”to correlate roughly with the actual illuminant” (Fin-
layson, 2013) as shown in Fig 1 i.e. they occupy
roughly the same region in the chromaticity plane. To
have a better insight into this phenomenon, some ad-
ditional numerical analysis is required.
Figure 2: Values of SAE averaged over 1000 random sub-
sets of the Sony benchmark dataset (Cheng et al., 2014) for
various subset sizes.
As described in more detail later in Section 4.1,
the error measure for accuracy of illumination esti-
mation is the angular error i.e. the angle between the
vectors of ground-truth illumination and illumination
estimation. One way to see how well a set of illu-
mination estimations numerically resembles the set
of ground-truth illuminations in terms of occupying
the same region in the chromaticity space is to rear-
range the existing illumination estimations between
images in order to minimize the sum of overall an-
gular errors obtained for such rearranged illumination
estimations. More formally this is the same as mak-
ing pairs of illumination points under three following
constraints: first, in each pair one point comes from
VISAPP 2018 - International Conference on Computer Vision Theory and Applications
182
(a) (b) (c)
Figure 3: Percentage of ground-truth with specified angle to the closest estimation (first row) and vice versa (second row)
obtained with Shades-of-Gray on images of the Sony benchmark dataset (Cheng et al., 2014) for (a) n = 1, (b) n = 4, and
(c) n = 8.
the set of ground-truth illuminations and the other
from the set of illumination estimations; second, ev-
ery point from both sets is a member of exactly one
pair; and third, the pairs are formed so that the the sum
of angular errors between all pair points is minimized.
Effectively, this boils down to solving the optimal as-
signment problem (Burkard et al., 2012). In the rest
of the paper the minimal possible mean angular error
between pair points for given sets will be denoted as
Sets’ Angular Error (SAE). It must be clearly stressed
here that a low SAE does not implicate an accurate
method; by definition an inaccurate method can un-
der certain conditions produce estimations with a low
SAE. As the number of points in the sets grows, SAE
should decrease since every point will have more pair-
ing opportunities. This is shown in Fig. 2 where the
values of SAE averaged over 1000 random subsets of
the Sony benchmark dataset (Cheng et al., 2014) de-
crease as the size of the used subsets increases. Based
on the empirical evidence, including the results shown
in Fig. 2, the impact of the method choice on SAE
seems to be higher then the impact of the set size.
These results show that by applying well chosen
methods to a sufficient number of given images it is
possible to obtain a low SAE, which is a proof of con-
cept that a relatively accurate approximation of the
set of unknown ground-truth illuminations for these
images is feasible. This definitely motivates to ex-
ploit the demonstrated concept further, but to have
a practical use of it, at least two questions need to
be answered: first, what other information useful for
a more accurate illumination estimation can be ex-
tracted from a set of ground-truth illumination ap-
proximations, and second, how to obtain such approx-
imated sets that have a low SAE?
As for the first question, the ground-truth illumi-
nations or their approximations for many images can
reveal in which chromaticity space regions are future
illumination estimations of new images most likely
to appear. There are several methods that rely on
such kind of information (Bani
´
c and Lon
ˇ
cari
´
c, 2015a;
Mazin et al., 2015; Cheng et al., 2015; Bani
´
c and
Lon
ˇ
cari
´
c, 2015b) with probably the least demand-
ing one being the Color Dog method (Bani
´
c and
Lon
ˇ
cari
´
c, 2015b). During the training phase it clus-
ters the ground-truth illuminations by using the k-
means clustering (Vassilvitskii, 2007) with angular
instead of Euclidean distance. The cluster centers
obtained in this process become the only illumina-
tion estimations that the method will ever produce
when used later in production. When applied to a
new image, Color Dog first runs the parameterless
White-patch and Gray-world methods. The angular
distances between their illumination estimations and
the learned cluster centers are used as weighted votes
to determine which center should represent the illu-
mination on the given image. Well positioned centers
in the chromaticity plane result in relatively small er-
rors (Bani
´
c and Lon
ˇ
cari
´
c, 2015b) and despite its sim-
plicity, Color Dog was reported to be very successful.
The centers and their number are learned by means of
nested cross-validation (Japkowicz and Shah, 2011).
Since accurate ground-truth illuminations are needed
for such learning, using approximations gives poor re-
sults, but the main idea of Color Dog can be the basis
for a method that learns from approximations. Such a
new method is proposed in the following section.
Unsupervised Learning for Color Constancy
183
(a) (b)
Figure 4: The rb-chromaticities of the ground-truth illuminations and SoG illumination estimations for n = 8 on images of
the Samsung benchmark dataset (Cheng et al., 2014) (a) before and (b) after trimming with t = 0.3 (best viewed in color).
3 THE PROPOSED METHOD
Nested cross-validation can be circumvented by sim-
ply fixing the number of centers. Using more cen-
ters increases the upper limit for accuracy because
of the finer chromaticity space splitting, but it also
poses a harder classification problem for which the
upper accuracy limit may be rarely reached. Thus
the new method proposed here uses only two cen-
ters and assumes that most images can be classified
as having either a warmer i.e. reddish or a cooler i.e.
blueish illumination, which is effectively a simplifica-
tion of the Planckian locus (Schanda, 2007) that has
already been used for illumination estimation (Bani
´
c
and Lon
ˇ
cari
´
c, 2015a; Mazin et al., 2015). A some-
what similar rough division to an indoor and outdoor
type illumination has been successfully used for a
slightly different purpose in (Cheng et al., 2016).
With the answer to the first question from the pre-
vious section proposed, it remains to resolve the sec-
ond one i.e. which illumination estimations should
be clustered to get centers that are well positioned
among the ground-truth illumination? A single
statistics-based method with fixed parameter values
may achieve a relatively low SAE, but with unknown
ground-truth illuminations, it cannot be said which
parameter values will result in minimal SAE. To solve
this problem, it can be assumed that for any set of pa-
rameter values for a statistics-based method in most
cases there will be a number of training images for
which the method’s illumination estimations will be
accurate. Other parameter values should again give
accurate estimations for some other images. If this
is repeated for more sets of parameter values, then
the region with the actual ground-truth illumination
should be more densely filled with illumination esti-
mations than other regions. Such behaviour can also
be observed for the Shades-of-Gray (SoG) (Finlayson
and Trezzi, 2004) method, which uses the Minkowski
norm p for illumination estimation
Z
( f
c
(x))
p
dx
1
p
= ke
c
(3)
where k is a constant that assures the unit length of
e. SoG already offers a diversity of illumination esti-
mations by only changing the value of its single pa-
rameter. While other statistics-based methods like
Gray-Edge may be more accurate, this holds only if
their multiple parameters are well chosen. In order
to avoid possible problems related to parameter value
tuning, the proposed method clusters combined SoG
illumination estimations for p {1, 2, ..., n}. Fig. 3
shows the influence of n on percentage of ground-
truth with specified angle to the closest estimation and
vice versa. It can be observed that using combined
SoG estimations for various values of p can indeed
result in a more accurate coverage of the chromatic-
ity plane regions populated with ground-truth illumi-
nations. Theoretically this should also improve the
accuracy of obtained clustering centers.
Figure 5: Percentage of SoG estimations for n = 8 with
specified angle to the closest ground-truth for the Sony
benchmark dataset (Cheng et al., 2014).
However, beside putting more points around the
actual chromaticity plane region with the ground-
truth, combining estimations for several values of
p also introduces a lot of estimations that are far
VISAPP 2018 - International Conference on Computer Vision Theory and Applications
184
away from all ground-truth illuminations and repre-
sent noise. Under the used assumption such estima-
tions should be scattered and less dense than the es-
timations closer to the ground-truth region and this
could be used to reduce their influence. A direct so-
lution would be to use clustering techniques that con-
sider outliers and simply ignore them with one ex-
ample being DBSCAN (Ester et al., 1996). How-
ever, since DBSCAN and some other similar meth-
ods determine the number of centers on their own
and additionally the problem here does not involve
some arbitrarily shaped clusters, another solution is
proposed. After the initial clustering with k-means,
for each cluster center 100 · t% of its furthest estima-
tions are removed i.e. trimmed and then clustering is
repeated only on the remaining estimations to obtain
the final cluster centers. This trimming procedure is
summarized in Algorithm 1. Fig. 4 shows an exam-
ple of such an outlier removal. The numerical effect
of it can be observed when comparing the lower right
histogram in Fig. 3 and the histogram in Fig. 5, which
shows that after trimming the remaining illuminations
are much closer to the ground-truth. Default parame-
ter values are set to n = 8 and t = 0.3 since they were
empirically found to work well. In the following sec-
tion on experimental results these values have been
used for all benchmark datasets. Tuning them for each
set individually would result in a significantly higher
accuracy, but that would defeat the whole purpose of
unsupervised learning since ground-truth illumination
would be needed for such fine tuning of parameters.
For simpler notation in the experimental results
and because the proposed method learns the values
of its parameters from images obtained in the wild
without knowing their ground-truth illumination, it is
named Color Tiger (CT). Now that the whole theo-
retical background with all required assumptions has
been explained, Color Tiger’s training procedure can
be simply described as learning the centers of two
clusters from a specifically trimmed set of illumina-
tion estimations obtained by applying Shades-of-Gray
to training images for every p {1, 2, ..., 8}, which is
summarized in Algorithm 2. The illumination estima-
tion for new images resembles the one of Color Dog
and it is given in Algorithm 3.
4 EXPERIMENTAL RESULTS
4.1 Experimental Setup
The following benchmark datasets have been used to
compare the accuracy of the proposed method to the
accuracy of other well-known methods: the GreyBall
Algorithm 1: Trimming.
Input: data D, number of centers k, threshold t
Output: trimmed data T
1: C = kmeans(D, k) Use angular distance
2: T = {}
3: r = 1 t
4: for c
i
C do
5: D
c
i
= {d D | c
i
= arg min
c
j
C
(c
j
, d)}
6: r
0
= b100 · rc-th percentile of { (c
i
, d) | d
D
c
i
}
7: D
0
c
i
= {d | (c
i
, d) r
0
}
8: T.AddAll(D
0
c
i
)
9: end for
Algorithm 2: Color Tiger Training.
Input: images I, SoG upper power n, trimming t
Output: set of two centers C
1: E = {}
2: for I I do
3: for p {1, 2, ..., n} do
4: e = ShadesO f Gray(I, p)
5: E.Add(e)
6: end for
7: end for
8: E
0
= Trimming(E, 2, t)
9: C = kmeans(E
0
, 2) Use angular distance
Algorithm 3: Color Tiger Application.
Input: image I, set of two centers C
Output: illumination estimation e
1: e
GW
= GrayWorld(I)
2: e
W P
= W hitePatch(I)
3: e = arg max
c
i
C
c
i
·e
GW
||c
i
||·||e
GW
||
+
c
i
·e
W P
||c
i
||·||e
W P
||
dataset (Ciurea and Funt, 2003), its approximated
linear version, and eight linear NUS dataset (Cheng
et al., 2014). The ColorChecker dataset (Gehler et al.,
2008; L. Shi, 2015) has not been used to avoid confu-
sion over various results mentioned in numerous pub-
lications during ColorChecker’s history of partially
wrong usage when reporting the results of older meth-
ods despite warnings from leading experts in the area
of color constancy going back as far as 2013 (Lynch
et al., 2013). Since in digital devices illumination es-
timation is mostly performed on linear images (Kim
et al., 2012) similar to the model described by Eq. (1),
linear datasets are preferred over the non-linear ones.
Each dataset has images and their ground-truth il-
luminations obtained by putting a calibration object
Unsupervised Learning for Color Constancy
185
in the image scene, e.g. a color checker or a gray ball
that is masked out during testing to avoid bias.
Various illumination estimation accuracy mea-
sures have been proposed (Gijsenij et al., 2009; Fin-
layson and Zakizadeh, 2014; Bani
´
c and Lon
ˇ
cari
´
c,
2015a). The most commonly used one is the angular
error i.e. the angle between the illumination estima-
tion vector and the ground-truth illumination. All an-
gular errors obtained for a given method on a chosen
dataset are usually summarized by different statistics.
Because of the non-symmetry of the angular error dis-
tribution, the most important of these statistics is the
median angular error (Hordley and Finlayson, 2004).
Cross-validation on all used datasets was per-
formed with the same folds as in other publications.
The source code for recreating the results given
in the following subsection is publicly available at
http://www.fer.unizg.hr/ipg/resources/color constancy/.
Table 1: Combined performance of different color con-
stancy methods on eight NUS dataset (lower median is bet-
ter). The used format is the same as in (Barron and Tsai,
2017).
Algorithm Mean Med. Tri.
Best
25%
Worst
25%
Avg.
White-Patch (Funt and Shi, 2010) 9.91 7.44 8.78 1.44 21.27 7.24
Pixels-based Gamut (Gijsenij et al.,
2010)
5.27 4.26 4.45 1.28 11.16 4.27
Grey-world (Buchsbaum, 1980) 4.59 3.46 3.81 1.16 9.85 3.70
Edge-based Gamut (Gijsenij et al.,
2010)
4.40 3.30 3.45 0.99 9.83 3.45
Shades-of-Gray (Finlayson and Trezzi,
2004)
3.67 2.94 3.03 0.98 7.75 3.01
Natural Image Statistics (Gijsenij and
Gevers, 2011)
3.45 2.88 2.95 0.83 7.18 2.81
Local Surface Reflectance
Statistics (Gao et al., 2014)
3.45 2.51 2.70 0.98 7.32 2.79
2nd-order Gray-Edge (Van De Weijer
et al., 2007a)
3.36 2.70 2.80 0.89 7.14 2.76
1st-order Gray-Edge (Van De Weijer
et al., 2007a)
3.35 2.58 2.76 0.79 7.18 2.67
Bayesian (Gehler et al., 2008) 3.50 2.36 2.57 0.78 8.02 2.66
General Gray-World (Barnard et al.,
2002)
3.20 2.56 2.68 0.85 6.68 2.63
Spatio-spectral Statistics (Chakrabarti
et al., 2012)
3.06 2.58 2.74 0.87 6.17 2.59
Bright-and-dark Colors PCA (Cheng
et al., 2014)
2.93 2.33 2.42 0.78 6.13 2.40
Corrected-Moment (Finlayson, 2013) 2.95 2.05 2.16 0.59 6.89 2.21
Color Tiger (proposed) 2.96 1.70 1.97 0.53 7.50 2.09
Color Dog (Bani
´
c and Lon
ˇ
cari
´
c,
2015b)
2.83 1.77 2.03 0.48 7.04 2.03
Shi et al. 2016 (Shi et al., 2016) 2.24 1.46 1.68 0.48 6.08 1.74
CCC (Barron, 2015) 2.38 1.48 1.69 0.45 5.85 1.74
Cheng 2015 (Cheng et al., 2015) 2.18 1.48 1.64 0.46 5.03 1.65
FFCC (Barron and Tsai, 2017) 1.99 1.31 1.43 0.35 4.75 1.44
4.2 Accuracy
Tables 1, 2, and 3 show the comparisons between
the accuracies of the proposed method and other il-
lumination estimation methods on various datasets.
The proposed method outperforms all statistics-based
methods and many learning-based methods. For all
datasets except for the GreyBall dataset its median
Table 2: Performance of different color constancy methods
on the original GreyBall dataset (lower median is better).
Algorithm Mean Median Trimean
do nothing 8.28 6.70 7.25
Low-level statistics-based methods
Gray-world (GW) (Buchsbaum, 1980) 7.87 6.97 7.14
White-Patch (WP) (Funt and Shi, 2010) 6.80 5.30 5.77
Shades-of-Gray (Finlayson and Trezzi, 2004) 6.14 5.33 5.51
General Gray-World (Barnard et al., 2002) 6.14 5.33 5.51
1st-order Gray-Edge (Van De Weijer et al.,
2007a)
5.88 4.65 5.11
2nd-order Gray-Edge (Van De Weijer et al.,
2007a)
6.10 4.85 5.28
Learning-based methods
Pixel-based gamut (Finlayson et al., 2006) 7.07 5.81 6.12
Edge-based gamut (Finlayson et al., 2006) 6.81 5.81 6.03
Intersection-based gamut (Finlayson et al.,
2006)
6.93 5.80 6.05
Natural Image Statistics (Gijsenij and Gevers,
2011)
5.19 3.93 4.31
Exemplar-based learning (Joze and Drew,
2012)
4.38 3.43 3.67
Color Cat (CC) (Bani
´
c and Lon
ˇ
cari
´
c, 2015a) 4.22 3.17 3.46
Smart Color Cat (SCC) (Bani
´
c and Lon
ˇ
cari
´
c,
2015b)
4.62 3.52 3.80
Color Dog
W P,GW
(Bani
´
c and Lon
ˇ
cari
´
c, 2015b)
5.27 3.71 4.16
Color Dog
CC
(Bani
´
c and Lon
ˇ
cari
´
c, 2015b) 4.50 2.86 3.50
Color Dog
SCC
(Bani
´
c and Lon
ˇ
cari
´
c, 2015b) 4.80 3.08 3.71
Color Tiger (proposed) 5.61 3.39 4.31
Table 3: Performance of different color constancy methods
on the linear GreyBall dataset (lower median is better).
Algorithm Mean Median Trimean
do nothing 15.62 14.00 14.56
Low-level statistics-based methods
Gray-world (GW) (Buchsbaum, 1980) 13.01 10.96 11.53
White-Patch (WP) (Funt and Shi, 2010) 12.68 10.50 11.25
Shades-of-Gray (Finlayson and Trezzi, 2004) 11.55 9.70 10.23
General Gray-World (Barnard et al., 2002) 11.55 9.70 10.23
1st-order Gray-Edge (Van De Weijer et al.,
2007a)
10.58 8.84 9.18
2nd-order Gray-Edge (Van De Weijer et al.,
2007a)
10.68 9.02 9.40
Learning-based methods
Pixel-based gamut (Finlayson et al., 2006) 11.79 8.88 9.97
Edge-based gamut (Finlayson et al., 2006) 12.78 10.88 11.38
Intersection-based gamut (Finlayson et al.,
2006)
11.81 8.93 10.00
Natural Image Statistics (Gijsenij and Gevers,
2011)
9.87 7.65 8.29
Exemplar-based learning (Joze and Drew,
2012)
7.97 6.46 6.77
Color Cat (CC) (Bani
´
c and Lon
ˇ
cari
´
c, 2015a) 8.73 7.07 7.43
Smart Color Cat (SCC) (Bani
´
c and Lon
ˇ
cari
´
c,
2015b)
8.18 6.28 6.73
Color Dog
W P,GW
(Bani
´
c and Lon
ˇ
cari
´
c, 2015b)
10.27 7.33 8.20
Color Dog
CC
(Bani
´
c and Lon
ˇ
cari
´
c, 2015b) 8.81 5.98 6.97
Color Dog
SCC
(Bani
´
c and Lon
ˇ
cari
´
c, 2015b) 8.51 5.55 6.56
Color Tiger (proposed) 9.51 7.11 7.66
angular error is below the acceptable 3
(Finlayson
et al., 2005; Fredembach and Finlayson, 2008).
4.3 Discussion
Beyond the fact that the proposed method outper-
formed all statistics-based methods and many state-
of-the-art learning-based methods, a far more im-
portant thing to stress here is that it did so with-
out having any ground-truth illumination data avail-
able. Not only does this show the abundance of in-
formation available in even the simplest natural im-
VISAPP 2018 - International Conference on Computer Vision Theory and Applications
186
age statistics, but it also opens a simple and effec-
tive way of achieving highly accurate illumination es-
timation for a given sensor by only providing train-
ing images without ground-truth illumination data.
Skipping the calibration of training images can save
a significant amount of time and in some cases this
can make the proposed method more suitable for
practical applications than other learning-based meth-
ods. Since in production it only executes two of
the fastest statistics-based methods with practically
no memory requirements, namely Gray-world and
White-patch (Cheng et al., 2014), and then performs a
small and constant number of calculations for voting,
the proposed method is hardware-friendly and thus
widely applicable. Unlike Color Dog, the proposed
method is also immune to problems of false ground-
truth data (Zakizadeh et al., 2015). Finally, if the as-
sumptions and steps devised here have led to the de-
scribed results, it is can be assumed that higher accu-
racy could be achieved by using more sophisticated
image statistics, voters, and trimming procedures.
5 CONCLUSIONS AND FUTURE
RESEARCH
A fast and hardware-friendly unsupervised learning-
based method that learns its parameter values from
images with unknown ground-truth illumination has
been proposed. In terms of accuracy the method out-
performs all statistics-based and many state-of-the-art
learning-based methods. This demonstrates how to
achieve highly accurate color constancy for a given
sensor without carrying out the usually time con-
suming calibration of training images. The proposed
method could possibly also be an important step
in color constancy philosophy, especially now when
there are large amounts of uncalibrated images avail-
able on the Internet. Future research will focus on ex-
tracting more useful information from statistics-based
illumination estimations obtained on training images
without ground-truth illumination and on other ways
of outlier removal.
REFERENCES
Aytekin, C¸ ., Nikkanen, J., and Gabbouj, M. (2017). INTEL-
TUT Dataset for Camera Invariant Color Constancy
Research. arXiv preprint arXiv:1703.09778.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2015a). Color Cat: Remember-
ing Colors for Illumination Estimation. Signal Pro-
cessing Letters, IEEE, 22(6):651–655.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2015b). Using the red chro-
maticity for illumination estimation. In Image and
Signal Processing and Analysis (ISPA), 2015 9th In-
ternational Symposium on, pages 131–136. IEEE.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2013). Using the Random
Sprays Retinex Algorithm for Global Illumination Es-
timation. In Proceedings of The Second Croatian
Computer Vision Workshopn (CCVW 2013), pages 3–
7. University of Zagreb Faculty of Electrical Engi-
neering and Computing.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2014a). Color Rabbit: Guid-
ing the Distance of Local Maximums in Illumina-
tion Estimation. In Digital Signal Processing (DSP),
2014 19th International Conference on, pages 345–
350. IEEE.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2014b). Improving the White
patch method by subsampling. In Image Processing
(ICIP), 2014 21st IEEE International Conference on,
pages 605–609. IEEE.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2015a). A Perceptual Measure
of Illumination Estimation Error. In VISAPP, pages
136–143.
Bani
´
c, N. and Lon
ˇ
cari
´
c, S. (2015b). Color Dog: Guiding the
Global Illumination Estimation to Better Accuracy. In
VISAPP, pages 129–135.
Barnard, K., Cardei, V., and Funt, B. (2002). A compar-
ison of computational color constancy algorithms. i:
Methodology and experiments with synthesized data.
Image Processing, IEEE Transactions on, 11(9):972–
984.
Barron, J. T. (2015). Convolutional Color Constancy. In
Proceedings of the IEEE International Conference on
Computer Vision, pages 379–387.
Barron, J. T. and Tsai, Y.-T. (2017). Fast Fourier Color Con-
stancy. In Computer Vision and Pattern Recognition,
2017. CVPR 2017. IEEE Computer Society Confer-
ence on, volume 1. IEEE.
Bianco, S., Cusano, C., and Schettini, R. (2015). Color
Constancy Using CNNs. In Proceedings of the IEEE
Conference on Computer Vision and Pattern Recogni-
tion Workshops, pages 81–89.
Buchsbaum, G. (1980). A spatial processor model for object
colour perception. Journal of The Franklin Institute,
310(1):1–26.
Burkard, R., Dell’Amico, M., and Martello, S. (2012). As-
signment problems: revised reprint. SIAM.
Chakrabarti, A., Hirakawa, K., and Zickler, T. (2012). Color
constancy with spatio-spectral statistics. Pattern Anal-
ysis and Machine Intelligence, IEEE Transactions on,
34(8):1509–1519.
Cheng, D., Abdelhamed, A., Price, B., Cohen, S., and
Brown, M. S. (2016). Two Illuminant Estimation
and User Correction Preference. In Proceedings of
the IEEE Conference on Computer Vision and Pattern
Recognition, pages 469–477.
Cheng, D., Prasad, D. K., and Brown, M. S. (2014). Illu-
minant estimation for color constancy: why spatial-
domain methods work and the role of the color distri-
bution. JOSA A, 31(5):1049–1058.
Cheng, D., Price, B., Cohen, S., and Brown, M. S. (2015).
Effective learning-based illuminant estimation using
Unsupervised Learning for Color Constancy
187
simple features. In Proceedings of the IEEE Con-
ference on Computer Vision and Pattern Recognition,
pages 1000–1008.
Ciurea, F. and Funt, B. (2003). A large image database
for color constancy research. In Color and Imaging
Conference, volume 2003, pages 160–164. Society for
Imaging Science and Technology.
Ebner, M. (2007). Color Constancy. The Wiley-IS&T Se-
ries in Imaging Science and Technology. Wiley.
Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al. (1996).
A density-based algorithm for discovering clusters in
large spatial databases with noise. In Kdd, volume 96,
pages 226–231.
Finlayson, G. D. (2013). Corrected-moment illuminant es-
timation. In Proceedings of the IEEE International
Conference on Computer Vision, pages 1904–1911.
Finlayson, G. D., Hordley, S. D., and Morovic, P. (2005).
Colour constancy using the chromagenic constraint.
In Computer Vision and Pattern Recognition, 2005.
CVPR 2005. IEEE Computer Society Conference on,
volume 1, pages 1079–1086. IEEE.
Finlayson, G. D., Hordley, S. D., and Tastl, I. (2006). Gamut
constrained illuminant estimation. International Jour-
nal of Computer Vision, 67(1):93–109.
Finlayson, G. D. and Trezzi, E. (2004). Shades of gray and
colour constancy. In Color and Imaging Conference,
volume 2004, pages 37–41. Society for Imaging Sci-
ence and Technology.
Finlayson, G. D. and Zakizadeh, R. (2014). Reproduction
angular error: An improved performance metric for
illuminant estimation. perception, 310(1):1–26.
Fredembach, C. and Finlayson, G. (2008). Bright chroma-
genic algorithm for illuminant estimation. Journal of
Imaging Science and Technology, 52(4):40906–1.
Funt, B. and Shi, L. (2010). The rehabilitation of MaxRGB.
In Color and Imaging Conference, volume 2010,
pages 256–259. Society for Imaging Science and
Technology.
Gao, S., Han, W., Yang, K., Li, C., and Li, Y. (2014). Ef-
ficient color constancy with local surface reflectance
statistics. In European Conference on Computer Vi-
sion, pages 158–173. Springer.
Gao, S.-B., Zhang, M., Li, C.-Y., and Li, Y.-J. (2016). Im-
proving Color Constancy by Discounting the Varia-
tion of Camera Spectral Sensitivity. arXiv preprint
arXiv:1609.01670.
Gehler, P. V., Rother, C., Blake, A., Minka, T., and Sharp, T.
(2008). Bayesian color constancy revisited. In Com-
puter Vision and Pattern Recognition, 2008. CVPR
2008. IEEE Conference on, pages 1–8. IEEE.
Gijsenij, A. and Gevers, T. (2007). Color Constancy using
Natural Image Statistics. In CVPR, pages 1–8.
Gijsenij, A. and Gevers, T. (2011). Color constancy using
natural image statistics and scene semantics. IEEE
Transactions on Pattern Analysis and Machine Intel-
ligence, 33(4):687–698.
Gijsenij, A., Gevers, T., and Lucassen, M. P. (2009). Per-
ceptual analysis of distance measures for color con-
stancy algorithms. JOSA A, 26(10):2243–2256.
Gijsenij, A., Gevers, T., and Van De Weijer, J. (2010). Gen-
eralized gamut mapping using image derivative struc-
tures for color constancy. International Journal of
Computer Vision, 86(2):127–139.
Gijsenij, A., Gevers, T., and Van De Weijer, J. (2011).
Computational color constancy: Survey and exper-
iments. Image Processing, IEEE Transactions on,
20(9):2475–2489.
Hordley, S. D. and Finlayson, G. D. (2004). Re-evaluating
colour constancy algorithms. In Pattern Recognition,
2004. ICPR 2004. Proceedings of the 17th Interna-
tional Conference on, volume 1, pages 76–79. IEEE.
Hu, Y., Wang, B., and Lin, S. (2017). Fully Convolutional
Color Constancy with Confidence-weighted Pooling.
In Computer Vision and Pattern Recognition, 2017.
CVPR 2017. IEEE Conference on, pages 4085–4094.
IEEE.
Japkowicz, N. and Shah, M. (2011). Evaluating learning
algorithms: a classification perspective. Cambridge
University Press.
Joze, H. R. V. and Drew, M. S. (2012). Exemplar-Based
Colour Constancy. In British Machine Vision Confer-
ence, pages 1–12.
Kim, S. J., Lin, H. T., Lu, Z., S
¨
usstrunk, S., Lin, S., and
Brown, M. S. (2012). A new in-camera imaging
model for color computer vision and its application.
IEEE Transactions on Pattern Analysis and Machine
Intelligence, 34(12):2289–2302.
L. Shi, B. F. (2015). Re-processed Version of the Gehler
Color Constancy Dataset of 568 Images.
Land, E. H. (1977). The retinex theory of color vision. Sci-
entific America.
Lynch, S. E., Drew, M. S., and Finlayson, k. G. D. (2013).
Colour Constancy from Both Sides of the Shadow
Edge. In Color and Photometry in Computer Vision
Workshop at the International Conference on Com-
puter Vision. IEEE.
Mazin, B., Delon, J., and Gousseau, Y. (2015). Estima-
tion of illuminants from projections on the planck-
ian locus. IEEE Transactions on Image Processing,
24(6):1944–1955.
Schanda, J. (2007). Colorimetry: Understanding the CIE
System. John Wiley & Sons.
Shi, W., Loy, C. C., and Tang, X. (2016). Deep Specialized
Network for Illuminant Estimation. In European Con-
ference on Computer Vision, pages 371–387. Springer.
Van De Weijer, J., Gevers, T., and Gijsenij, A. (2007a).
Edge-based color constancy. Image Processing, IEEE
Transactions on, 16(9):2207–2214.
Van De Weijer, J., Schmid, C., and Verbeek, J. (2007b). Us-
ing high-level visual information for color constancy.
In Computer Vision, 2007. ICCV 2007. IEEE 11th In-
ternational Conference on, pages 1–8. IEEE.
Vassilvitskii, S. (2007). K-means: Algorithms, Analyses,
Experiments. Stanford University.
Zakizadeh, R., Brown, M. S., and Finlayson, G. D. (2015).
A Hybrid Strategy For Illuminant Estimation Target-
ing Hard Images. In Proceedings of the IEEE Inter-
national Conference on Computer Vision Workshops,
pages 16–23.
VISAPP 2018 - International Conference on Computer Vision Theory and Applications
188