Estimating Spectral Sensitivity of Human Observer
for Multiplex Image Projection
Koji Muramatsu, Fumihiko Sakaue and Jun Sato
Department of Computer Science and Engineering, Nagoya Institute of Technology, Nagoya 466-8555, Japan
Keywords:
Spectral Sensitivity, Multi-band Imaging, Multiplex Image Projection.
Abstract:
In this paper, we propose an efficient method for estimating the spectral sensitivity of human retina. The pro-
posed method is used for the multiplex image projection by using multi-band projectors. The multiplex image
projection can represent different images to different observers by using the differences of spectral sensitivi-
ties of these observers. Although precise spectral sensitivity is required for the multiplex image projection, the
ordinary estimation methods require a lot of time for estimation. Thus, we in this paper propose an efficient
method for estimating the spectral sensitivity of human eye. The experimental results show the efficiency of
the proposed method.
1 INTRODUCTION
Recently, multi-band imaging is widely studied in the
field of computer vision, image processing and many
other applications (Miyake et al., 1999; Tominaga,
1999; Monno et al., 2012; Miao and Qi, 2006; Par-
mar and Reeves, 2010; Park et al., 2007; Ng and
Allebach, 2006). In particular, multi-band cameras
are used for obtaining precise spectral information of
natural scene. From the multi-band images taken by
the multi-band cameras, we can obtain valuable color
information which can not be obtained from ordinary
RGB color cameras.
On the other hand, multi-band displaying is also
studied in recent years (INFITEC, 2002; Nonoyama
et al., 2013; Miyazaki et al., 2013). Nonoyama et
al.(Nonoyama et al., 2013) showed that it is possible
to represent different images to different observers si-
multaneously by controlling the spectral distribution
of displayed images precisely by using a multi-band
projector. It is called multiplex image projection. The
difference of observation is occurred by the difference
of the spectral sensitivity of observers. The method
can be applied to various applications, such as 3D dis-
playing system without using special glasses.
In order to represent different images accurately
in the multiplex image projection, it is necessary to
obtain precise spectral sensitivities of the observers.
If the observer is a camera or digital imaging devices,
we can obtain their spectral sensitivity directly from
their observations. However, it is very difficult to ob-
tain the spectral sensitivity of human retina, since we
cannot obtain the observed signals in human retina
numerically. Thus, the efficient method for estimat-
ing the spectral sensitivity of human visual system is
required for multiplex image projection.
Traditionally, the spectral sensitivity of human vi-
sual system has been measured by using severalmeth-
ods, such as brightness matching, flicker photometry,
etc. The CIE published the standard spectral sensi-
tivity of human visual system. However, the ordinary
measurement method needs a lot of time to obtain pre-
cise spectral sensitivity, since many observations are
required to obtain the precise spectral sensitivity.
In this paper, we propose a new parametrization of
the spectral sensitivity of human visual systems. Fur-
thermore, an efficient method for measuring the spec-
tral sensitivity based on this new parametrization is
proposed. In the proposed method, the measurement
time of human spectral sensitivity is much smaller
than that of the ordinary measurement method.
2 METAMERISM
We first consider the relationship among light spec-
trum, observed signals and spectral sensitivity of ob-
servers. In general, ordinary sensors, such as hu-
man retina and CCD, cannot receive light spectrum
directly. They encode the light spectrum to a lim-
ited number of signals by using their receivers. The
Muramatsu, K., Sakaue, F. and Sato, J.
Estimating Spectral Sensitivity of Human Observer for Multiplex Image Projection.
DOI: 10.5220/0005722701830191
In Proceedings of the 11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2016) - Volume 4: VISAPP, pages 183-191
ISBN: 978-989-758-175-5
Copyright
c
 2016 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved
183

receivers have characteristic spectral sensitivity, and
thus, received signals depend on their spectral sensi-
tivity and input light spectrum. Each receiver in the
sensors has different spectral sensitivity, and thus, the
encoded signals are different from each other. The
combination of the signals determines observed col-
ors. For example, a human retina has 3 different re-
ceivers, ¯x(λ), ¯y(λ) and ¯z(λ), and each receiver en-
codes the spectrum of light E(λ) into X, Y and Z as
follows:
X = K
Z
700
400
E(λ) ¯x(λ)dλ (1)
Y = K
Z
700
400
E(λ) ¯y(λ)dλ (2)
Z = K
Z
700
400
E(λ)¯z(λ)dλ (3)
where K is a constant for normalization. Note that,
human retina can encode only from about 400nm to
700 nm, and thus, the range of integral is limited as
shown in the above equations.
As described in the above equations, DoF (degree
of freedom) of received signals is much smaller than
that of the input light spectrum. Therefore, the same
set of signals X, Y and Z can be observed, even if
the input light spectrum is different. In this case, we
cannot distinguish these lights, and we recognize that
the input lights have the same color. This property
is called as metamerism and such colors are called
as metameric colors. The metamerism is important
property in this paper. It is used for multiplex image
projection and measurement of human retina sensitiv-
ity.
3 MULTIPLEX IMAGE
PROJECTION
We next consider multiplex image projection pro-
posed by Nonoyama et al.(Nonoyama et al., 2013).
Equations from (1) to (3) represents that there are a
lot of metameric colors for each color, and thus, we
can choose many different light spectrum to represent
a specific color. In addition, the received signal de-
pends not only on input light spectrum, but also on the
spectral sensitivity of the observer. As a result, a pair
of metameric colors for a certain observer may not be
metameric for other observers. Thus, we can change
the observed color for a particular observer without
changing the observed color of other observers by us-
ing the metamerism.
If we can control whole light spectrum in image
representation, we can represent arbitrary colors one
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
380 480 580 680 780
Wavelength[nm]
Spectrum
(a) Ordinary projector
0.0
1.0
2.0
3.0
4.0
5.0
380 480 580 680
Wavelength[nm]
Intensity
(b) Multi-band projector
Figure 1: Light spectrum of ordinary and multi-band pro-
jectors.
by one. For example, we can emit a special light
which is recognized as red by an observer, while it
is recognized as blue by the other observer. Thus, we
can represent different images to different observers.
However, the existing ordinary displaying de-
vices, such as displays and projectors, cannot control
the spectrum of light freely. This is because these
devices represent colors by combining few specific
colors, i.e. red, green and blue. In multiplex im-
age projection, they do not have sufficient ability for
exploiting metamerism. Therefore, we construct a
multi-band projector in order to achieve multiplex im-
age projection. The multi-band projector can project
much more color channels to represent colors, and
thus, its DoF of light spectrum is much higher than
that of the ordinary projectors.
Figure 1 shows the example of light spectrum
of an ordinary projector and a multi-band projector.
Each projector represents colors by the combination
of each band lights. In Fig.1, the ordinary projector
can emit three wide-band lights, while the multi-band
projector can emit nine narrow-band lights. There-
fore, the DoF of light spectrum emitted from the
multi-band projector is much higher than that of the
ordinary projector.
4 MULTIPLEX IMAGE
ENCODING/DECODING
By using the multi-band projector, we encode mul-
tiple images into a single multi-band image. In this
section, we first explain decoding of multi-band im-
ages, and then explain encoding of multiple images.
VISAPP 2016 - International Conference on Computer Vision Theory and Applications
184

4.1 Image Decoding
We first explain image decoding from a particular
light spectrum. In fact, we do not need any processes
for the image decoding, since the image decoding
can be realized just by observing images by sensors,
which have different spectral sensitivities. Now let us
consider the projection and observation from an N-
band projector. Let E
i
(λ) and b
i
(i = 1,2,··· , N) be
the light spectrum and the radiance of i-th band re-
spectively. Suppose the projected light is observed by
M different receivers, whose spectral sensitivities are
x
j
(λ) ( j = 1, ··· ,M). Then, the observed signal X
j
of
the j-th receiver can be described as follows:
X
j
=
N
∑
i=1
Z
b
i
E
i
(λ)x
i
(λ)dλ. (4)
Let C
ji
be an observed signal when b
i
= 1 in Eq.(4) as
follows:
C
ji
=
Z
E
i
(λ)x
i
(λ)dλ. (5)
Then, Eq.(4) is rewritten by using C
ji
as follows:
X
j
=
N
∑
i=1
C
ji
b
i
(6)
From these equations, the relationship between the ra-
diance b
i
of each band and the received signal X
j
of
each sensor can be described as follows:




X
1
X
2
.
.
.
X
M




=




C
11
C
12
··· C
1N
C
21
C
22
··· C
2N
.
.
.
.
.
.
.
.
.
.
.
.
C
M1
C
M2
··· C
MN








b
1
b
2
.
.
.
b
N




(7)
We call the matrix in the right side of the above equa-
tion a spectral sensitivity matrix. The equation (7)
indicates that we can control observed signals X
j
by
changingthe projecting radiance b
i
of the N-band pro-
jector.
4.2 Multiplex Image Encoding
We next consider the encoding of multiple images
into a single multi-band image. In order to achieve
multiplex image encoding, we solve Eq.(7) and obtain
projection intensities b
i
from objective(observed)sig-
nals X
j
and the spectral sensitivity matrix C
ji
by us-
ing least means square method. The equation (7) has
one or more than one solutions, if N ≥ M. However,
the solution b
i
includes negative values in general. In
ordinary case, general projectors cannot project neg-
ative intensities, and thus, we should avoid this prob-
lem. For this objective, we use virtual negative in-
tensities, where the values lower than a virtual zero
0
positive
negative
x
Virtual
zerolevel
Figure 2: Virtual negative intensities.
level x are regarded as negative intensities as shown
in Fig.2.
After this zero level correction, the radiance of the
projection image is computed as follows:
{
ˆ
b
1
,··· ,
ˆ
b
N
} = argmin
{b
1
,···,b
N
}
||X
j
−
N
∑
i=1
C
ji
b
i
||
subject to 0 ≤ b
j
≤ I
max
(8)
where I
max
is maximum value of radians, such as 255
for an 8 bit image.
5 SPECTRAL SENSITIVITY
ESTIMATION
The multiplex image projection can be applied to not
only cameras vs humans, but also a human vs the
other human. Furthermore, it may be applied to the
left eye vs the right eye if there are sufficient differ-
ence among their spectral sensitivities. In order to re-
alize them, we should measure precise spectral sensi-
tivities of these receivers. However, ordinary estima-
tion method requires a lot of time, e.g. 1 or 2 hours,
to estimate the sensitivity and thus, it is not easy to
estimate the spectral sensitivity of many people pre-
cisely. In order to estimate the spectral sensitivity in
short time, we propose a parametric representation of
the spectral sensitivity. Furthermore, we propose a
new estimation method of spectral sensitivities by us-
ing the new representation method. By using the pro-
posed method, the time required for the estimation be-
comes much smaller than that of the ordinary method,
and thus, we can estimate the spectral sensitivity of
many people easily.
5.1 Measurement of Spectral Sensitivity
We first explain a general measurement of a spec-
tral sensitivities of human retina. As described in
the previous sections, a spectral sensitivity of the hu-
man retina is determined by three types of cone cell.
Therefore, we should estimate characteristics sensi-
tivities of them to estimate the spectral sensitivity.
Estimating Spectral Sensitivity of Human Observer for Multiplex Image Projection
185

Figure 3: Human spectral sensitivity measurement: radi-
ance of the reference lights are controlled, so that they be-
come a metameric color of the target light.
However, it is not easy to represent the characteris-
tic numerically because the sense of human cannot be
extracted directly. Thus, the metamerism described at
2 is used for measuring the spectral sensitivities.
As described in 2, a human cannot classify the dif-
ference of light spectrum when received values X, Y
and Z are the same. By using this property, we can
estimate relative relationship among spectral sensitiv-
ities.
For this objective, several reference lights and a
target light is observed simultaneously as shown in
Fig.3. The number of reference light is 3 in gen-
eral. The spectrum of the targetlight and the reference
lights are known. In this measurement system, a hu-
man observer observes the target light and combined
reference light. Then, the radiance of each reference
light is controlled, so that the reference light is ob-
served as a metameric color of the target light. When
the reference light becomes a metameric color of the
target light, the relationship among the target light E
T
,
the reference lights E
Ri
and the spectral sensitivity f
j
of the target human observer can be described as fol-
lows:
Z
700
400
E
T
(λ) f
j
(λ)dλ =
N
∑
i=1
b
i
Z
700
400
E
Ri
(λ) f
j
(λ)dλ
(9)
where E
T
and E
Ri
denote light spectrum of the target
light and the i-th reference light. N denotes a num-
ber of reference lights, and f
j
indicates j-th spectral
sensitivity of the target human observer. In ordinary
case, human observer has three types of cone cell, and
thus, j = {1,2, 3}. b
i
denotes controlled radiance of
i-th light.
When the target person observes a pair of
metameric colors, we have three equation on f
j
from
Eq.(9). Thus, we can estimate the spectral sensitivity
f
j
from several number of metameric color observa-
tions.
Note that we need large number of equations when
a precise spectral sensitivity is required. For example,
if the spectral sensitivity is sampled for every 10 nm,
31 variables are required for representing the whole
Figure 4: Spectral sensitivity defined by CIE (1964).
Figure 5: Representation by using Gaussian Mixture
Model(GMM).
spectral f
j
. Thus, 31 pairs of metameric colors are re-
quired for estimating the sensitivity. In addition, it is
not easy to control radiance for obtaining metameric
colors. Therefore, long measuring time is required for
estimating accurate spectral sensitivity, and thus more
efficient sensitivity estimation method is required.
5.2 Spectral Sensitivity Representation
by using GMM
In order to realize efficient estimation of human spec-
tral sensitivity, we propose a new representation of
the spectral sensitivity. Fig.4 shows representative
spectral sensitivity defined by CIE(1964). As we
can see in Fig.4, the spectral sensitivity of human
observer is similar to the Gaussian function. Thus,
we in this paper utilize Gaussian Mixture Model
(GMM)(Reynolds, 1992) for representing the spectral
sensitivity of human retina. The GMM can represent
general functions by the combination of Gauss func-
tion as shown in Fig.5. Especially, the spectral sensi-
tivity can be represented by small number of Gaussian
functions, since the shapes of spectral sensitivities are
similar to the Gaussian function.
By using the GMM, a spectral sensitivities f
j
(λ)
can be described as follows:
f
j
(λ) =
K
∑
k=1
a
kj
q
2πσ
2
kj
e
−
(λ−µ
kj
)
2
2σ
2
kj
(10)
VISAPP 2016 - International Conference on Computer Vision Theory and Applications
186

where K is a number of Gaussian functions, a
kj
is
scale parameter of k-th Gaussian function for j-th sen-
sitivity, µ
kj
and σ
kj
are a center and standard deviation
the function. By using the GMM, the spectral sensi-
tivity can be represented by 3K parameters. Empiri-
cally, the number of Gaussian K can be small for rep-
resenting the spectral sensitivities, and thus, the spec-
tral sensitivity can be estimated stably and efficiently.
5.3 Sensitivity Estimation by using
GMM
We next consider the estimation of spectral sensitivity
by using the GMM. Let f(a
kj
,µ
kj
,σ
kj
) be a spectral
sensitivity with parameters a
kj
,µ
kj
,σ
kj
. Then, Eq.(9)
can be rewritten as follows:
Z
700
400
E
T
(λ) f
j
(a
kj
,µ
kj
,σ
kj
)dλ
=
N
∑
i=1
b
i
Z
700
400
E
Ri
(λ) f
j
(a
kj
,µ
kj
,σ
kj
)dλ (11)
From this equation, an evaluation function E
GMM
can
be defined as follows:
E
GMM
=
L
∑
l=1
|
Z
700
400
E
l
(λ) f
j
(a
kj
,µ
kj
,σ
kj
)dλ
−
N
∑
i=1
b
li
Z
700
400
E
Ri
(λ) f
j
(a
kj
,µ
kj
,σ
kj
)dλ|
2
(12)
where L is a number of observed metameric colors.
Then, by minimizing the cost function E
GMM
, we can
estimate the parameters for representing target spec-
tral sensitivities. The DoF of spectral sensitivity is
3× 3K when j = {1,2, 3}, and thus, we can estimate
the spectral sensitivities, if 9K ≤ 3L. For example, if
the number of Gaussian K is 2, the spectral sensitiv-
ities can be estimated from 6 or more than 6 pairs of
metameric colors.
As described in this section, we can estimate the
spectral sensitivity from much smaller number of ob-
servations by using GMM. Therefore, estimation time
can be much smaller than that of the ordinary estima-
tion method.
5.4 Spectral Sensitivity Representation
by using Linear Bases
We next consider another representation of spectral
sensitivities under general statistical assumptions. In
the previous sections, we described that the spectrum
sensitivities of humans are different from each other.
Although this assumption is valid, the differences are
not so large, and thus, we can represent the sensitivi-
ties by combining the small number of linear bases.
In this paper, we compute the linear bases by using
PCA (Principal Component Analysis). Let f
i
denote a
spectral sensitivity of i-th person. The covariance ma-
trix Σ of spectral sensitivities derived from K inputs is
computed as follows:
Σ =
1
K
K
∑
i=1
(f
i
−
¯
f)(f
i
−
¯
f)
⊤
(13)
where
¯
f = 1/K
∑
K
i=1
f
i
and it is an average of the sen-
sitivities. The covariance matrix is decomposed into
an orthogonal linear bases Q and a diagonal matrix Λ
by using the eigenvalue decomposition as follows:
σ = QΛQ
⊤
(14)
where, Λ = diag(λ
1
,··· ,λ
K
) is from K eigenvalues
λ
1
≥ ··· ≥ λ
K
, and Q = [q
1
,··· , q
K
] is a set of eigen-
vectors q
i
.
By using the eigenvectors q
i
, the sensitivity f can
be represented linearly as follows:
f =
J
∑
i=1
c
i
q
i
+
¯
f (15)
where J(≤ K) is the number of bases and c
i
is the
coefficient of the i-th base q
i
. This equation indicates
that the spectral sensitivity f can be represented by J
coefficients.
The coefficients can be estimated from several
number of observations like the previous estimation
methods. By using the linear representation, Eq.(9)
can be rewritten as follows:
Z
700
400
E
T
(λ) f
j
(c,λ)dλ =
N
∑
i=1
b
i
Z
700
400
E
Ri
(λ) f
j
(c,λ)dλ
(16)
where f(c) indicates the spectral sensitivity from a set
of coefficients c. Thus, an evaluation function E
lin
for
linear estimation can be described as follows:
E
lin
=
L
∑
l=1
|
Z
700
400
E
T
(λ) f
j
(c,λ)dλ
−
N
∑
i=1
b
i
Z
700
400
E
Ri
(λ) f
j
(c,λ)dλ| (17)
By minimizing E
lin
, a set of coefficients c which rep-
resents the spectral sensitivity f can be estimated.
The linear representation of the spectral sensitiv-
ity is efficient when we have a data set of spectral
sensitivities. By using the linear method, we can es-
timate the spectral sensitivity with a small computa-
tional cost.
Estimating Spectral Sensitivity of Human Observer for Multiplex Image Projection
187

Figure 6: Multi-band projector: This projector is con-
structed by 8 projectors and 8 different band-pass filters.
Each filter is equipped in front of each projector.
Figure 7: Spectral characteristics of the multi-band projec-
tor. Each line indicates spectral distribution of each band.
6 EXPERIMENTAL RESULTS
6.1 Environment
In this section, we show experimental results by using
our proposed method. At first, we show experimen-
tal environment for these experiments. In the series
of experiments, we used 8-band multi-band projector
as shown in Fig.6. This projector was constructed by
8-projectors and each projector equipped a different
band-pass filter. The light spectrum of each projec-
tor with band-pass filter is shown in Fig.7 The pro-
jectors are calibrated in advance by using homogra-
phy estimation, and thus, they can project arbitrary
images onto the same area on the screen simultane-
ously. In general, a characteristic of projected inten-
sities are not nonlinear because projected images in-
clude gamma correction. Therefore, these gamma pa-
rameters were calibrated beforehand, and thus, linear
image projection was achieved
6.2 Sensitivity Estimation Result
We first show estimation result of a human retina’s
spectral sensitivity. In this experiment, the spectral
sensitivity of the human’s retina was estimated by
(a) general measurement method described in 5.1,
(b) proposed method using GMM and (c) proposed
(i) Principal bases for 1st sensitivity (f
1
)
(ii) Principal bases for 2nd sensitivity (f
2
)
(iii) Principal bases for 3rd sensitivity (f
3
)
Figure 8: The average and the principal bases of three spec-
tral sensitivities, f
1
, f
2
and f
3
.
method using PCA.
In general method (a), 3 narrow band lights were
used for reference lights. As the target light, 31 lights
were used, and metameric colors for each target light
were observed by controlling the radiance of refer-
ence lights. From these observations, 3 spectral sen-
sitivities were derived.
In the proposed method (b) and (c), 8 fixed lights
were used. One of the 8 lights was used as the tar-
get light and the other 7 lights were used as reference
lights. The target light was changed in order and 8
sets of metameric colors were observed, and the 3
spectral sensitivities f
1
, f
2
, f
3
were estimated from the
metameric colors.
At first, the spectral sensitivities of 8 persons were
estimated by the general method (a), and their aver-
age and principal bases were computed by using the
PCA. The estimated bases and the average are shown
in Fig 8. Then, three principal bases and the average
were used for method (c), Thus, 3 parameters were
estimated in the method (c).
Figure 9 shows the spectral sensitivity of a person
estimated from these three methods. In this figure, the
solid lines indicate results from (a) general method,
the dashed lines indicate results from (b) GMM, and
the dash-dot lines indicate results from (c) PCA. Table
1 shows the time required for these estimations and
VISAPP 2016 - International Conference on Computer Vision Theory and Applications
188

Figure 9: Estimated spectral sensitivity of the target human.
The solid lines show results from the general method, the
dashed lines show results from the GMM and dash-dot lines
show results from the PCA.
Table 1: Estimation time for the proposed method and the
ordinary method.
ordinary GMM PCA
# of observations 60 18 18
time required 5 hours 40 min 40 min
the number of metameric colors for each estimation
method. Note, minimum number of metameric colors
required for method (b) and method (c) is different,
since the number of parameters for these estimations
is different. We however used the same number of
the metameric colors for stable estimation in this ex-
periment. Thus, the number of metameric colors and
measurement time for (b) and (c) is the same in the
Tab. 1.
As shown in Figure 9, the results from the pro-
posed methods are very similar to the results from
the ordinary method. These results indicate that the
GMM and the linear bases from PCA can represent
the sensitivity of human retina efficiently, and the pro-
posed methods can work well, even if the number of
metameric colors is small. In addition, Table 1 shows
that the estimation time of the proposed methods is
much smaller than that of the ordinary method. These
results show that the proposed methods can estimate
spectral sensitivity of human retina more efficiently
than the ordinary measurement method.
Figure 10 shows the spectral sensitivity of another
person estimated from the proposed methods and the
ordinary method. These results shows that the pro-
posed method can estimate spectral sensitivity for ar-
bitrary human observers. Furthermore, we can ob-
serve the difference between the estimated result in
Fig.9 and Fig.10. The fact indicates that the proposed
method can estimate slight sensitivity difference be-
tween two different human observers. Furthermore,
the results shows that there is explicit deference in the
spectral sensitivity of two different human observers.
Therefore, we can use the proposed method for realiz-
ing multiplex image projection for human vs human.
Figure 10: Spectral sensitivity estimated by our proposed
methods and the ordinary method.
6.3 Multiplex Image Projection Result
We next show the results of multiplex image projec-
tion using the estimated spectral sensitivities. In this
experiment, multi-band images were synthesized for
a camera and a target human. The multi-band im-
ages were synthesized from a known camera spec-
tral sensitivity and estimated sensitivity of the human
retina. The spectral sensitivity of the human retina
was estimated by the proposed method and the ordi-
nary method. Result from the ordinary method was
used as ground-truth.
For comparison, multi-band images were syn-
thesized from our two proposed estimation results,
ground-truth and CIE (1964) color function. In this
case, three signals, X, Y and Z, were exchanged to
RGB and displayed as an RGB image. These images
were projected onto the screen and observed by the
target human and the target camera.
Figure 11 shows objective images and observed
results. Note that the observed results of human retina
cannot be represented directly, and thus, these results
show images observed by the ground-truth spectral
sensitivity. From these results, we confirm that the
appearances of these observation results are similar
to the objective images. In particular the observa-
tion results in (d) is very close to the objective images
in (a). This is because the multi-band images were
synthesized from ground-truth spectral sensitivity in
(d). The results indicate that multiplex image projec-
tion works well, if the spectral sensitivities of the ob-
servers are known. The observation results in (b) and
(c) were derived from the proposed methods. In these
results, observed images are slightly different from
the objective images in (a). The difference occurred
from the difference between the estimated spectral
sensitivity and the ground truth spectral sensitivity as
shown in Fig.9. The results indicate that multiplex
image projection is very sensitive to the difference of
spectral sensitivity. However, multi-band images can
be roughly synthesized from our estimated results as
shown in (b) and (c).
The observation results in (e) were derived from
Estimating Spectral Sensitivity of Human Observer for Multiplex Image Projection
189

(a) Objective Images (b) Results by GMM (c) Results by PCA
(d) Results by General
method
(e) Result by CIE
(i) Images for camera observations in first experiment.
(ii) Images for human observations in first experiment.
(iii) Images for camera observations in second experiment.
(iv) Images for human observations in second experiment.
Figure 11: Objective images and observed results. First and second rows shows experimental results from first experiment
and third and fourth rows shows results from second experiment. The column (a) shows objective images for a camera and a
human. The columns (b) and (c) show observed results of a multi-band image generated from the spectral sensitivity estimated
by the proposed method. The column (d) shows those from the ordinary method. The column (e) shows those from the CIE
standard function.
the CIE color function. In this result, human obser-
vations are very different from the objective images.
This is because the CIE color function is very differ-
ent from the spectral sensitivity of the target person,
and thus, multi-band images could not be synthesized
properly. This means that the spectral sensitivity of
human retina is different one by one, and thus, we
should estimate individual spectral sensitivity for re-
alizing proper multiplex image projection. Although
the proposed method cannot measure the spectral sen-
sitivity completely, the measurement time is much
shorter than the ordinary measurement method, and
thus, our method is very efficient for realizing multi-
plex image projection.
6.4 Multiplex Image Projection for
Human vs Human
Finally, we show multiplex image projection results
for human vs human. In this experiment, the CIE
standard function is used for the sensitivity of a hu-
man and the estimated sensitivity is used for the other
human. The synthesized multi-bandimage is virtually
observed by the CIE color function and the estimated
spectral sensitivity respectively. Figure 12 shows im-
ages observed by the CIE color function and the target
human. As shown in these results, the two humans
observed different images, since their spectral sensi-
tivities were different from each other. The results
VISAPP 2016 - International Conference on Computer Vision Theory and Applications
190

(i) for target human
(ii) for CIE
(a) Objective images.
(i) target human
(ii) CIE
(b) Observed images.
Figure 12: Multiplex image projection for human vs human.
(a) shows objective images for two humans, and (b) shows
images observed by these two humans.
indicates that we can present different images to dif-
ferent human observers, if they have different spectral
sensitivities.
7 CONCLUSIONS
In this paper, we proposed a new method for estimat-
ing the spectral sensitivity of human observers. In
the proposed method, we represent the spectral sen-
sitivity function by using GMM and linear combin-
ing of principal bases. As a result, the number of pa-
rameters for representing the spectral sensitivity was
extremely decreased. Therefore, estimation time of
spectral sensitivity is also decreased from 5 hours to
40 minutes. The experimental results show our pro-
posed method is useful for realizing multiplex im-
age projection by using a multi-band projector. We
will consider more efficient and accurate estimation
method in future work.
REFERENCES
INFITEC (2002). Infitec-excellence in 3d. http://
www.infitec.net/.
Miao, L. and Qi, H. (2006). The design and evaluation of a
generic method for generating mosaicked multispec-
tral filter arrays. IEEE Trans. on Image Processing,
15(9):2780–2791.
Miyake, Y., Yokoyama, Y., Tsumura, N., Haneishi, H.,
Miyata, K., and Hayashi, J. (1999). Color imaging:
Device-independent color. In Color Hardcopy and
Graphic Arts IV (Proc. SPIE), volume 3648, pages
218–225.
Miyazaki, D., Takahashi, K., Baba, M., Aoki, H., Fu-
rukawa, R., Aoyama, M., and Hiura, S. (2013). Mix-
ing paints for generating metamerism art under 2
lights and 3 object colors. In The IEEE International
Conference on Computer Vision (ICCV) Workshops.
Monno, Y., Kitao, T., Tanaka, M., and Okutomi, M. (2012).
Optimal spectral sensitivity functions for a single-
camera one-shot multispectral imaging system. In
Proc. IEEE International Conference on Image Pro-
cessing (ICIP2012), pages 2137–2140.
Ng, D. and Allebach, J. P. (2006). A subspace match-
ing color filter design methodology for a multispectral
imaging system. IEEE Trans. on Image Processing,
15(9):2631–2643.
Nonoyama, M., Sakaue, F., and Sato, J. (2013). Multiplex
image projection using multi-band projectors. In The
IEEE International Conference on Computer Vision
(ICCV) Workshops.
Park, J., Lee, M., Grossberg, M. D., and Nayar, S. K.
(2007). Multispectral imaging using multiplexed il-
lumination. In Proc. of IEEE Int. Conf. on Computer
Vision (ICCV), pages 1–8.
Parmar, M. and Reeves, S. J. (2010). Selection of opti-
mal spectral sensitivity functions for color filter ar-
rays. IEEE Trans. on Image Processing, 19(12):3190–
3203.
Reynolds, D. (1992). A Gaussian Mixture modeling Ap-
proach to Text-independent Speaker Identification.
PhD thesis, Georgia Institute of Technology.
Tominaga, S. (1999). Spectral imaging by a multichannel
camera. J. Electron. Imaging, 8(4):332–341.
Estimating Spectral Sensitivity of Human Observer for Multiplex Image Projection
191