Experimental Evaluation of Bayesian Image Reconstruction Combined
with Spatial-Superresolution and Spectral Reflectance Recovery
Yusuke Murayama, Pengchang Zhang and Ari Ide-Ektessabi
Graduate School of Engineering, Kyoto University, Kyoto, Japan
Keywords:
Multispectral Image, Spectral Reflectance Recovery, Image Superresolution, Bayesian Estimation, Digital
Archiving.
Abstract:
Acquisition of a multispectral image and analysis of the object based on spectral information recovered from
the image has recently received attention in digital archiving of cultural assets. However multispectral imaging
faces such problems as long image acquisition time and severe registration between band images. In order to
solve them, we have proposed an extended method combining Bayesian image superresolution with spectral
reflectance recovery. In this study we evaluated quantitatively the performance of the proposed technique
using a typical 6-band multispectral scanner and a Japanese painting. The accuracy of recovered spectral
reflectance was investigated with respect to the ratio of the capturing resolution to the recovering resolution.
The experimental result indicated that the spatial resolution can be increased by around 1.7 times, which means
image capturing time can be reduced almost by one third and besides the angle of view can be extended by 1.7
times.
1 INTRODUCTION
A multispectral imaging device such as a multispec-
tral camera or scanner is a preferable tool for digi-
tal archiving of cultural assets: paintings, documents,
textile fabrics and other art works. The first reason
is that a multispectral image has the ability to pro-
duce color with higher-fidelity than a commonly-used
trichromatic image (Yamaguchi et al., 2002). Sec-
ond, the higher spectral resolution of a multispectral
image enables recovering spectral reflectance in vis-
ible region of the object and then analyzing spectro-
scopic properties (DiCarlo and Wandell, 2003; Shi-
mano et al., 2007). One example is pigment identifi-
cation based on the recovered spectral reflectance of
paintings (Pelagotti and Mastio, 2008; Toque et al.,
2009), and such applications have recently received
increasing attention.
A typical multispectral camera is made up of a
monochromatic camera, light source for illuminant
and band-pass filters which transmit radiation of se-
lected wavelengths (Fukuda et al., 2005; Shimano
et al., 2007). Similarly, a typical multispectral scan-
ner can be assembled from monochromatic scanner
(Toque et al., 2009). Band images represented as
monochromatic images are captured sequentially by
attaching different band-pass filters to the front of the
camera, thus forming a multispectral image.While the
band number of a trichromatic camera or scanner is
fixed to three of red, green, and blue, that of a multi-
spectral camera or scanner can be easily increased as
necessary.
Unfortunately, long acquisition time and image
registration make multispectral image acquisition a
demanding work. A half dozen to a dozen of color
filters are required to recovery spectral reflectance so
it takes a time several or more times longer to obtain a
multispectral image than capturing a trichromatic im-
age. The situation becomes even worse when deal-
ing with large-size objects or high spatial resolutions.
Another problem is subpixel level position shifts be-
tween band images caused by the small translation of
camera in changing a filter or the mechanical error of
starting position of scanner. Such misregistration de-
clines color accuracy especially along edge lines, and
leads to incorrect recovery of spectral reflectance.
In order to overcome them, we have proposed
an extended method for recovering spectral re-
flectance and increasing spatial resolution simultane-
ously, namely Bayesian image reconstruction com-
bined with spatial-superresolution and spectral re-
flectance recovery (Murayama and Ide-Ektessabi,
2012). Image superresolution refers to an image pro-
cessing technique which increases the spatial resolu-
139
Murayama Y., Zhang P. and Ide-Ektessabi A..
Experimental Evaluation of Bayesian Image Reconstruction Combined with Spatial-Superresolution and Spectral Reflectance Recovery.
DOI: 10.5220/0004346001390142
In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP-2013), pages 139-142
ISBN: 978-989-8565-47-1
Copyright
c
2013 SCITEPRESS (Science and Technology Publications, Lda.)
tion for a set of images sampling the same object with
different spatial points in sub-pixel levels. The appli-
cation of image superresolution avoids the misregis-
tration band captured images and besides makes itself
a chance for increasing spatial resolution. The sig-
nificant decrease in acquisition time of a multispec-
tral image derives from the fact that image superreso-
lution allows capturing band images at lower spatial
resolution than the required resolution. If the spa-
tial resolution of image capturing is reduced by half,
the camera’s exposure time or scanning time can be
reduced by quarter, and the angle of view becomes
twice as wide.
The purpose of this study is to evaluate the quan-
titative accuracy of the proposed method and indi-
cate its effectiveness. An experiment was conducted
using a typical 6-bands multispectral scanner and a
Japanese painting. Sets of band images were captured
with different spatial resolutions, and then errors of
recovered spectral reflectance with a certain resolu-
tion were evaluated.
2 PROPOSED BAYESIAN IMAGE
RECONSTRUCTION
In this section, We outline Bayesian image recon-
struction combined with spatial-superresolution and
spectral reflectance recovery (Murayama and Ide-
Ektessabi, 2012).
Let {y
k
}
K
k=1
be a multispectral image which con-
sists of K band images withW
L
× H
L
= N
L
pixels, and
{x
t
}
T
t=1
be a hyperspectral image of the same object
with T spectral samplings and W
H
× H
H
= N
H
pixels.
Here, y
k
is a monochromatic image acquired by using
the k-th band-pass filter and, x
t
is also a monochro-
matic image which represents reflectance at wave-
length λ
t
. These monochromatic images are vector-
ized in lexicographical order. We denotes the fully
vectorized images of {y
k
}
K
k=1
and {x
t
}
T
t=1
by
y =
y
1
.
.
.
y
K
, x =
x
1
.
.
.
x
T
. (1)
This method recovers {x
t
}
T
t=1
from {y
k
}
K
k=1
un-
der the following conditions: a) T > K. b)
W
H
W
L
=
H
H
H
L
= r > 1, where r is the rate of superresolution.
c) each band image y
k
of the multispectral image in-
cludes a small amount of position shift to other band
images, but the hyperspectral image {x
t
}
T
t=1
is com-
pletely registered.
The likelihood of {x
t
}
T
t=1
is derived from a linear
model of multispectral image acquisition by
p({y
k
}
K
k=1
|{x
t
}
T
t=1
) =
K
∏
k=1
p(y
k
|{x
t
}
T
t=1
)
=
K
∏
k=1
N
y
k
|(a
T
k
⊗ B
k
)x, βI
N
L
,
(2)
where a T-dimensional vector a
k
represents the sys-
tem sensitivity of the k-th band, a N
L
-by-N
H
matrix B
k
spatial degradations: geometrical transform, blurring,
and downsampling, β the variance of the sensor noise,
⊗ the Kronecker product, and N (z|µ, Σ) the multi-
variate Gaussian distribution of z whose mean and co-
variance matrix are µ and varSigma. a
k
is the product
of the following functions at the same sampling points
as {x
t
}
T
t=1
: spectral sensitivity of the camera and
spectral transmittance of the attached filter and spec-
tral power distribution of the light source. B
k
depends
on the point correspondencesbetween y
k
and {x
t
}
T
t=1
,
and the point spread function of blurring (PSF). In
this study, we assumed position shifts in rotation
and translation and assumed the PSF as the Gaussian
function, and denote the four-dimensional parameter
of B
k
by θ
k
, which includes the one-dimensional ro-
tation angle and the two dimensional translation, and
the one-dimensional radius of the PSF.
The posterior of {x
t
}
T
t=1
is obtained as a multi-
variate Gaussian distribution again by introducing a
Gaussian prior p({x
t
}
T
t=1
):
p({x
t
}
T
t=1
) = N (x|0, R⊗ I
N
H
). (3)
The mean of posterior µ, which is also the mode, is
derived by
µ =
(Ra
1
) ⊗ B
T
1
··· (Ra
K
) ⊗ B
T
K
S
−1
y. (4)
with
S =
(a
T
1
Ra
1
)(B
1
B
T
1
) ··· (a
T
1
Ra
N
)(B
1
B
T
N
)
.
.
.
.
.
.
(a
T
N
Ra
1
)(B
N
B
T
1
) ··· (a
T
N
Ra
N
)(B
N
B
T
N
)
+ βI
KN
L
.
(5)
Though mu is the reconstructed hyperspectral im-
age, The model parameters {θ
k
}
K
k=1
have to be de-
termined before calculating Eq. (4). {θ
k
}
K
k=1
is deter-
mined by maximizing the log-marginalizedlikelihood
h({θ
k
}
T
k=1
):
h({θ
k
}
T
k=1
)
= ln
Z
p({y
k
}
K
k=1
|{x
t
}
T
t=1
)p({x
t
}
T
t=1
)dx
1
···dx
T
= −
1
2
lndet(2πS) −
1
2
y
T
S
−1
y.
(6)
VISAPP2013-InternationalConferenceonComputerVisionTheoryandApplications
140
3 EXPERIMENTAL EVALUATION
A multispectral scanner was assembled with a
monochromatic camera, LED lights, and six band-
pass filters. The transmittance of the used filters had
peaks at 420 nm, 450 nm, 500 nm, 530 nm, 550 nm
and 600 nm with full width half maximum around
55 nm. The imaging subject was a Japanese paint-
ing which includes fine drawings and balanced col-
ors shown in Figure 1. The size of the painting was
220 mm by 275 mm. The accuracy of the proposed
method was evaluated in two areas illustrated in Fig-
ure 1 as well as in the whole area.
Figure 1: A Japanese painting used as a test object.
Seven sets of multispectral images were acquired
with varying spatial resolutions: 200 to 500 DPI (dots
per inch) at intervals of 50 DPI. Each band image
in a multispectral image involved sub-pixel transla-
tion due to mechanical property of the scanner. The
proposed Bayesian image reconstruction was applied
to these seven multispectral images respectively and
spectral reflectance from 400 nm to 700 nm was re-
covered. The rate of superresolution was set so as to
recover a 600-DPI image of spectral reflectance from
each multispectral image, namely set to 3, 2.4, 2, 1.71,
1.5, 1.33 and 1.2 for the acquired multispectral im-
ages with 200, 250, 300, 350, 400, 450 and 500 DPI
respectively. The radius of PSF was set to 0.5 pixel at
the acquired images, and the noise variance β was set
to 1× 10
−6
where the brightness of band images was
standardized up to 1.
For quantitative evaluation of the accuracy of re-
covered spectral reflectance, a 600-DPI multispectral
image was acquired using the same scanner and ob-
ject. Each band image was captured multiple times
and carefully selected so that the acquired multispec-
tral image included no position shifts, and spectral re-
flectance was then recovered by Wiener estimation.
In this study this reflectance data was the reference
to compare with spectral reflectance recovered by the
proposed method. The root mean squared error
(RMSE) of spectral reflectance and the average color
difference were calculated for each recovered data.
Figure 2: The errors of spectral reflectance of recovered
600-DPI images by the proposed method. Dashed lines il-
lustrate the errors when spectral reflectance was recovered
from multispectral image with 600 DPI and no misregistra-
tions. Area A and B are depicted in Figure 1.
Figure 2 showed the result. Note that in the calcu-
lation of RMSE reflectance was represented as a frac-
tion (not percentage), and the squared error was di-
vided by the number the wavelength sampling points
as well as by the number of color samples before
taking the square root. In Figure 2, the background
RMSE or the average color difference were added be-
cause they include some errors involving spectral re-
flectance recovery. These background errors were es-
timated by acquiring a multispectral image of an IT8
color chart where spectral reflectance of its 288 color
patches was known. Figure 3 depicts color images
reproduced from recovered spectral reflectance of the
painting. Figure 2 indicated that there is a trade-off
between accuracy and efficiency but both the RMSE
and the average color difference were quite small
when spectral reflectance was recoveredwith 600 DPI
from multispectral images acquired with higher than
or equal to 350 DPI. It means the rate of superres-
olution can be set to 1.71. Area A includes many
fine lines and high-spatial-frequencycomponents, but
ExperimentalEvaluationofBayesianImageReconstructionCombinedwithSpatial-SuperresolutionandSpectral
ReflectanceRecovery
141
(a) r = 2.4
(b) r = 1.71
Area AArea A
Area BArea B
Area AArea A
Area BArea B
Figure 3: Color images reproduced from the recovered spectral reflectance. 600-DPI images were recovered from (a) 250-DPI
images and (b) 350-DPI images, namely at the rate of superresolution r = 1.71, 2.4 respectively.
Area B includes few drawings. It had been expected
that the errors would become much higher in Area A,
but there was not a big difference between the results
of these two areas. Figure 3 shows close-up color im-
ages reproduced from recovered 600-DPI images of
spectral reflectance. There appears no false colors
along edge lines. This means that registration of each
band images was performed with high accuracy. In
Figure 3 (a) the rate of superresolution r was set to
2.4, and there appeared grain-like noise. This could
be because the number of sampling points in acquired
image was not enough to recovery image with a cer-
tain high resolution and some pixels failed to be re-
covered. In Figure 3 (b) r was set to 1.71 and images
were recovered successfully.
4 CONCLUSIONS
We tested experimentally the performance of
Bayesian image reconstruction combined with
spatial-superresolution and spectral reflectance re-
covery. Multispectral images of a Japanese painting
was acquired with various spatial resolutions (less
than or equal to 600 DPI) by using a typical 6-band
multispectral scanner, then 600-DPI images and its
spectral reflectance were recovered from each multi-
spectral image. The experimental results showed that
the spatial resolution could be increased by around
1.7 times and misregistrations between captured band
images were completely removed. This indicated that
the acquisition time of multispectral image can be
reduced almost by one third (the negative 2nd power
of the spatial resolution) and in addition the angle of
view can be extended by around 1.7 times at the same
time.
REFERENCES
DiCarlo, J. M. and Wandell, B. a. (2003). Spectral estima-
tion theory: beyond linear but before Bayesian. Jour-
nal of the Optical Society of America. A, Optics, image
science, and vision, 20(7):1261–1270.
Fukuda, H., Uchiyama, T., Haneishi, H., Yamaguchi, M.,
and Ohyama, N. (2005). Development of a 16-band
multispectral image archiving system. Proceedings of
SPIE, 5667:136–145.
Murayama, Y. and Ide-Ektessabi, A. (2012). Application of
bayesian image superresolution to spectral reflectance
estimation. Optical Engineering, 51(11):111713.
Pelagotti, A. and Mastio, A. D. (2008). Multispectral imag-
ing of paintings. IEEE Signal Processing Magazine,
25(4):27–36.
Shimano, N., Terai, K., and Hironaga, M. (2007). Recov-
ery of spectral reflectances of objects being imaged
by multispectral cameras. Journal of the Optical So-
ciety of America. A, Optics, image science, and vision,
24(10):3211–3219.
Toque, J. A., Sakatoku, Y., and Ide-Ektessabi, A. (2009).
Pigment identification by analytical imaging using
multispectral images. Image Processing ICIP 2009
16th IEEE International Conference, pages 2861–
2864.
Yamaguchi, M., Teraji, T., Ohsawa, K., Uchiyama, T.,
and Motomura, H. (2002). Color image reproduction
based on multispectral and multiprimary imaging: ex-
perimental evaluation. Proceedings of SPIE, 4663:15–
26.
VISAPP2013-InternationalConferenceonComputerVisionTheoryandApplications
142
