MULTISPECTRAL IMAGING

The Influence of Lighting Condition on Spectral Reflectance Reconstruction and

Image Stitching of Traditional Japanese Paintings

Jay Arre Toque, Yuji Sakatoku, Julia Anders, Yusuke Murayama and Ari Ide-Ektessabi

Advanced Imaging Technology Laboratory, Graudate School of Engineering, Kyoto Univeristy

Yoshida-honmachi, Sakyo-ku, 606-8501, Kyoto, Japan

Keywords: Multispectral imaging, Analytical imaging, Spectral reflectance, Image stitching.

Abstract: Illumination condition is one of the most important factors in imaging. Due to the relatively complex

interaction occurring when an incident light is irradiated on the surface of an object, it has been a topic of

researches and studies for quite a while now. In this study, its influence on the reconstruction of spectral

reflectance and image stitching was explored. A traditional Japanese painting was used as the target.

Spectral reflectance was estimated using pseudoinverse model from multispectral images captured with

seven different filters with spectral features covering 380-850 nm wavelengths. It was observed that the

accuracy of the estimation is dependent on the quality of multispectral images, which are greatly influenced

by lighting conditions. High specular reflection on the target yielded large amount of estimation errors. In

addition, the spectral feature of the filters was shown to be important. Data from at least four filters are

necessary to get a satisfactory reconstruction. On the other hand, it was observed that in addition to specular

reflection, the distribution of light highly affects image stitching. Image stitching is important especially

when acquiring images of large objects. It was shown that multispectral images could be used for the

analytical imaging of artworks.

1 INTRODUCTION

Multispectral imaging finds wide array of

applications in the field of medicine, remote sensing,

satellite imaging and others (Elaksher, 2008; Lane,

et al., 2008; Biehl, et al., 2002). This involves taking

images at different wavelengths to capture spectral

features that cannot be detected by the naked human

eye. The spectral characteristics can be regarded as

signatures, which can help in analyzing the object

being imaged. In a way, multispectral imaging is

different from “conventional” imaging techniques.

Conventional imaging is carried out in the visible

region of the electromagnetic spectrum. This region

covers wavelengths from 400-700nm, which

corresponds to frequencies from 428-750 THz. This

is called visible region because the human eyes are

only sensitive within this range (Lee, 2005).

Normally, this involves images with tristimulus

values corresponding to red, green and blue colors

(RGB). In applications such as display and

visualization, this imaging technique is more than

sufficient. The information that can be extracted

from an image is only as good as the amount of data

it contains. For a typical image with tristimulus

values, its information is limited to color. However,

if images are to be used for analytical imaging,

conventional imaging might not be enough because

of the limited amount of data.

Analytical imaging refers to techniques, which

provides useful information about an object being

imaged beyond its “conventional” visual content. In

conventional imaging, normally, “what you see is

what you get”. This is based on the paradigm using

three variables to characterize an image (MacAdam,

1993). With analytical imaging, it is desired that

images provide more information, which may

include material characteristics, surface and

topographic information and spectroscopic data. It is

based on the assumption that similar to other

electromagnetic spectrum (e.g x-ray, microwave,

etc.); material interaction within the visible light-

near infrared (VL-NI) range can be quantified.

However, this interaction is quite complex. In order

Toque J., Sakatoku Y., Anders J., Murayama Y. and Ide-Ektessabi A. (2009).

MULTISPECTRAL IMAGING - The Inﬂuence of Lighting Condition on Spectral Reﬂectance Reconstruction and Image Stitching of Traditional Japanese

Paintings .

In Proceedings of the First International Conference on Computer Imaging Theory and Applications, pages 13-20

DOI: 10.5220/0001788000130020

 SciTePress

perform sufficient analysis at VL-NI range; the

amount of data an image contains should be

increased. This may be accomplished using

multispectral imaging (Conde, et al., 2004).

In this study, multispectral images were captured

from 380-850 nm using image filters with different

spectral transmittances. The images obtained contain

information from the visible up to the near infrared

range. The study focused on how the lighting

conditions affect the reconstruction of spectral

reflectance of Japanese paintings. Paintings were

chosen as target because it normally requires non-

destructive and non-invasive analysis. This is

especially true if it is a cultural heritage (Balas, et

al., 2003). Since paintings vary in sizes, it may

sometimes not be possible to acquire the image of

the whole painting at once if high-resolution images

are desired. This will require image stitching. In this

case, the influence of lighting is also of particular

interest.

This study defines lighting condition as the

cumulative effect of the various illumination factors

affecting the perceived image. This includes

intensity, type of light and angle of incidence. In

reality, the factors affecting the lighting condition

are not limited to the three mentioned. Since we are

interested in the perceived image as detected by the

image-capturing device, the factors can also include

the distance of the light source to the target, surface

property of the target and many more others. As the

number of factors increase, the complexity of the

interaction also increase. In this study, the factors

are limited to the main light source characteristics

based on preliminary investigations.

2 EXPERIMENT

Multispectral images were captured using a

monochromatic CCD camera with spectral

sensitivity from 350-1000 nm, which peaks at

around 520 nm. The distance of the camera to the

target was approximately 480 mm. The images were

acquired using four types of filters (i.e. band pass

filter, special purpose filter, sharp cut filter and

infrared filter). A total of seven filters were used

(BPB-50, BPB-55, BPB-60, SP-9, SC-64, SC-70 and

IR-76). These filters have different peak

sensitivities, which enable the images to contain

more information from specific wavelength range. In

some cases, an IR-cut filter was used, specifically

BPB and SP filters, because they have unwanted

sensitivities at the near infrared region. The sharp

cut and IR filters were used for obtaining

information at longer wavelengths. The schematic

representation of the multispectral imaging system is

shown in Figure 1.

Figure 1: Schematic representation of the multispectral

imaging set-up.

In order to investigate the effects on spectral

reflectance reconstruction and image stitching of

lighting conditions, images were acquired using four

variations summarized in Table 1. Three parameters

were selected, such as type of light source, light

source angle and intensity. Based on preliminary

investigations, these parameters were found to

greatly affect the quality of the image and the

corresponding information it contains.

Table 1: Lighting conditions used in acquiring the

multispectral images. Note: E1 corresponds to experiment

1 and so on.

Parameters E1 E2 E3 E4

Type Halogen Halogen Halogen Fluorescent

Angle (°) 30 30 60 N.A.

Intensity

(%) 100 30 80 N.A.

2.1 Image Acquisition

In order to facilitate spectral reflectance

reconstruction and image stitching, the target was

imaged using the orientation shown in Figure 2. At

first, the image of the upper half of the target was

acquired. Then after that, the target was rotated 180°

to capture the image of the other half.

A total of three targets were used; a white

background, a learning sample and a Japanese

painting. The white background is used to calibrate

the uneven distribution of light when imaging the

learning sample and Japanese painting. The learning

sample, which was composed of conventionally used

Japanese mineral pigments, was employed as a basis

IMAGAPP 2009 - International Conference on Imaging Theory and Applications

for estimating the spectral reflectance. A Japanese

painting was chosen as the main target because of

the technical challenges it presents (e.g. non-

invasive, non-destructive, etc.). There exist other

more advanced analytical technique for studying

paintings, which are commonly x-ray-based

(Marengo, 2006). However, x-ray-based technique is

relatively non-destructive but not entirely non-

invasive. Usually, a small piece of the sample is

required. For paintings with high cultural value,

taking even a minute sample is unacceptable.

Figure 2: Orientation of the target during image

acquisition.

2.2 Spectral Reflectance

Reconstruction and Image

Calibration

The effect of the lighting condition during imaging

was evaluated based on the accuracy of the spectral

reflectance reconstruction and the quality of image

stitching. Before reconstructing the spectral

reflectances, the images were calibrated using a

white background to compensate for the effect of the

uneven distribution of light shone on the surface of

the target. This helps facilitate better image

stitching. The pixel values of the images were

adjusted using Eq.1

= T

⎛

⎝

⎜

⎞

⎠

⎟

(1)

where T

corresponds to the i

pixel value of the

uncompensated target image,

is the average

pixel value of the white background, B

is the i

pixel value of the white background and T

’ is the

new i

pixel value of the white background-adjusted

target.

After the images were adjusted using the white

background, the spectral reflectance was estimated.

In general, the spectral characteristic of an image is

described by Eq.2 (Shimano, et al, 2007)

p =CLr +e (2)

where, p is the pixel value of the image captured at a

certain band, C is the spectral sensitivity of the

capturing device, L is the spectral power distribution

of the light source, r is the spectral reflectance and e

is an additive noise corresponding to the

measurement errors of the spectral characteristics of

the sensors, illumination and reflectances. All of the

quantities in Eq.2 are functions of the wavelength. In

this case, in order to estimate the spectral

reflectance, the spectral characteristic of the camera

and light source should be known. However, this

information is often unavailable. Using

pseudoinverse model, the spectral reflectance can be

estimated without prior knowledge of the spectral

characteristics of the camera and light source.

The pseudoinverse model is a modification of the

Wiener estimation by regression analysis (Shimano,

et al, 2007). In this model, a matrix W is derived by

minimizing

−

WP from a known spectral

reflectance of a learning sample, R, and the

corresponding pixel values, P, captured at a certain

band. The matrix W is given by Eq.3

W=RP

(3)

Where P

represents the pseudoinverse matrix of P.

By applying the derived matrix W to the pixel value

of the target image, p, the spectral reflectance

r can

be estimated using Eq.4

r = Wp

(4)

The size of the matrices used in Eq.3 and Eq.4 is

a function of the number of learning sample k,

number of multispectral bands M and number of

spectral reflectances N measured at 10nm interval

from 380-850 nm. In this study, k is 98, M is 7 and N

is 48. The learning sample used in this study is a

collection of 98 commonly used Japanese pigments

and the spectral reflectance is measured using a

spectrometer.

Reconstruction of the spectral reflectance was

carried out using multispectral images because it can

MULTISPECTRAL IMAGING - The Influence of Lighting Condition on Spectral Reflectance Reconstruction and Image

Stitching of Traditional Japanese Paintings

contain both spectroscopic and spatial information.

With the conventional spectroscopes and

spectrometers, the data acquired are only

spectroscopic in nature. The information is confined

to reflectance, transmittance and absorbance.

However, by manipulating some image acquisition

parameters (e.g. lighting angle, camera position,

etc.) in multispectral imaging, it is possible to get

spatial information about the object such as surface

features, topography and other physical aspects of

the material’s surface.

Figure 3: Image of the Japanese painting used as target to

evaluate the effect of lighting condition. The spectral

reflectances are estimated from three regions on the

painting.

In order to evaluate how the lighting condition

affects the spectral reflectance estimation, three

regions on the Japanese painting were selected

namely Region 1, 2 and 3 as depicted by Figure 3.

3 RESULTS AND DISCUSSION

3.1 Image Stitching

Image stitching is important in acquiring images of

large objects that cannot be captured entirely at

once. This problem might seem trivial. In principle,

as long as the object is in the line of sight of the

capturing device, the size of the image that can be

acquired is virtually unlimited. This can be

accomplished by increasing the distance between the

camera and the target object. However, as the

distance increases, the resolution of the image

decreases. This affects the quality of data the image

contains. It is possible to solve this issue by

performing some processing on the image but it is

usually better to use the image with little alteration

as possible in order to preserve the information it

holds.

Figure 4: Stitched image of a Japanese painting using

uncalibrated images.

Figure 5: 3D representation of the distribution of light as

reflected by the white background. The pixel values of the

1360x1024 image were used to create this 3D impression.

In this study, the influence of lighting condition

on image stitching was investigated. Since the

lighting was varied several times, it is to be expected

to get images with different characteristics. Figure 4

shows an example of a stitched image acquired

using IR 76 filter. The stitching line is very obvious

which is a result of uneven light distribution. Figure

5 depicts a 3D representation. It can be seen that a

portion of the target receives more intense light as

compared to the other parts. As a result, some area

appears to be brighter than the other. In addition to

IMAGAPP 2009 - International Conference on Imaging Theory and Applications

the obvious stitching line, some specular reflection

can also be seen.

Generally, when an object is subjected to a

radiation, three common interaction occurs. The

incident radiation may be reflected, absorbed and

transmitted as illutrated by Figure 6. It can be a

mixture of any of the three. Depending on the

characteristics of the material and the energy of the

radiation, this interaction can be more complex.

However, radiation in the form of visible light, the

three mentioned is likely to occur. In imaging, the

reflected portion of the radiation is more significant.

This is the quantity that the camera sensors used to

form the images. As shown in the simplified model

below, reflected light is further classified into

specular and diffused reflection (Vargas, 2006).

Specular reflection is the potion of the reflected

light that is also known as a mirror reflection. This

results to saturation in some parts of the target

painting. Once the image is saturated, little

information can be extracted from it and worse it

obscure the stitching of the images.

Figure 6: Simple interaction model when a material is

subjected to radiation.

In order to solve the issues brought by the

uneven light distribution, the images were calibrated

using a white background. The details of the

calibration process are described in the previous

section. The images were acquired using various

position in order to have a specular reflection-free

image. Figure 7 shows an example of the final

stitched image. After calibration and removal of

regions with high specular reflections, the stitching

line is barely visible. It is believed that the quality of

the stitching can be further improved by using more

sophisticated calibration techniques. In this study,

only a simple technique was implemented. It was

discovered that this method is only effective within a

certain threshold. If the standard deviation of the

average pixel values of the white standard is less

than 20 pixels, then method employed here is

sufficieint. If it goes beyond the 20 pixels threshold,

stitching lines eventually become visible. Therefore

there is a need to improve the white background

calibration by using other techniques.

Figure 7: Stitched image of the Japanese painting

calibrated using a white background.

3.2 Reconstruction of Spectral

Reflectance

In this study, three regions on the painting were

selected where the spectral reflectances were

reconstructed. The three areas possess distinct

charateristics which was the reason for its selection.

For example, Region 1 is the area on the Japanese

painting that experienced high specular reflection.

This region has gold foil laid on the the surface.

Using gold foils in painting is a common practice in

Japanese art. Since the region has metallic

constituent, it explains why it has high specualr

reflection. This significantly affected the estimated

spectral reflectance as shown in Figure 8. The figure

shows five reflectances, four from the different

lighting conditions and one for the spectral

reflectance measured using a spectrometer. This acts

as reference spectral reflectance. It can be observed

that the reconstructed spectral reflectances of the

multispectral images on Region 1 is quite poor. This

might due to the reflectance characteristics of the

gold foil in the region. Specular reflection was not

MULTISPECTRAL IMAGING - The Influence of Lighting Condition on Spectral Reflectance Reconstruction and Image

Stitching of Traditional Japanese Paintings

observed in all experiments, when this was the case,

Region 1 appeared to be dark. This results to very

low spectral reflectance but still not close to the

measured reflectance. Unfortunately, the lighting

parameters used in the experiment were not

optimum. However, the main aim of the study is to

observe how the lighting condition affects the

estimation. Based on the phenomena observed, it can

be concluded that the issue on specular reflection

needs to be addressed in order to have better

reconstruction.

Figure 8: Reconstructed spectral reflectance from

multispectral images on Region 1 of the Japanese painting.

Figure 9: Reconstructed spectral reflectance from

multispectral images on Region 2 of the Japanese painting.

On the other hand, Region 2 was selected

because it did not show any specular reflection in all

the experiments. It is painted with an orange mineral

pigment resembling atumn leaves. Compared to

Region 1, the reconstructed spectral reflectance is

close to the measured reflectance up to wavelengths

of 600 nm. Between 600-700 nm however, the

estimation was relatively poor except for E4. The

estimated spectral reflectance of E4 was close to the

measured one up to 700 nm. At the near infrared

region, no comparison can be made because the data

from the spectrometer is only available from 400-

700 nm.

Finally, Region 3 was selected because it yielded

multispectral images with and without specular

reflection across the different parameters. What is

unique in this region is that it has both mineral and

metallic pigment. The metal consitituents in this

case are traces of silver particle instead of foil.

Figure 10 shows the reconstructed spectral

reflectance. The estimation is still not as accurate as

it should be but it is better compared to Region 1.

Again, the poor reconstruction may be attributed to

specular reflection. In this case since the metallic

constituents are in particle form, the effect on the

multispectral images is not as severe when compared

to the gold foil.

Figure 10: Reconstructed spectral reflectance from

multispectral images on Region 1 of the Japanese painting.

The accuracy of the spectral reflectance

reconstruction may also be explained by the

characteristics of the filters used to capture

multispectral images. In order to get a good

estimation, the filters should be able to get

significant amount of data. Figure 11 shows the

spectral characteristics of the seven filters used

along with the measured spectral reflectances of the

selected regions. Among the selected regions,

Region 2 has the most number of filters that are able

to collect spectral information. A total of four filters

were able to collect the necessary data especially

between 420-580 nm. This can explain why the

accuracy of the estimation is relatively better if

compared to the other wavelengths.

On the other hand, the filters with short

wavelengths and long wavelengths collected the data

from Region 3. No filter was able to collect any

useful information between 480-680 nm, which can

explain why the reconstructed spectral reflectance

within this range deviated from that of the measured.

Finally as for Region 1, on top of the severe effect of

specular reflection of the gold-laden surface, it can

IMAGAPP 2009 - International Conference on Imaging Theory and Applications

be observed that only two filters were able to collect

information.

In addition, it is interesting to note how well the

estimated spectral reflectances above 700 nm were

convergent except for Region 1. Evidently, the

influence of high specular reflection also affected

the reconstructed reflectance at that wavelength. It is

difficult to ascertain its accuracy because the

measured data only goes up to 700 nm but it may be

assumed that it might be good enough.

Why is important to reconstruct the spectral

reflectance above the visible range? This is because

previous studies have shown that some materials

have unique spectral features at the near infrared

range (Anderson, 1947). In addition, acquiring

images beyond the visible range can help increase

the amount of information available from the image.

Figure 11: Spectral characteristics of the filters used in

capturing the multispectral images along with the

measured spectral reflectance of the three selected regions.

4 CONCLUSIONS

In this study, the influence of lighting condition on

image stitching and spectral reflectance

reconstruction was explored. Using multispectral

images, spectral reflectance of a traditional Japanese

painting was estimated by the pseudoinverse

method. Results show that specular reflection, which

is influenced by illumination, significantly affects

the accuracy of the reconstruction. It was also

observed that the spectral features of the filters used

play an important role. According to the comparison

of spectral reflectance curves, the estimation is more

accurate between 420-580 nm especially for Region

2 because at least four filters were able to collect

information within that range. On the other hand, it

was shown that image stitching was greatly

influenced by the light distribution on the target and

its surface reflection characteristics. Stitching lines

were highly visible when specular reflection is

severe. It is also observed that the calibration

technique was only effective within a 20-pixel

standard deviation-threshold. Beyond this, more

advanced calibration technique is necessary.

ACKNOWLEDGEMENTS

This work has been done as part of the project “An

Integrated System for Secure and Dynamic Display

of Cultural Heritage” sponsored by Japan Science

and Technology Agency, Regional Resources

Development Program. This collaborative project

was organized by Kyoto University Graduate School

of Engineering, S-tennine Kyoto (Ltd) and Kyushu

National Museum. The Authors would like to

express their thanks to Imazu Setsuo of Kyushu

National Museum and other staff of the museum

and, Oshima of S-tennine Kyoto and his group for

supporting this work.

REFERENCES

Anderson, J.A., 1947. The diffuse reflectance of paints in

the near-infrared. Journal of the Optical Society of

America. 37(10), pp. 771-777.

Balas, C., Papadakis, V., Papadakis, N., Papadakis, A.,

Vazgiouraki, E., Themelis, G., 2003. A novel hyper-

spectral imaging apparatus for the non-destructive

analsyis of objects of artistic and histori values.

Journal of Cultural Heritage, 4, pp 330s-337s.

Biehl, L., and Landgrebe, D., 2002. MultiSpec—a tool for

multispectral–hyperspectral image data analysis

Computers & Geosciences, 28(10), pp.1153-1159

Conde, J., Haneisihi, H., Yamaguchi, M., Ohyama, N.,

Baez, J., 2004. Spectral reflectance estimation of

ancient Mexican codices, multispectral images

approach. Revisita Mexhicana De Fisica, 50(5), pp.

484-489.

Elaksher, A., 2008, Fusion of hyperspectral images and

lidar-based dems for coastal mapping. Optics and

Lasers in Engineering, 46(7), pp. 493-498

Lane, G.R., Martin, C., Pirard, E., 2008. Techniques and

applications for predictive metallurgy and ore

characterization using optical image analysis.

Minerals Engineering, 21(7), pp. 568-577.

Lee, H-C., 2005. Introduction to color imaging science.

Cambridge, UK, Cambridge University Press.

MacAdam, D., 1993. Selected Papers on Colorimetry-

Fundamentals, volume 77 of Milestone Series.

Bellingham, Washington, SPIE Optical Engineering

Press.

Marengo, E., Liparota, M.C., Robotti, E., Bobba, M.,

2005. Multivariate calibration applied to te field of

MULTISPECTRAL IMAGING - The Influence of Lighting Condition on Spectral Reflectance Reconstruction and Image

Stitching of Traditional Japanese Paintings

cultural heritage: analysis of pigments on the sureface

of a painting. Analytica Chimica Acta, 553, pp. 111-

122.

Shimano, N., Terai, K., Hironaga, M., Recovery of

spectral reflectance of objects being imaged by

multispectral cameras. Journal of Optical Society of

America A, 24(10), 3211-3219.

Vargas, W., Amador, A., Niklasson, G., 2006. Diffuse

reflectance of TiO

pigmented paints: spectral

dependence of the average pathlength parameter and

forward scattering ratio. Optics Communication, 261,

pp. 71-78.

IMAGAPP 2009 - International Conference on Imaging Theory and Applications