Light Field Scattering in Participating Media
Takuya Mokutani, Fumihiko Sakaue and Jun Sato
Department of Computer Science, Nagoya Institute of Technology, Gokiso Showa, Nagoya, Japan
Keywords:
Participating Media, Light Scattering Model, Light Field Scattering.
Abstract:
In this paper, we propose a representation of the light scattering in participating media, which can represent
all order light scattering simply. To achieve the model, we focus on the light field in the participating media,
and it is shown that the convolution of the light field can describe the attenuation of the light rays and scat-
tering of them. By analyzing the convolution kernel, we derive a simple kernel that represents all order light
scattering. Also, we introduce the estimation method of the characteristics of the participating media based on
our proposed model. Several experimental results show that our proposed model can describe light scattering
more appropriately than existing models.
1 INTRODUCTION
In recent years, image sensors such as cameras are
one of the most important devices to obtain scene in-
formation, and they ordinary utilized for various ap-
plications, e.g., 3D measurement, object recognition,
and so on. In ordinary cases, we assume that the cam-
era obtains ‘clear’ information from the scene. How-
ever, images taken in the outdoor scenes are often dis-
turbed by participating media such as fogs, smokes,
and so on. Figure 1 shows example images taken
in the participating media. The figure shows that the
fogs and smokes disturb appropriate imaging in par-
ticipating media. Furthermore, water also disturbs
imaging if we want to take underwater images. The
effect of the participating media disturbs various ap-
plications based on the camera images such as au-
tonomous driving and driving assistant systems. In
addition, 3D sensors, e.g., LiDAR, also cannot obtain
an accurate distance in participating media because
several sensors in the system also cannot get the ap-
propriate data as same as the cameras.
In order to solve the problem, various kinds of
methods are studied(Narasimhan et al., 2006; He
et al., 2011; Kitano et al., 2017; Naik et al., 2015;
Figure 1: Example images taken in participating media,
fogs, smokes, and water.
Satat et al., 2018). To solve the problem essentially,
we need to use the accurate light scattering model to
describe the phenomena in the participating media.
In the field of computer graphics, several models are
used to describe the light behavior accurately(Pharr
et al., 2016). In tradition, light behavior is classi-
fied into single scattering and multiple scattering, and
several models are proposed for each scattering. Al-
though the models can describe the light scattering
in the participating media in limited case, there are
lots of situations which cannot be explained by these
models. In order to describe the light scattering ap-
propriately, light ray tracing techniques and the pho-
ton mapping techniques are utilized. Although they
provide more realistic results, they are not suitable for
analyzing images because they require a high compu-
tational cost. In this research, we propose a light scat-
tering description model that can explain the single
scattering as well as multiple scattering in low com-
putational cost. In addition, we also show the partici-
pating media analysis method based on the proposed
model. In this light ray explanation, we focus on the
5D light field in the scattering media, and we show
that light scattering can be described efficiently and
effectively using the light field.
544
Mokutani, T., Sakaue, F. and Sato, J.
Light Field Scattering in Participating Media.
DOI: 10.5220/0009180505440549
In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 5: VISAPP, pages
544-549
ISBN: 978-989-758-402-2; ISSN: 2184-4321
Copyright
c
 2022 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

2 LIGHT SCATTERING IN
PARTICIPATING MEDIA
2.1 Participating Media and Light
Transport Equation
We first explain the characteristics of participating
media(Mukaigawa et al., 2010). The participating
media includes lots of micro-particles, and the par-
ticles affect input light rays. For example, fog and
smoke are representatives of the participating media,
and they consist of particles such as water and dust.
The particles scatter light rays, and thus, observed im-
ages in the media become unclear, as shown in Fig.1.
The effect of the media on the light rays are clas-
sified into light attenuation, in-scattering, and out-
scattering. These effects are described by light trans-
port equations (LTE). Let us consider the case when
a light ray L(x, ω) passes through the media at a point
x directed to ω. Under the assumption that dL(x,ω)
denote the change of the light ray, the effect of the
participating media is described as follows:
L
0
(x,ω) = L(x, ω) + dL(x,ω) (1)
where L
0
is an affected light ray by the media.
The dL(x,ω) consists in attenuation term, in-
scattering term and out-scattering term. First, the at-
tenuation describes light absorbing by the media, and
it is described as follows:
dL(x,ω) = −σ
a
(x)L(x,ω)ds (2)
where σ
a
is absorption coefficient and ds shows thick-
ness of the media. Second, several light rays contact
the particles and reflected in directions other than ω.
Therefore, light rays directed to ω is attenuated, and
it is described as follows:
dL(x,ω) = −σ
s
(x)L(x,ω)ds (3)
where σ
s
denotes a scattering coefficient at x. Fi-
nally, scattered light rays from any directions other
than ω are added to light rays as out-scattering. The
out-scattering is described as follows:
dL(x,ω) = σ
s
(x)
Z
Ω
Lp(x,ω
0
,ω)L(x,ω
0
)dω
0
ds (4)
where p denote phase function, and it describes the
probability that the light ray from ω
0
is reflected di-
rection ω.
Several functions are proposed as phase function
p. In this paper, we utilize the Henyei-Greenstein
phase function defined as follows:
p(θ) =
1
4π
1 − g
2
(1 + g
2
− 2g cos θ)
2/3
(5)
Figure 2: Single scattering in the participating media.
where θ = arccos(ω, ω
0
), and g (−1 ≤ g ≤ 1) de-
termine directional characteristics of the scattering.
By these three equations, Eq.(1) is rewritten as fol-
lows:
L
0
(x,ω) = L(x, ω) − σ
a
(x)L(x,ω)ds
−σ
s
(x)L(x,ω)ds
+σ
s
R
Ω
p(x,ω
0
,ω)L(x,ω
0
)dω
0
ds
(6)
By solving the LTE, the status of the light ray in scat-
tering media can be described. However, the deriva-
tion of an analytical solution to the equation is dif-
ficult because of the out-scattering term. Therefore,
several algorithms based on numerical analysis are
utilized in the field of computer graphics(Pharr et al.,
2016). However, they require lots of computational
costs, and it is not easy to utilize them for analysis of
the media.
2.2 Single Scattering Model
In order to relax the computational complexity, the
approximated model is utilized generally. Under the
assumption that the density of the micro-particles is
low, a single scattering model is used for describing
the light scattering. In this case, we assume that the
light rays are scattered (reflected) just one time by
micro-particles in the media. Under this assumption,
routes of the light rays are constrained completely by
an input light ray L(x
i
,ω
i
) and an output light ray
L(x
o
,ω
o
) as shown in Fig.2.
In this case, L
0
(x
o
,ω
o
) is described as follows:
L
0
(x
o
,ω
o
) = σ
s
p(θ)e
−(σ
a
+σ
s
)(d
1
+d
2
)
L(x
i
,ω
i
) (7)
where d
1
and d
2
denote distance from x
i
to s and s to
x
o
, respectively. The θ denotes an angle between ω
i
and ω
o
.
The computation of the single scattering model is
straightforward. In addition, this model is approxi-
mately valid for shallow participating media. There-
fore, the model is often utilized for analysis of the
scattering media.
2.3 Multiple Scattering Model
We next consider the case when the density of the
media is very high. In this case, multiple scatter-
Light Field Scattering in Participating Media
545

ing model is utilized for describing the light scatter-
ing. In this model, it is assumed that the light rays
are scattered repeatedly, and then, the directionality
of the light ray is lost. Therefore, light rays into the
dense participating media are distributed in any direc-
tion evenly. By using this model, light scattering by
dense media such as milk is appropriately described.
2.4 Low-order Scattering
By using the two models, light scattering in the
participating media is described. However, general
scenes often include more complicated scattering.
Mukaigawa et al.(Mukaigawa et al., 2010) indicate
light rays input to the participating media scattered
a few times, and then most scenes include not sin-
gle scattering but low-order scattering. Although the
method which separates each order scattering is pro-
posed by them, the method requires several numbers
of images because the method uses high-frequent pat-
tern projection(Nayar et al., 2006) to separate the light
rays. Therefore, it is difficult to apply the method for
ordinary image analysis.
3 LIGHT FIELD SCATTERING
MODEL
3.1 Light Field Scattering
In order to describe the light scattering not only single
and multiple scattering but also any order scattering in
the participating media, we focus on the light field in
the media. As described in Eq.(1), LTE describes the
relationship between input light rays and output rays.
Therefore, we describe the light scattering by light ray
transition in the light field. By using this proposed de-
scription, light scattering can be explained efficiently.
In addition, this description achieves describing sin-
gle scattering, low-order scattering, and multiple scat-
tering in the same manner.
In this manner, we separately consider input light
rays to a point and output light rays from the point. By
combining the input light rays and output light rays,
we achieve a description of the light scattering effi-
ciently.
3.2 Input Light Rays
We first consider input light rays to a point in the
scene. Let L(x,ω) denote a light ray at a point x
directed to ω. Under the assumption that light rays
go straight in scattering media other than reflection
by particles, integrated input light ray L
0
(x,ω) to the
point x directed to ω are computed as follows:
L
0
(x,ω) =
Z
d
e
σ
t
d
L(x + dω, ω)dd (8)
Let g(d) denote a function which describe light atten-
uation and it is defined as follows:
g(d) =

e
σ
t
d if d ≥ 0
0 otherwise
(9)
By using the g, Eq.(8) is rewritten as follows:
L
0
(x,ω) =
Z
d
g(d)L(x + dω,ω)dd (10)
This is convolution of L(x,ω) by a convolution kernel
g. Therefore, integration of the input light ray can be
computed by just convolution of the light field.
3.3 Output Light Rays
We next consider output light rays scattered by parti-
cles. The integrated input light rays L
0
are scattered
by the particles based on a phase function p. In this
scattering, we focus on an output light ray L
00
(x,ω
0
)
directed to ω
0
. Under the assumption that the phase
function is isotropic, the scattering is computed as fol-
lows:
L
00
(x,ω
0
) = σ
s
Z
ω
p(ω,ω
0
)L
0
(x,ω + ω
0
)dω (11)
In this case, the phase function p can be regarded as a
convolution kernel. Therefore, the light scattering by
the particles is also described by just convolution.
Note that ω +ω
0
in Eq.(11) is not correct since this
convolution is done on a sphere. The effective convo-
lution and fourier transfomation on the sphere can be
done by using shpherical harmocis(Su and Grauman,
2017). Therefore, we simply describe this convolu-
tion by just ω + ω
0
for convinience.
3.4 N-th Order Scattering Description
As described in the previous sections, integration of
input light rays and scattering of the input light rays
are described by just convolution. By integrating both
of convolutions, light scattering in participating media
is described as follows:
L
0
(x,ω) = σ
s
g ∗ s ∗ L(x,ω) = σ
s
k ∗ L(x,ω) (12)
where
0
∗
0
denotes convolution and k = g ∗ s is the in-
tegrated kernel.
By convolution σ
s
k, the input light field L is up-
dated to the first-order scattering light L
0
. Since the
scattered light is also scattered by media, again and
VISAPP 2020 - 15th International Conference on Computer Vision Theory and Applications
546

Figure 3: 2D image as a slice of the light field.
again, N-th order scattering light L
N
is described as
follows:
L
N
(x,ω) = σ
s
k ∗ L
N−1
(x,ω) (13)
where L
0
= L and L
0
= L
1
. This equation indicates
that σ
s
L
N
is scattered by the media, and the rest (1 −
σ
s
)L
N
is observed directly. Therefore, the observed
light field L
a
, including all scattering component, is
described as follows:
L
a
= (1 − σ
s
)
∞
∑
j=0
L
j
(14)
In this equation, we derive N-th order light scattering
by image convolution.
Note that the image convolution can be described
by just element multiplication in the frequency do-
main. In addition, the techniques of the fast Fourier
transformation require low computational cost for im-
age convolution. Therefore, the light scattering repre-
sentation based on image convolution can be achieved
by just low computational cost.
4 PARTICIPATING MEDIA
CHARACTERISTICS
ESTIMATION
In this section, we describe an estimation method for
the participating media characteristics based on a light
scattering model described in the previous section.
Under the assumption that the participating media is
uniform, an attenuation coefficient σ
a
, symmetry fac-
tor g in a phase function, and a scattering coefficient
σ
s
determine the characteristics in this model. There-
fore, we need to estimate the three parameters.
In ordinary cases, we cannot obtain the status of
the light field directly because conventional sensors
cannot observe a 5D light field. Therefore, we need
to estimate the parameters from a 2D image, which is
a slice of the5D light field, as shown in Fig.3.
Let an image I(x,y) denote a slice of the light field
L by a direction ω
o
to a camera and a plane in the
scene. Under the assumption that input light ray L
i
is
known, we define an evaluation function E as follows:
E = kI(x, y) − S (k(σ
t
,σ
s
,g)L
i
)k
2
(15)
Figure 4: Experimental environment.
where k(σ
t
,σ
s
,g) is a convolution kernel determined
by the parameters, and S denotes slicing of the light
field correspond to the input image. Values that min-
imize the E are suitable parameters to describe the
characteristics of the participating media.
5 EXPERIMENTAL RESULTS
5.1 Environment
In this section, we show several experimental results
by our proposed method. In this experiment, we com-
bined a few amounts of white ink and water. The com-
bined water was used as the participating media. The
white water filled a water tank, and the tank was ob-
served from the upper side, as shown in Fig.4. In this
environment, light rays were input to the tank by a
projector. The relative position between a camera and
a projector was calibrated beforehand, and then the
input light ray is controlled and known. An Example
of the input image is shown in Fig.5. In order to help
the understanding, the grayscale images are colored,
as shown in the right image. In these images, densities
of the white ink were changed to survey our proposed
model can describe the different density participating
media. Not only our proposed model but also a single
scattering model was used for the estimation of the
parameters for comparison.
5.2 Results
Figure 6 shows input images and estimated results.
In this results, result images were synthesized from
input light rays and estimated parameters by our
proposed method. For comparison, images including
1st order scattering (single scattering) and until
several order scatterings are shown, respectively.
RMSE between an input image and a reconstructed
image is shown below of each image.
Light Field Scattering in Participating Media
547

Figure 5: Examples of observed images. The left images
show the original grayscale images, and the right image
shows the colored images. In the upper image, the density
of the participating media is high, and it is low in the lower
images.
(a) input image
(b) 1st order scattering
(RMSE = 7.73)
(c) 3rd order scattering
(RMSE = 6.29)
(d) 20th order scattering
(RMSE = 5.68)
Figure 6: Reconstructed images and input images when the
density of the participating media is low. Figure(a) shows
an input image and (b)∼ (d) show images including until
1st order, 3rd order and 20th order scatterings.
In this result, RMSE between the input image and
the reconstructed image becomes smaller according
to scattering order increasing. The result indicates
that the input image includes not only lower light scat-
tering but also higher-order scattering even if the den-
sity of the media is not so high. Our proposed method
can describe the various order scatterings, and thus
RMSE becomes lower when the scattering order be-
comes higher.
Figure 7 shows estimated results of the light scat-
tering in denser participating media. In these re-
sults, RMSE becomes In these results, RMSE be-
comes lower according to the scattering order as sim-
(a) input image
(b) 1st order scattering
(RMSE = 8.13)
(c) 3rd order scattering
(RMSE = 4.10)
(d) 20th order scattering
(RMSE = 2.70)
Figure 7: Reconstructed images and input images when the
density of the participating media is high. Figure(a) shows
an input image and (b)∼ (d) show images including until
1st order, 3rd order and 20th order scattering.
ilar to the previous results. The results indicate that
our proposed method can describe not only shallow
participating media, but also the dense media by us-
ing the same manner.
6 CONCLUSION
In this paper, we propose a light scattering model that
can describe not only low-order scattering but also
high-order scattering in participating media. In this
model, we describe light attenuation and light scat-
tering by the convolution of the light field. By using
the kernel, light scattering in the participating media
can be described accurately and efficiently. Based on
the convolution description, we introduce the estima-
tion method of characteristics of participating media.
Several experimental results show that the proposed
method can describe the light scattering in the par-
ticipating media. Besides, our proposed method es-
timate the characteristics of the participating media.
We will consider a more stable estimation method and
also consider the case when the participating media is
not uniform in future work.
REFERENCES
He, K., Sun, J., and Tang, X. (2011). Single image haze
removal using dark channel prior. IEEE Trans. Pattern
VISAPP 2020 - 15th International Conference on Computer Vision Theory and Applications
548

Anal. Mach. Intell., 33(12):2341–2353.
Kitano, K., Okamoto, T., Tanaka, K., Aoto, T., Kubo, H.,
Funatomi, T., and Mukaigawa, Y. (2017). Recover-
ing temporal psf using tof camera with delayed light
emission. IPSJ Transactions on Computer Vision and
Applications, 9(15).
Mukaigawa, Y., Yagi, Y., and Raskar, R. (2010). Anal-
ysis of light transport in scattering media. In
Proc.CVPR2010.
Naik, N., Kadambi, A., Rhemann, C., Izadi, S., Raskar,
R., and Kang, S. B. (2015). A light transport model
for mitigating multipath interference in time-of-flight
sensors. In Proc. CVPR2016, pages 73–81.
Narasimhan, S. G., Gupta, M., Donner, C., Ramamoorthi,
R., Nayar, S. K., and Jensen, H. W. (2006). Acquiring
scattering properties of participating media by dilu-
tion. In ACM Transactions on Graphics (TOG), vol-
ume 25, pages 1003–1012. ACM.
Nayar, S. K., Krishnan, G., Grossberg, M. D., and Raskar,
R. (2006). Fast separation of direct and global com-
ponents of a scene using high frequency illumination.
SIGGRAPH.
Pharr, M., Jakob, W., and Humphreys, G. (2016). Phys-
ically Based Rendering: From Theory to Implemen-
tation. Morgan Kaufmann Publishers Inc., San Fran-
cisco, CA, USA, 3rd edition.
Satat, G., Tancik, M., and Raskar, R. (2018). Towards
photography through realistic fog. 2018 IEEE Inter-
national Conference on Computational Photography
(ICCP), pages 1–10.
Su, Y.-C. and Grauman, K. (2017). Learning spherical con-
volution for fast features from 360 imagery. In Ad-
vances in Neural Information Processing Systems 30,
pages 529–539.
Light Field Scattering in Participating Media
549