Time-to-Contact in Scattering Media

Wooseong Jeong, Laksmita Rahadianti, Fumihiko Sakaue and Jun Sato

Nagoya Institute of Technology, Gokiso, Showa, Nagoya, 466-8555, Japan

Keywords:

Time-to-Contact, Scattering Media, Single Scattering.

Abstract:

In this paper, we propose a method for estimating time-to-contact in scattering media, such as fog. Images

taken in the scattering media are unclear, and thus, we cannot detect appropriate geometric information from

images for computing 3D information. In this paper, we consider not geometric information but photometric

information such as observed intensity. In our method, we can eliminate the effect of scattering media and

estimate the time-to-contact toward objects without any prior knowledge.

1 INTRODUCTION

Recently, computer vision techniques are widely used

for video surveillance, object recognition and many

other applications. In particular, 3D distance mea-

surement is one of the most important problems in

the ﬁeld of computer vision.

When we measure 3D distance by using a stereo

camera system, a set of image correspondences are re-

quired. In ordinary case, image features points are ex-

tracted by using feature point detectors, such as SIFT

and SURF, and then 3D distance is estimated from the

extracted points. Although the stereo method works

well if we have proper images as shown in Fig.1 (a),

it does not work well when the input images are un-

suitable for feature point detection. For example, if

there is fog or smoke in the input scene as shown in

Fig.1 (b), the feature points may not be extracted suf-

ﬁciently and accurately. This is because the light rays

reﬂected from the object surface are scattered by mi-

cro particles as shown in Fig.5. In this case we cannot

obtain original colors, feature points and much other

visual information properly. These kinds of media,

which scatter light rays such as fog and smoke, are

called as scattering media.

In the scattering media, many existing compute vi-

sion techniques do not work well. In order to avoid

this problem, many methods which eliminate the

effect of scattering media are proposed (Schechner

et al., 2001; Namer and Schechner, 2006; Kopf et al.,

2008; Narasimhan and Nayar, 2003a; Narasimhan

and Nayer, 2001; Narasimhan and Nayar, 2003b; Fat-

tal, 2008; He et al., 2011; Tarel and Hautiere, 2009;

Narasimhan et al., 2005). In these methods, the scat-

tering media is considered as obstacles for observing

proper images, and the effect of the scattering media

is eliminated. One of the most popular techniques for

eliminating the effect of scattered media is to control

the light(Narasimhan et al., 2005). In this method,

a controllable light source, such as projector is used

for eliminating the effect of scattering media. By us-

ing the method, we can separate observed intensity

into a direct component which does not include the

effect of scattering media, and an indirect component

which is generated by the scattering media. Although

their method is useful, it requires a controllable light

source, and thus it cannot be used in the ordinary light

source environment.

In this paper, we propose a new approach to esti-

mate the distance from a camera to an object in the

scattering media. From the images taken in scattering

media, we can obtain image intensity, even if image

feature points cannot be extracted properly. Further-

more, the image intensity in scattering media changes

according to the distance from the camera to the ob-

ject because of the scatter of light. Thus the change

in intensity is an important clue to obtain the 3D in-

formation. Thus, we in this paper consider a method

for estimating time-to-contact toward objects from the

change in image intensity obtained in scattering me-

dia.

The estimation of time-to-contact is a traditional

technique, which estimates a time toward collision

of a moving camera to an object. In the existing

methods, the time-to-contact is estimated by using

the change in size of geometric features, such as dis-

tance between two image points in consecutive im-

ages. In recent years, Watanabe et al. (Watanabe

658

Jeong W., Rahadianti L., Sakaue F. and Sato J..

Time-to-Contact in Scattering Media.

DOI: 10.5220/0005362706580663

In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP-2015), pages 658-663

ISBN: 978-989-758-091-8

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

(a) Clear environment

(b) Scattering media

Figure 1: Difference of observed images in clear environ-

ment and in scattering media.

Figure 2: Light scatter in scattering media. The light rays

from a light source are reﬂected at an object surface and

scattered by micro particles.

et al., 2014) proposed a method for estimating time-

to-contact from the change in intensity in images.

Their method does not require geometric image fea-

tures, and hence it can be used even if the geomet-

ric features are not available in images, such as night

images. In this paper, we extend their method and

propose a new method for estimating time-to-contact,

which can be used even in the scattering media, such

as fog and smoke. By using the proposed method,

we can estimate time-to-contact even if we cannot

obtain correspondences from image taken in scatter-

ing media. The proposed method does not need the

reﬂectance of object surface, the properties of light

source and the property of scattering media. Thus,

the method can be applied to various applications in

scattering media.

2 ESTIMATION OF

TIME-TO-CONTACT

In the existing methods, the time-to-contact is esti-

mated from geometric information, such as object

size in images(Cipolla and Blake, 1992; Subbarao,

1990; Horn et al., 2007). In this section, we revisit

the basic theory of the estimation of time-to-contact.

Let us consider the case where an object whose

width is W is observed by a camera as shown in

Fig.3(a). When the distance between the camera and

the object is Z, the apparent width w of the observed

object in a camera image can be described as follows:

w =

(1)

where f is a focal length of the camera. We next con-

sider the case where the camera moves with ∆Z to-

ward the object as shown in Fig.3(b). Then, the ob-

served length w

′

after the motion can be described as

follows:

′

Z + ∆Z

(2)

In this case, the time-to-contact is T = Z/∆Z. The

time-to-contact can be estimated from just observed

images as follows:

T =

′

−w

(3)

In this equation, we do not need any information, such

as the focal length f , distance Z and the real width of

object W for estimating time-to-contact. We just need

w and w

′

in observed images. The technique is very

useful, since we can estimate time-to-contact without

calibrating cameras and without knowing 3D geome-

try. Furthermore, we can reconstruct a real distance Z

if we have the relative speed ∆Z of the camera with

respect to the object.

However, this technique cannot be used when

we cannot obtain appropriate geometric information,

such as corners and edges, in images. For exam-

ple, we cannot observe geometric features appropri-

ately in night images in general, and thus we can-

not estimate time-to-contact from these images. In

order to avoid the problem, Watanabe et al. (Watan-

abe et al., 2014) proposed a method for estimating

time-to-contact from photometric information. They

focused not on geometric information but on photo-

metric information in images for estimating time-to-

contact. By using their method, we can estimate time-

to-contact even if we cannot obtain geometric fea-

tures, such as corners and edges.

Now, let us consider the case where the objects

are in a scattering media, such as fog and smoke,

as shown in Fig.1 (b). In this case, we cannot ob-

serve geometric information appropriately like a night

scene. In addition, the change in intensity caused

by the camera motion cannot be described by a sim-

ple point light source model unlike (Watanabe et al.,

2014). Therefore, we cannot estimate time-to-contact

from the existing geometric and photometric meth-

ods. Thus, we in this paper propose a new method

for estimating time-to-contact in scattering media. In

(a) image at time t (b) image at time t + 1

Figure 3: Geometric information for estimating time-to-

contact.

Time-to-ContactinScatteringMedia

659

Figure 4: Light ray reﬂection by scattering media.

this method, we do not need any information on target

object, such as reﬂectance of object surface. Further-

more, we do not need any information on the scat-

tering media, such as scattering coefﬁcients. Even if

we do not have these informations, we can estimate

time-to-contact just from observed images.

3 LIGHT RAY SCATTER IN

SCATTERING MEDIA

3.1 Single Scattering Model

Now, let us consider a representation of observed in-

tensity in scattering media. In this paper, we assume

that the scattering media is homogeneous, and its den-

sity is not so high. In this case, the behavior of light

rays in scattering media can be represented by a single

scattering model. Thus, we consider the single scat-

tering model (Narasimhan et al., 2005) in this section.

In the single scattering model, input light rays are

reﬂected by micro particles only once. Note that the

reﬂected light rays go forward not to a single direction

but to all directions from the micro media. This prop-

erty is represented by using a phase function P . In

this paper, we use the ﬁrst-order approximation of the

phase function (Narasimhan et al., 2005) as follows:

P (g,α) =

4π

(1 + gcos α) (4)

where g ∈ (−1,1) indicates a parameter controlling

shape of phase function. The α represents angle be-

tween an input light ray and reﬂected direction as

shown in Fig.4.

By using the function P , observed intensity of

light reﬂected by a media is described as follows:

EτP (g,α)e

−τ(d

)

(5)

where τ is a scattering coefﬁcient, E is a radiance of

light ray and d

and d

are distances from the light

source to the particle and the particle to the camera

respectively.

Figure 5: Light ray scatter by scattering media. The light

rays reﬂected by objects are diffused by micro particles in

scattering media.

3.2 Direct and Back-Scattered

Components

We next consider reﬂection in scattering media. In

this paper, we assume that a camera and a light source

are very close to each other. In addition, they are

ﬁxed as shown in Fig.5. This assumption is reason-

able, since in many cases light sources and cameras

are ﬁxed to each other, such as a head lump and a

camera on a vehicle.

In this case, observed intensity I

consists of three

components as follows:

= I

+ I

(6)

where I

is a direct reﬂection of the light from object

surface,

is a back-scattered component from scatter-

ing media and I

is an indirect reﬂection from object

surface as shown in Fig.5.

Let us consider the detail of these components.

We ﬁrst consider the direct component I

. In this

component, light rays from the light source are re-

ﬂected at the object surface, and are observed by the

camera as shown in Fig.5. The light rays are atten-

uated by scattering in the scattering media. In addi-

tion, the magnitude of light is inversely proportional

to squared distance from the light source. Suppose

and d

indicate distances from the light source to

the surface and distance from surface to camera re-

spectively. Then, the observed intensity I

can be de-

scribed as follows:

= Eρ cos θ

−(d

)τ

(7)

where E denotes a power of light, ρ denotes a re-

ﬂectance of object surface, θ denotes an angle be-

tween the light direction and a surface normal and

τ denotes an attenuation coefﬁcient of the scattering

media. In this paper, we assume that the light source

and the camera are very close to each other, and thus

the distance d

and d

can be described by a single

component d approximately. Therefore, Eq.(7) can

be rewritten as follows:

= Eρ cos θ

−2dτ

(8)

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

660

In addition, we assume that the camera and light

source face to the object, i.e. θ ∼ 0, in this paper.

Under this assumption, Eq.(8) can be rewritten as fol-

lows:

= Eρ

−2dτ

(9)

In this paper, we use Eq.(9) as a representation of the

direct component.

We next consider the back-scatter components I

In this component, light rays from the light source are

reﬂected by micro particles and observed by the cam-

era. Note that observed light rays are the integration

of reﬂected light rays by all the particles. Therefore,

observed I

can be represented as follows:

∫

EτP (g,α)e

−2τx

dx = B(d) (10)

As described above, positions of the light source and

the camera are the same, and thus the angle α is 0

approximately. In this case, I

depends only on the

distance d, and hence we describe it as B(d).

Finally, we consider the indirect component I

. In

this component, light rays from the light source are

reﬂected on a object surface. After that, the reﬂected

light rays are scattered by micro particles. In general,

the effect of this component is sufﬁciently small com-

paring with the other components, since the amount

of light reﬂected by the object surface is much smaller

than the light rays from the light source. Thus we ig-

nore the indirect component in this paper.

Therefore, by substituting (9) and (10) into (6),

the observed intensity I

in scattering media can be

described as follows:

= Eρ

−2dτ

+ B(d) (11)

In the next section, we derive a method for estimat-

ing time-to-contact in scattering media by using the

observed intensity I

shown in (11).

4 TIME-TO-CONTACT IN

SCATTERING MEDIA

Let us consider the estimation of time-to-contact in

scattering media. For estimating time-to-contact, we

assume that the observed object faces to the camera

and the light source. Thus, the angle between a sur-

face normal and a light source direction is equal to

0. Also, the object has a texture, which are the re-

ﬂectances at two different points are different from

each other, although these reﬂectances are unknown.

Let ρ(x

) and ρ(x

) denote reﬂectance at point x

and x

. In this case, observed intensity I(x

), I(x

) of

these two points can be described as follows:

I(x

) = Eρ(x

)

−2τd

+ B(d) (12)

I(x

) = Eρ(x

)

−2τd

+ B(d) (13)

Since these equations have the same component B(d),

the back-scattered component B(d) can be eliminated

as follows:

I(x

) −I(x

) = E(ρ(x

) −ρ(x

))

−2τd

= E

−2τd

I (14)

We next consider the case where the light source and

the camera move with the speed of ∆d toward the ob-

ject. Then, the observed intensities I

′

) and I

′

)

after the motion with ∆d are described as follows:

′

) = Eρ(x

)

(d + ∆d)

−2τ(d+∆d)

+ B(d + ∆d)

(15)

′

) = Eρ(x

)

(d + ∆d)

−2τ(d+∆d)

+ B(d + ∆d)

(16)

From these intensities, we can compute

′

after the

motion as follows:

′

) −I

′

) = E(ρ(x

) −ρ(x

))

(d +∆d)

−2τ(d+∆d)

= E

−2τ∆d

(d +∆d)

−2τd

′

(17)

By dividing

I by

′

, we can eliminate the reﬂectance

and the light power as follows:

′

(d + ∆d)

2τ∆d

(18)

Similarly, we compute

′

′′

as follows:

′

′′

(d + 2∆d)

(d + ∆d)

2τ∆d

(19)

where,

′′

denotes

I after the camera motion with 2∆d.

Equation(18) and Eq.(19) includes the same unknown

component e

2τ∆d

. Thus, we eliminate it by taking a

ratio between

′

and

′

′′

as follows:

′2

′′

(d + 2∆d)

(d + ∆d)

(20)

Squared root of this equation can be represented as

follows:

J =

√

′2

′′

d(d + 2∆d)

(d + ∆d)

(21)

Time-to-ContactinScatteringMedia

661

By using the time-to-contact T , the distance d can be

described as d = T ∆d. Thus, Eq.(21) can be rewritten

by using T as follows:

J =

T ∆d(T ∆d + 2∆d)

(T ∆d + ∆d)

T (T + 2)

(T + 1)

(22)

Finally, we can derive the estimation of time-to-

contact T from the solution of Eq.(22) as follows:

T =

−J + 1 ±

√

−J + 1

J −1

(23)

Note that, the equation has two solutions. One of

them indicates time-to-contact when the observed in-

tensities changed from I to I

′

and I

′′

and the other one

indicates when the observed intensities changed from

′′

to I

′

and I. If the light source moves forward to

the object, one of them is positive and the other one is

negative. Therefore, we choose positive one for esti-

mation of time-to-contact.

As shown in the above equation, we do not need

any information about the observed object, camera

and the scattering media, and we can estimate time-

to-contact just from the observed intensities. In the

following section, we show experimental results by

using the proposed method.

5 EXPERIMENTAL RESULTS

5.1 Environment

We show some experimental results by using the pro-

posed method. In this experiment, we utilized syn-

thesized images in order to control the dense of the

scattering media accurately. We assume that a tar-

get object as shown in Fig.6 was situated in scatter-

ing media. The object was taken by orthogonal cam-

era, and thus the position and the scale of the object

are unchanged, even if the distance from the object

to the camera and the light is changed. This is be-

cause the main focus of this paper is the estimation of

time-to-contact from intensities. The number of bits

of these images is 12. The correspondence problem is

not considered in this experiment. The distance from

the camera/light to the object was changed from 20

cm to 10 cm with the speed of 1 cm/sec. Images were

taken by the camera in each second. Examples of the

taken images are shown in Fig.7. The scattering co-

efﬁcient τ was changed from 0.05 to 0.20 as shown

Fig.7. When τ becomes large, the object intensity

decreases rapidly. From these images, the time-to-

contact was estimated by using the proposed method.

Figure 6: Target object.

(i)10 cm (ii)15 cm (iii)20 cm

(a) τ = 0.05

(i)10 cm (ii)15 cm (iii)20 cm

(b) τ = 0.10

(i)10 cm (ii)15 cm (iii)20 cm

Figure 7: Examples of input images: (a), (b) and (c) show

images in scattering media with τ of 0.05, 0.1 and 0.2 re-

spectively.

5.2 Results

We next show the experimental results from the pro-

posed method. Figure8 shows estimated time-to-

contact by the proposed method. Note that estimated

time-to-contact does not appear sometimes in this ﬁg-

ure, since the time-to-contact is not computed when

the change in intensity is smaller than a threshold.

This is because our method is based on the change

in intensity, and thus it cannot provide valid results

when the change in intensity is extremely small.

From these results, we ﬁnd that our method can

estimate time-to-contact when the distance is small

under small scattering coefﬁcient. However, the es-

timated results become unstable when the distance is

large or the scattering coefﬁcient is very small. This

is because the change in intensity becomes very small

under these conditions. In these cases, the estimated

results are affected by various kinds of noise includ-

ing the quantization error.

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

662

8 10 12 14 16 18 20

Ground truth

τ=0.05

τ=0.10

τ=0.20

Distance[cm]

Time-to-Contact[sec]

Figure 8: Estimated time-to-contact under τ = 0.05, τ =

0.10 and τ = 0.20.

15 20 25 30

number of bit

Time-to-contact[sec]

Ground truth

τ=0.05

τ=0.10

τ=0.20

Figure 9: Relationship between the estimated time-to-

contact and the number of bits. The distance from the light

to the object was 15 cm.

In order to clarify the relationship between the

number of bits and the accuracy of estimation, time-

to-contact was estimated with different number of

bits. Figure 9 shows the relationship between the

number of bits and the accuracy of estimation. In

this experiment, distance between the light and the

target was 15 cm, and thus the ground truth of time-to-

contact was 15 seconds. In this experiment, we could

not estimate time-to-contact correctly when a num-

ber of bits are small because of the quantization error.

However, we can estimate the time-to-contact cor-

rectly when the number of bits becomes large. These

results indicate that the proposed method can estimate

time-to-contact from high accuracy images even if the

scattering coefﬁcient is high.

6 CONCLUSION

In this paper, we proposed a method for estimating

time-to-contact in scattering media. In the scattering

media, the behavior of light rays is very complex, and

we need a lot of information on the scattering media

and objects for obtaining 3D information in general.

In spite of the complexity in scattering media, the

proposed method can estimate time-to-contact easily

even if we do not have any information about ob-

served objects, cameras and scattering media. The

method is very useful, since we does not need any

priory knowledge for estimating the time-to-contact.

Thus, the method can be applied to various applica-

tions in scattering media.

REFERENCES

Cipolla, R. and Blake, A. (1992). Surface orientation and

time to contact from image diver-gence and deforma-

tion. In Proc. European Conference on Computer Vi-

sion, pages 465–474.

Fattal, R. (2008). Single image dehazing. In Proc. SIG-

GRAPH, pages 1–9.

He, K., 0001, J. S., and Tang, X. (2011). Single image haze

removal using dark channel prior. In CVPR, pages

1956–1963. IEEE.

Horn, B., Fang, Y., and Masaki, I. (2007). Time to con-

tact relative to a planar surface. In Proc. Intelligent

Vehicles Symposium, pages 68–74.

Kopf, J., Neubert, B., Chen, B., Cohen, M. F., Cohen-

Or, D., Deussen, O., Uyttendaele, M., and Lischin-

ski, D. (2008). Deep photo: Model-based photo-

graph enhancement and viewing. ACM Transactions

on Graphics (Proceedings of SIGGRAPH Asia 2008),

27(5):116:1–116:10.

Namer, E. and Schechner, Y. (2006). Blind haze separation.

In Proc. Conference on Computer Vision and Pattern

Recognition(CVPR), pages 1984–1991.

Narasimhan, S., Nayar, S., Sun, B., and Koppal, S. (2005).

Structured light in scattering media. In Proc. Inter-

national Conference on Computer Vision, pages 420–

427.

Narasimhan, S. and Nayer, S. (2001). Chromatic frame-

work for vision in bad weather. In Proc. Conference

on Computer Vision and Pattern Recognition, pages

598 – 605.

Narasimhan, S. G. and Nayar, S. (2003a). Interactive

deweathering of an image using physical models.

In IEEE IEEE Workshop on Color and Photomet-

ric Methods in Computer Vision, In Conjunction with

ICCV.

Narasimhan, S. G. and Nayar, S. K. (2003b). Contrast

restoration of weather degraded images. IEEE PAMI,

25(6):713 – 724.

Schechner, Y., Narasimhan, S., and Nayar, S. (2001). In-

stant dehazing of images using polarization. In Proc.

Conference onf Computer Vision and Pattern Recog-

nition, volume I, pages 325–332.

Subbarao, M. (1990). Bounds on time-to-collision and rota-

tional component from ﬁrst-order derivatives of image

ﬂow. Computer Vision, Graphics, and Image Process-

ing, 50(3):329–341.

Tarel, J.-P. and Hautiere, N. (2009). Fast visibility restora-

tion from a single color or gray level image. In ICCV,

pages 2201–2208.

Watanabe, Y., Sakaue, F., and Sato, J. (2014). Time-to-

contact from photometric information. IPSJ Transac-

tions on Computer Vision and Applications, 6:39–42.

Time-to-ContactinScatteringMedia

663