to short-range capture and monocular projection. The
work reported in this paper is mainly concerned with
long range and stereoscopic projection. We present
our investigations in extending the above techniques
and/or development of new techniques, as may be
needed, due to the new problems posed by long-range
capture, calibration and projection.
3 SYSTEM OVERVIEW
An overview of our system is shown in Figure 2. In
the first stage, the system is calibrated. This includes
calibration of the individual cameras, calibration of
each camera with respect to other cameras, and the
calibration of the projector with respect to the cam-
eras. Next, the geometry of the projection surface is
captured using a structured-light scanning technique.
Using the images and geometry of the surface, the re-
flectance properties at each point are estimated. Sur-
face points with complex reflectance properties i.e.
transparent or translucent are identified. Finally, the
original stereoscopic content is compensated to ac-
count for the reflectance properties of the projection
surface prior to projecting it on the surface.
4 SYSTEM CALIBRATION
Accurate calibration is of imperative importance
when dealing with long-range vision-projection sys-
tems, such as in this case. A small error in image
space i.e. pixels, can lead to vast displacements in
the projected space. In this section we describe our
system calibration process, which involves:
• the calibration of the cameras
• the pose recovery with respect to the cameras and
intrinsic parameter calibration of the projector
4.1 Camera Calibration
Perhaps the most popular technique for calibrating a
camera is the one proposed by Tsai et al. (Tsai, 1987)
and Zhang et al. (Zhang, 2000). Given a set of points
in world space and their corresponding image points,
one can recover both the intrinsic and extrinsic pa-
rameters of the camera. The pinhole camera model is
used to describe these parameters which are specified
by the camera matrix C in equation 1,
C =
α −αcot(θ) u
0
0
β
sin(θ)
v
0
0 0 1
|
{z }
intrinsic
r
11
r
12
r
13
t
x
r
21
r
22
r
23
t
y
r
31
r
32
r
33
t
z
| {z }
extrinsic
(1)
where α = k f
x
, β = k f
y
, ( f
x
, f
y
) is the focal length
on the x and y axis respectively, θ is the skew angle,
u
0
,v
0
is the principal point on the x and y axis respec-
tively and r
1−3
,t
x−z
determine the camera’s rotation
and translation relative to the world. Lens distortion
is also taken into account and is modeled by:
D =
k
1
k
3
P
1
P
2
k
3
(2)
which Bell et al. (Bell et al., 2016) explained in
their work.An inherent assumption during camera cal-
ibration using the above method is that the captured
images of 3D objects with known calibration, say, a
checkerboard, are in-focus. This is indeed the case
for many applications. However, when dealing with
projection mapping in outdoor areas this is not the
case. The cameras and projector are focused on the
projection surface, which is far away. Calibrating the
system using a traditional technique means that the
checkerboard must be also positioned at the same far
distance. Although this does not pose a physical lim-
itation, it most often leads to inaccurate estimation
of the parameters. The reason being that at greater
distances the checkerboard will only occupy a very
small area of the captured image, therefore making
the distribution of world-image correspondences de-
generate. This leads to erroneous calculations. In par-
ticular, distortion parameters cannot be accurately re-
covered in the case where the captured images of the
checkerboard do not provide good coverage across the
entire area covered by the camera. In order to over-
come this problem, we follow an approach similar to
Tyler Bell et al. (Bell et al., 2016) which, instead of
a calibration object, uses projected patterns which are
by-design robust to out-of-focus cameras.
This method encodes feature points into phase
shifted patterns being displayed on a monitor visible
to the cameras. The feature points can then be accu-
rately decoded even when blurred because this does
not affect the phase of the pattern sequence. One
vertical and one horizontal phase map are required
where each vertical/horizontal line has a unique phase
value. Thus, each pixel appearing on the monitor has
a unique pair of (Φ
v
,Φ
h
) to identify the feature. These
phase maps are carried by the phase shifting patterns.
Equation 3 is used to generate N equally phase-shifted
vertical and horizontal fringe patterns and is given by,
I
i
v
(u,v) = 0.5
1 + cos (Φ
v
+ 2iπ/N)
,
I
i
h
(u,v) = 0.5
1 + cos (Φ
h
+ 2iπ/N)
(3)
GRAPP 2017 - International Conference on Computer Graphics Theory and Applications
292