derivative images would have to recalculated in their
entirety. The most promising was a multi-feature fit-
ting strategy that combined, in a Bayesian fashion, a
set of different differentiable cost functions designed
to extract different aspects of the image; for exam-
ple, edges and particular illumination artefacts such
as specular reflection (Romdhani, 2005). Like previ-
ous methods, these functions were differentiable and
required a good initial estimate of parameters. Fag-
gian et al. adapted the method for multiple views of
the same face, however we will be working with just
one view (Faggian et al., 2008).
Xiao et al. used a 2D to 3D method whereby
an Active Appearance Model was constructed from
a 3DMM. Thus methods developed to fit and track
AAMs can be used with 3D models. However the
combined model also spans a large set of parameter
values that result in invalid 3D shape models (Xiao
et al., 2004). These methods all suffer from both
the local-minima and windowing problems described
above.
Fitting a model by matching it to prominent fea-
tures in the target image is an appealing option. The
most obvious of these are the boundaries such as
those between the face and background and inter-
nal boundaries such as the edges of eyes, the mouth
etc. Moghaddam et al. used face silhouette taken
of the same source from multiple angles to capture
a 3DMM. They used an XOR based cost function
where a high cost was applied to silhouette edge
points that are found in one image but not at the equiv-
alent point in the other. Not all the boundaries on the
images and models are appropriate for fitting. Hair,
for example, provides false edges, and the model it-
self can provide false silhouettes as it is defined over
the face only and not the full head. The cost func-
tion was therefore weighted towards appropriate sil-
houettes (Moghaddam et al., 2003).
A number of techniques make use of shape-from-
shading, solving a partial differential equation linking
the image intensity to the reflectance map based on
the assumption that the surface is Lambertian. Patel
and Smith estimated the 3D shape by minimising the
arc-distance between the surface normal of the Mor-
phable Model and the illumination cone. These con-
straints applied only to vertex points and as such al-
lowed the shape-from-shading model to capture fine-
scale surface details. Current Shape from shading for-
mulations rely on specific lighting and camera set-
ups, for example a distant light source or a light
source at the optical centre of the image. This con-
straint is not present when the lighting model is calcu-
lated by physical simulation (Patel and Smith, 2009).
3 CONSTRUCTING A
MORPHABLE MODEL
Three dimensional Morphable Models, introduced by
Blanz and Vetter, use Principle Components Analysis
to describe the space of human faces as a set of or-
thogonal basis vectors. Given a set of 3D dimensional
face models we find a set of one-to-one correspon-
dences between vertices by delineating key points on
the models, such as eyes, nose, mouth etc. The ex-
emplar is warped into alignment with the target face
using the landmarks to drive a 3D thin-plane-spline
model. Correspondences between face models and an
exemplar face model are found by casting rays out
from the vertices of the exemplar models in the di-
rection of the surface normal at the vertex, the posi-
tion on the target model intersected by the ray is con-
sidered to be the corresponding vertex. The meshes
are remapped by warping the vertices of the exem-
plar mesh to the corresponding vertices of the target
mesh, thus creating a new mesh with the vertex count
and structure of the exemplar and the shape of the tar-
get mesh. Colour is warped similarly using the cor-
respondences defined to between the two shapes. We
concatenate the resulting vertex positions and colour
values as,
s = (x
1
, y
1
, z
1
, x
2
, y
2
, z
2
, ··· , x
n
, y
n
, z
n
)
T
, (1)
t = (r
1
, g
1
, b
1
, r
2
, b
2
, g
2
, ··· , r
m
, g
m
, b
m
)
T
(2)
Each face is centred by subtracting the mean of all
the faces and PCA performed. A reduced set of 40
eigenvectors for each of shape and colour were used
to describe the face space. The shape s and colour t
of a new face are generated as a linear combination of
weighted PCA vectors s
j
, t
j
and the averages
ˆ
s and
ˆ
t.
s =
ˆ
s +
k
∑
j=1
α
j
s
j
, t =
ˆ
t +
k
∑
j=1
β
j
t
j
(3)
With the probability distribution over the PCA face-
space defined as
p(s) ≈ e
−
1
2
∑
i
α
2
i
σ
s,i
(4)
where σ
s,i
is the standard deviation of the i
th
shape
component. The PDF for colour is defined similarly.
The weights α
j
and β
j
form the parameter vectors
α and β. New faces are created by varying these pa-
rameters. In order to render the model a set of camera
parameters specifying the position, pose and scale of
the face relative to a camera position are required. In
the rest of this paper we will be referring to the con-
catenated shape and colour parameters α, β, together
with the camera parameters, as the Morphable Model
GRAPP 2012 - International Conference on Computer Graphics Theory and Applications
116