Laplacian Unitary Domain for Texture Morphing

Antoni Gurgu

, Debora Gil

and Enric Mart

Computer Vision Center of Catalunya, Universitat Aut

onoma de Barcelona, Bellaterra , Barcelona, Spain

Department of Computing Sciences, Universitat Aut

onoma de Barcelona, Bellaterra , Barcelona, Spain

Keywords:

Facial Metamorphosis, Morphing, Laplacian.

Abstract:

Deformation of expressive textures is the gateway to realistic computer synthesis of expressions. By their good

mathematical properties and ﬂexible formulation on irregular meshes, most texture mappings rely on solutions

to the Laplacian in the cartesian space. In the context of facial expression morphing, this approximation can

be seen from the opposite point of view by neglecting the metric. In this paper, we use the properties of the

Laplacian in manifolds to present a novel approach to warping expressive facial images in order to generate a

morphing between them.

1 INTRODUCTION

Image morphing is the process of obtaining the se-

quence of intermediate images that transforms an im-

age into another one. Morphing techniques create

powerful visual effects that are widely used in the en-

tertainment industry and have applications in medi-

cal visualization and industrial design (Smythe, 1990;

Zell and Botsch, 2013).

The morphing sequence is obtained as an interpo-

lation between two or more images and it is essen-

tially based on two steps (Wolberg, 1998):

• Warping: Deformation of an image A to another B

and vice versa according to a percentage between

two symmetric deformations (40% and 60%, 70%

and 30%) generating two images.

• Cross Dissolve: Merging two images to get the

ﬁnal one.

As there may be inﬁnite transformations from one im-

age to another, the desired transformation is computed

from a correspondence established between pairs of

key line segments, points or meshes which specify

image features or landmarks. This feature correspon-

dence is used to compute warps that interpolate the

positions of such features across the morph sequence

and extend it to all image pixels. The deﬁnition of

the warping at pixel level is a main challenge, espe-

cially in case of facial images morphing which has to

address the large variability across individuals facial

shape and the elasticity associated to emotions.

There exists several image morphing methods, de-

pending on how the correspondence is established and

how the images are warped. Feature-based morphing

(Beier and Neely, ) uses line segments to establish the

correspondence between the two images. To gener-

ate the sequence at pixel level, the inﬂuence of each

line is blended to interpolate line correspondence to

the whole image warp. A main limitation (Wu and

Liu, 2012) is the identiﬁcation of the key lines de-

scribing the geometry of the image objects that have

to be matched, especially for objects with curved bor-

ders (like faces). Another approximation is to employ

scattered-data interpolation methods (like radial ba-

sis function or thin plate splines (Lee et al., 1996))

to extend landmark correspondence to the whole im-

age. In this case, the warp between images is obtained

by constructing two surfaces (one per image) inter-

polating the set of scattered key points. Though fast

and intuitive, scattered-data interpolation are prone to

introduce distortion artifacts near object boundaries

(Lipman et al., 2008) and blurring and ghosting ef-

fects across in-between images (Wu and Liu, 2012).

Recent methods for reducing the impact of ghosting

and blurring (Wu and Liu, 2012) are complex strate-

gies that require the minimization of an energy using

the alpha-expansion algorithm (Boykov et al., 2001).

Differential geometry representations are com-

monly used for texture mapping and three dimen-

sional meshes remeshing (Yoshizawa et al., 2004;

Floater and Hormann, 2005; Desbrun et al., 2002).

The idea is to map the 3D mesh onto the 2D image

plane and assign the mapped vertices coordinates to

the texture coordinates of original vertices. This is

693

Gurguí A., Gil D. and Martí E..

Laplacian Unitary Domain for Texture Morphing.

DOI: 10.5220/0005362206930699

In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP-2015), pages 693-699

ISBN: 978-989-758-089-5

 2015 SCITEPRESS (Science and Technology Publications, Lda.)

equivalent to ﬁnding a suitable parametric map be-

tween the 3D surface and the 2D texture domain (Spi-

vak, 1965). Many methods (Yoshizawa et al., 2004;

evy et al., 2002; Desbrun et al., 2002) deﬁne such

texture coordinates by using the solution to the Lapla-

cian to compute each parametric coordinate function.

By the mean value theorem (Evans, 1998), solutions

to the Laplacian can be computed on irregular meshes

as solutions to a sparse linear system that assigns

weights to adjacent vertices. Such weights determine

the deformation that the texture will undergo when

mapped back to the 3D surface.

Existing mappings depend on the approximation

considered to deﬁne the weights of the Laplacian

system: barycentric coordinates (Tutte, 1963), mean

value coordinates (Floater and Hormann, 2005), or

angle-based coordinates (either conformal (Eck et al.,

) or area-preserving (Desbrun et al., 2002)). In

any case, all of them approximate the Laplacian

in the cartesian domain which implicitly assumes

an identity metric for computing deformations. A

main limitation for a realistic texture deformation is

that they might introduce areas of high local stretch

(Yoshizawa et al., 2004) that visually distort textured

patterns. Another concern in the context of face ex-

pression synthesis is that none of them guarantees

that the assigned texture coordinates are consistent

across different subjects and expressions. Although,

such registration condition could be forced by using

Dirichlet constrains at some anatomical locations (Xu

et al., 2009; Vera et al., 2014), such constrains could

introduce folds in the ﬁnal parametrization in case of

large deformations across cases.

In this paper, we use the theoretical properties of

solutions to the Laplacian to deﬁne texture coordinate

changes suitable for the synthesis of facial expression

images morphing. In particular, we present the use

of an identity metric with ﬁxed boundary conditions

to deﬁne a unitary texture mapping that allows mor-

phing of texture expressions by straight interpolation

of texture values in the unitary domain. We have ap-

plied our method to the morphing of real frontal face

textures from the BosphorusDB (Savran and Sankur,

2008) public database. Several transitions illustrate

the accuracy and ﬂexibility of unitary coordinates for

expression and identity morphing.

2 EXPRESSIVE TEXTURE

MORPHING IN AN UNITARY

LAPLACIAN DOMAIN

A facial expression is given by the geometry of the

expressive face and a function deﬁning the color and

texture of the face. Therefore, expression morphing

should interpolate face vertex positions and its corre-

spondent texture values. We use the solutions to the

Laplacian to deﬁne a transformation that maps a given

facial geometry to a common unitary domain for the

interpolation of texture values.

Mathematically, a face is a 2D surface, S, that ad-

mits an embedding in R

. Thus, it exists a parametric

map, φ, from a regular domain, D ∈ R

to S:

⊃ D

−→ S ⊂ R

u = (u

, u

) −→ (x

(u), x

(u))

The map φ should be a differentiable bijection be-

tween the surface and D, so that the mapping of the

level curves of the coordinates (u

, u

) deﬁne a mesh

on the surface (Spivak, 1965). In the case of face

parametrization (Floater and Hormann, 2005), D is

usually a common unitary domain (either a square or

a circle), which boundary, δD, is mapped to the face

surface boundary, δS. Surface geometric properties

(Spivak, 1965) are encoded in the 2 × 2 matrix of the

ﬁrst fundamental form or metric, g = (g

i j

)

i, j

deﬁned

from φ ﬁrst partial derivatives (Jacobian).

In this context, face texture is a function from the

parametric domain to the real numbers:

I = I(u

, u

) : S −→ R

(1)

where n = 1 in case of grey level textures and n = 3

in case of color images. Any transformation on the

surface is given by a suitable coordinate change in D,

ϕ(u

, u

) = (v

(u), v

(u)). We will use the solutions

to the Laplacian to compute a coordinate change that

maps any mesh to common unitary coordinates.

Laplacian operators (Evans, 1998) are a powerful

mathematical tool to compute differentiable functions

on manifolds with values ﬁxed at some locations. Let

g = (g

i j

)

i, j

denote the manifold’s ﬁrst fundamental

form and g

−1

= (g

i j

)

i, j

its inverse matrix. The Lapla-

cian is deﬁned (Davies, 1989) as the divergence oper-

ator associated to that metric and a function is har-

monic if the divergence of its gradient cancels. A

function components v

, k = 1, 2 are computed as:

∆

= div

(∇

) =

det(g)

∑

∂

det(g)(

∑

i j

∂

)

= 0

with v

δD

= δv

(2)

for det denoting the determinant of a matrix and δv

differentiable function deﬁned on the boundary of the

domain, δD. In case that (δv

, δv

) ∈ δD, by the max-

imum principle for harmonic functions (Evans, 1998),

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

694

we have that ϕ = (v

, v

) deﬁnes a unique map from

D onto itself.

By setting g = Id, for Id the 2 × 2 identity ma-

trix, and δv

= u

δD

, we obtain the Laplacian in R

and the coordinate change deﬁnes a uniformly dis-

tributed mesh. Further, by setting boundary condi-

tions at equal ﬁxed values, the solution to (2) with

identity metric deﬁnes a coordinate change that maps

any facial surface to equal unitary coordinates.

In the case of interpolation of textured values, sur-

faces correspond to the image domain and the unitary

coordinates can be taken in D = [0, 1] × [0, 1]. Given

two different image surfaces, S

, S

, and their texture

intensities, I

, I

, the following diagram:

, Id)

−1

←−− (D, Id)

−1

−−→ (S

, Id)

induces a composition of the texture intensities I

∗

◦ φ

−1

= I

(φ

−1

), I

∗

:= I

◦ φ

−1

= I

(φ

−1

) deﬁned

at the same coordinate values. Therefore, they can

be linearly interpolated to deﬁne a transformation be-

tween the two textures in the unitary domain:

∗

:= tI

∗

+ (1 − t)I

∗

(3)

for t ∈ [0, 1] the texture morphing sequence parameter.

2.1 Implementation in a Discrete

Domain

Solutions to the Laplacian in manifolds (Davies,

1989) satisfy an interesting property for implemen-

tation of expression morphing. By the mean value

theorem (Evans, 1998), the function v

, k = 1, 2, can

be deﬁned by its values in a ball centered at each point

as the average given by:

(u) =

||B

δB

(

u)d

u (4)

for ||B

the ball area given by the metric g and d

the area element of the ball boundary, δB

The mean value formulation (4) can be easily im-

plemented for irregular surface meshes. Let M =

, T } be a triangular mesh on D, where V

= u

, u

) is a discrete sampling on D and T is a trian-

gulation deﬁning vertex connectivity. Then eq. (4) is

approximated by the weighted sum:

) =

∑

(detg)

∑

∈N

(detg)

) =

∑

i j

∑

∈N

i j

w(u

) (5)

for N

the 1-ring of each vertex in the discrete mesh,

, and (detg)

the metric determinant evaluated at

. It follows that solutions to the Laplacian are fast

to compute in the form of a linear system:

= b (6)

The matrix A codiﬁes the weighted average given by

(5) and b is a vector that contains the boundary con-

ditions. In case that we have the identity metric, i.e.

(detg)

= 1 ∀i, the matrix A is simply the adjacency

matrix of the triangulation and the map ϕ = (w

, w

)

deﬁnes a mesh uniformly distributed inside D, pro-

vided that the adjacency is constant.

Texture interpolation can be easily implemented

using the solutions to (6) as follows. Given two dif-

ferent meshes, M

, M

, and their texture images, I

, their morphing sequence is obtained by interpolat-

ing vertex positions, as well as, their texture values.

Intermediate vertex positions, M

= {V

, T }, can

be directly obtained by a linear interpolation:

= tV

+ (1 − t)V

for the sequence parameter t ∈ [0, 1]. In order to inter-

polate intensity values for all pixels between two dif-

ferent texture images, I

, I

, we should deform images

to obtain two warped versions, I

∗

, I

∗

, such that face

structures are matched. In the context of Laplacian

coordinate changes, this can be achieved by trans-

forming mesh vertices, M

, M

, to common unitary

coordinates in D that take equal values for points be-

longing to the same face structure. Such a transforma-

tion is computed using the solution to the Laplacian

with identity metric.

Intermediate texture values, are computed by eval-

uating formula (3) at the intermediate vertex positions

using the composition with the coordinate map, φ

that maps M

to the unitary coordinates:

:= I

∗

◦ φ

= I

∗

(φ

) =

= (tI

(φ

−1

(φ

)) +(1 −t)I

(φ

−1

(φ

))

for t ∈ [0, 1].

Figure 1 illustrates the whole strategy for texture

morphing using interpolation in Laplacian unitary co-

ordinates. We show meshes (in blue) and the corre-

sponding texture images for: the original space (left

column), the Laplacian unitary domain (central col-

umn) and the ﬁnal textured meshes using the interpo-

lated images in the Laplacian unitary domain (right

column). Top row shows the source data, M

, I

, bot-

tom row the target data, M

, I

, and the central row an

intermediate step of the deformation between source

and target. In the original domain, vertex positions

can be interpolated from source to target as they are

from corresponding points on same face structures.

However, texture values can not be directly interpo-

lated since images are not registered. After transform-

ing to Laplacian unitary coordinates, vertex positions

LaplacianUnitaryDomainforTextureMorphing

695

∗

)

∗

)

∗

)

−1

Image space

Unitary domain

Image space

)

∗

)

∗

Figure 1: Mesh morphing and texture morphing through Unitary Metric parametrization.

are equal for all meshes and, thus, the transformed im-

ages are registered and can be interpolated to create

the intermediate textures. In the right column, such

textures are rendered on the interpolated meshes.

3 EXPERIMENTS

The goal of our experiments is to illustrate the poten-

tial of Laplacian unitary coordinates for the morphing

of textures coming performing different facial expres-

sions. In order to do so, the model described in Sec-

tion 2.1 has been applied to deform 2D frontal face

meshes and textures selected from the BosphorusDB

(Savran and Sankur, 2008) public database

We have considered 3 different kinds of morphing

transitions between: different expressions from the

same person, different persons performing the same

expression and different persons performing different

expressions. The ﬁrst transition checks the compati-

bility/accuracy for expression morphing. The second

transition checks identity morphing accuracy. The

last one is the more general and complex case and

illustrates the ghost effect (Wu and Liu, 2012).

available at http://bosphorus.ee.boun.edu.tr/

1415

22 23

1415

2627

(a) (b)

Figure 2: BosphorusDB Examples in the cartesian, (a), (b),

and unitary Laplacian, (c),(d), domains.

Faces have been sampled using a sparse set of 36

landmarks that deﬁne the main facial structures (eyes,

lips, nose and face boundary). Figure 2 shows two

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

696

t=0 t=0.3 t=0.5 t=0.7 t=1

Image DomainUnitary Domain

Figure 3: Morphing of different expressions of the same subject.

t=0 t=0.3 t=0.5 t=0.7 t=1

Image DomainUnitary Domain

Figure 4: Morphing of the same expression from different subjects.

t=0 t=0.3 t=0.5 t=0.7 t=1

Image DomainUnitary Domain

Figure 5: Morphing of different expressions from different subjects.

examples of the BosphorusDB faces from two differ-

ent subjects, one expressing surprise (ﬁg.2(a)) and the

other one with a neutral expression (ﬁg.2(b)). The tri-

angulation deﬁned by the sparse set of selected land-

marks (numbered from 1 to 36) is shown in blue lines.

Such landmarks have been used to compute the uni-

tary Laplacian coordinates.

Bottom images in ﬁg.2 show the transformation

of texture coordinates and image to the unitary Lapla-

cian domain computed over the mesh of landmarks.

LaplacianUnitaryDomainforTextureMorphing

697

We observe that texture coordinates (blue lines) co-

incide for, both, expressions and subjects. Thus, in-

dividuals and expressions are registered in these new

coordinates. Since, by deﬁnition, such coordinate are

a diffeomorphism (i.e. inﬁnitely differentiable), reg-

istration in the unitary domain guarantees an implicit

registration in the image domain through the inverse

change. This allows the morphing between textures

from different expressions and subjects.

Figure 3 shows the texture deformation of two

different expressions (sadness and happiness) of the

same person, ﬁgure 4 the deformation of the same ex-

pression (anger) for two different persons and ﬁgure

5 the deformation between different identities and ex-

pressions. For all cases, deformations in the carte-

sian image domain are shown in ﬁrst rows, while

the corresponding deformations in the unitary Lapla-

cian domain are shown in second rows. Each column

corresponds to the deformation at different times,

t = 0, 0.1, 0.3, 0.5, 0.7, 1, with the original textures at

t = 0 and t = 1. The implicit registration of texture co-

ordinates achieved in the unitary Laplacian domain al-

lows smooth interpolation of texture color values, re-

gardless of the expression and subject identity. Origi-

nal expressions and identities are recovered by the in-

verse mapping from the unitary to the cartesian image

domain. Since Laplacian coordinate changes are in-

ﬁnitely differentiable, texture deformation is smooth

in both, unitary and cartesian image domains.

4 CONCLUSIONS

Solutions to the Laplacian constitute a unique tool

for deﬁning smooth coordinate changes that could put

into correspondence different meshes. Such an elastic

registration, could be the ﬁnal stage in the synthesis

of facial expressions including their texture. In or-

der to get realistic expressions, the Laplacian system

weights should be tuned according to the geometric

deformation that the surface undergoes, so that coor-

dinate changes solve the Laplacian in a manifold.

In this paper, we have presented a ﬁrst study on

the properties of the Laplacian in manifolds for under-

standing the deformation that the coordinate change

induces. We have shown the potential of such solu-

tions for expression synthesis by presenting a user-

friendly texture morphing and a texture expression

synthesis from a sketch. The promising results en-

courage further research on the use of Laplacian in

manifolds for affective avatars synthesis. First, the

role of the Dirichlet conditions will be investigated

to actually register anatomies without mesh folding.

Such registration will be used to get a model of face

identity and expression for the synthesis of realis-

tic avatars. Simultaneously, temporal models for and

4D data (3D geometry+time) will be used to build a

model of facial expression deformations to animate

static face meshes or to synthesize realistic avatars.

ACKNOWLEDGEMENTS

Work supported by Spanish projects TIN2012-33116

and FFI2012-39056-C02-01. The ﬁrst author is sup-

ported by FPI-MICINN BES-2013-063756 program.

REFERENCES

Beier, T. and Neely, S. Feature-based image metamorpho-

sis. SIGGRAPH’92.

Boykov, Y., Veksler, O., and Zabih, R. (2001). Fast approx-

imate energy minimization via graph cuts. TPAMI.

Davies, E. (1989). Heat Kernels and Spectral Theory. Cam-

bridge University Press.

Desbrun, M., Meyer, M., and Alliez, P. (2002). Intrinsic

parameterizations of surface meshes. In Comp Graph

Forum.

Eck, M., DeRose, T., Duchamp, T., Hoppe, H., Lounsbery,

M., and Stuetzle, W. Multiresolution analysis of arbi-

trary meshes. In SIGGRAPH ’95.

Evans, L. C. (1998). Partial Differential Equations. Amer-

ican Mathematical Society.

Floater, M. and Hormann, K. (2005). Survey in Advances in

multiresolution for geometric modelling.

Lee, S.-Y., CHWA, K.-Y., HAHN, J., and SHIN, S. Y.

(1996). Image Morphing Using Deformation Tech-

niques. J VCA.

evy, B., Petitjean, S., Ray, N., and Maillot, J. (2002). Least

squares conformal maps for automatic texture atlas

generation. ACM TOG.

Lipman, Y., Levin, D., and Cohen-Or, D. (2008). Green

Coordinates. ACM Trans. Graph.

Savran, A. and Sankur, B. (2008). Non-rigid registration of

3D surfaces by deformable 2D triangular meshes. In

CVPRW ’08.

Smythe, D. (1990). A two-pass mesh warping alogrithm for

object transformation and image interpolation. ILM

Computer Graphics Department.

Spivak, M. (1965). Calculus on manifolds. A modern ap-

proach to classical theorems of advanced calculus. W.

A. Benjamin, Inc.

Tutte, W. T. (1963). How to draw a graph.

Vera, S., Ballester, M. A. G., and Gil, D. (2014). Anatom-

ical parameterization for volumetric meshing of the

liver. In SPIE.

Wolberg, G. (1998). Image morphing: a survey. The Visual

Computer.

Wu, E. and Liu, F. (2012). Robust image metamorphosis

immune from ghost and blur. The Visual Computer.

VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications

698

Xu, K., Zhang, H., Cohen-Or, D., and Xiong, Y. (2009). Dy-

namic harmonic ﬁelds for surface processing. Com-

puters and Graphics.

Yoshizawa, S., Belyaev, A., and Seidel, H. P. (2004). A

fast and simple stretch-minimizing mesh parameteri-

zation. Proc. Shape Modeling Int.

Zell, E. and Botsch, M. (2013). ELASTIFACE: Matching

and Blanding Textured Faces. In NPAR ’13.

LaplacianUnitaryDomainforTextureMorphing

699