VISUAL FACIAL AGEING USING PLS

Visual Ageing of Human Faces in Three Dimensions using Morphable Models and

Projection to Latent Structures

D. W. Hunter and B. P. Tiddeman

School of Computer Science, Jack Cole Building, North Haugh, St. Andrews, KY16 9SX, U.K.

Keywords:

Face, Facial ageing, Morphable models, Statistical modelling, Partial least squares.

Abstract:

We present an approach to synthesising the effects of ageing on human face images using three-dimensional

modelling. We extract a set of three-dimensional face models from a set of two-dimensional face images by

ﬁtting a Morphable Model. We propose a method to age these face models using Partial Least Squares to

extract from the data-set those factors most related to ageing. These ageing related factors are used to train an

individually weighted linear model. We show that this is an effective means of producing an aged face image

and compare this method to two other linear ageing methods for ageing face models. This is demonstrated

both quantitatively and with perceptual evaluation using human raters.

1 INTRODUCTION

Accurate prediction of how a person’s appearance

will vary with age has a variety of applications, such

as aiding in the search for missing persons, planning

cosmetic surgery, as well as applications in the ﬁlm

industry and other visual arts. Since most researchers

have concentrated on manipulating 2D images, 3D

statistical models are a relatively recent innovation. In

this paper we develop 3D models of ageing by ﬁtting

a Morphable Model to a set of photographs and intro-

duce a new statistical ageing model based on Projec-

tion to Latent Structures (PLS) also known as Partial

Least Squares.

2 LITERATURE REVIEW

Most previous methods for ageing a facial image have

concentrated on transforming a 2D image. Cardioidal

Strain was an early method that relied on the simi-

larity between the mathematical function and facial

ageing in children (Pittenger and Shaw, 1975; Pit-

tenger et al., 1975; Mark and Todd, 1983; V. Bruce,

1989). This was later used in a modiﬁed form by

Ramanathan and Challappa (Ramanathan and Chel-

lappa, 2006). Rowland and Perrett used Triangulated

Linear Warping to deﬁne an ageing trajectory by the

average prototypes for two age-groups (Rowland and

Perrett, 1995). Lanitis et al. trained a statistical model

over a set of face images parametrised by a Principle

Components Analysis model (Lanitis et al., 2002).

Scandrett et al. also used PCA on a set of 2D images,

ageing them using a piecewise linear model, combin-

ing the ageing trajectories between age-groups with

an historical ageing trajectory from younger images

of the individual (Scandrett et al., 2006). Suo et al.

explored a different approach by describing the face

using a Grammatical Model, (Xu et al., 2005) con-

sisting of a hierarchical set of face components. An

input face was aged using a Dynamic Markov Chain

(Suo et al., 2007).

The idea of Modelling ageing using 3D models

has been around for some time. Mark and Todd

applied Cardioidal strain to a 3D model (Mark

and Todd, 1983), Hutton and Buxton used Kernel

Smoothing to create an ageing model of a set of

3D models (Hutton et al., 2003). More recently

Scherbaum el al. (Scherbaum et al., 2007) ﬁtted a

Three-dimensional Morphable Model to a database

of laser scanned cylindrical depth-maps. They used

these models to train a Support Vector Regression

model, synthesized a new face mode from by ‘step-

ping’ through the curved SVR space using a fourth or-

der Runge-Kutta algorithm. The curved nature of the

SVR model meant that the ageing paths were differ-

ent depending on the parameters of the input face thus

creating a semi-individualised model. However they

had only one sample per subject to train the model

340

W. Hunter D. and P. Tiddeman B. (2009).

VISUAL FACIAL AGEING USING PLS - Visual Ageing of Human Faces in Three Dimensions using Morphable Models and Projection to Latent

Structures.

In Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, pages 340-345

DOI: 10.5220/0001793403400345

 SciTePress

and so captured population variations and not nec-

essarily the variations due to ageing in a particular

individual. Park et al. (Park et al., 2008) ﬁtted a

three-dimensional Morphable Model to a set of face

images by ﬁtting an Active Appearance Model and

extracting a three-dimensional model from the AAM.

Ageing was performed by calculating a set of weights

between an input face and exemplar faces in the same

age group. These weights are then used to build an

aged face as a weighted sum of the corresponding

faces at the target age.

Since many of the statistical methods used lost

textural detail such as wrinkles, a few researchers de-

veloped methods that attempted to create appropri-

ate textural detail in aged images. Tiddeman et al.

used a wavelet transform (Tiddeman et al., 2001)

and Markov Models (Tiddeman et al., 2005), Hussein

used Bidirectional Reﬂectance Distribution Functions

(Hussein, 2002) and Gandhi used Gaussian ﬁlters

(Gandhi et al., 2004). These methods work by at-

tempting to replace or adjust the high-frequency com-

ponents of the image to match the high-frequency

components of a prototype at the target age.

3 OVERVIEW

Our aim is to be able to take an image of a particu-

lar person and to create an ageing trajectory speciﬁc

to that particular individual. Using a set of 3D face

models we ﬁrst separate those factors most related to

ageing. Given a training-set of 3D models contain-

ing a ‘snap-shot’ of a number of individuals at var-

ious age points, from childhood to early-adulthood,

we then train a set of ageing trajectories for each in-

dividual. Finally these trajectories are applied as a

weighted sum of trajectories from the training-set.

3D data-sets featuring face models from the same

individual at various age points are rare and incom-

plete, however 2D data-sets are more readily avail-

able. We therefore opted to use a face-ﬁtting method

to extract a 3D Morphable Model (Blanz and Vetter,

1999) from a two-dimensional image. We obtained a

set of photographs by asking some student volunteers

to supply images from a number of key ages. The re-

sulting image set was divided into three strata, Mid

Child containing individuals aged 5 to 8 year, Late

Child covering 8 to 12 year-olds and a Student age-

group between 17 and 23 years. The data-set con-

tained 35 individuals, with one face model per indi-

vidual in each strata.

A set of three-dimensional face models is required

to construct the Morphable Model. We captured

a set of 106 face models from volunteers using a

3dMD scanner (http://www.3dmd.com). The individ-

uals ranged in age from 2 to 65 years old, the average

age was 22.7 with a standard deviation of 17.45 years.

In this paper, we ﬁrst brieﬂy outline a process by

which we generate the 3D models. We then describe

and compare three ageing mechanisms. One, based

on average Prototypes is the 3D analog of the 2D

method used by Rowland and Perrett (Rowland and

Perrett, 1995). The second, an Individualised Lin-

ear model, is the 3D analog of work by Lanitis at

al (Lanitis et al., 2002) and is similar to the method

of Park et al. (Park et al., 2008). We introduce a

new technique based on Partial Least Squares (Wold,

1966).

4 THREE DIMENSIONAL

MORPHABLE MODELS

Three-dimensional Morphable Models introduced by

Blanz and Vetter use Principle Components Analy-

sis to describe the space of human faces as a set of

orthogonal basis vectors. Given a set of 3D dimen-

sional face models with a one-to-one correspondence

between vertices, we vectorise the vertex positions

and colour values and centre each face by subtract-

ing the mean of all the faces. PCA is then performed

on the shape and colour values separately to produce a

set of basis vectors. A reduced set of 40 eigenvectors

for each of shape and colour were used to describe

the face space, denoted s

, t

respectively. The shape

s and colour t of a new face are generated as,

s =

s +

∑

j=1

, t =

t +

∑

j=1

(1)

where

s and

t are the averages of the shape and colour

respectively. The weights α

and β

form the param-

eter vectors α and β, which we concatinate to form

p = α, β. New faces are created by varying these pa-

rameters. This process is described in more detain in

(Blanz and Vetter, 1999).

4.1 Fitting a Morphable Model to a

Face Image

Three-dimensional scanning equipment is a rela-

tively recent invention, and so databases of three-

dimensional models of the same individual taken over

a period of many years have yet to be built. However

two-dimensional images, in the form of photographs

are widely available. In order to build a set of face

models we attempt to extract three-dimensional in-

formation from these images. Our ﬁtting method

VISUAL FACIAL AGEING USING PLS - Visual Ageing of Human Faces in Three Dimensions using Morphable Models

and Projection to Latent Structures

341

was a simple adaptation of the Lucas Kanade Tomasi

algorithm (Baker and Matthews, 2002) from two-

dimensional face models to three-dimensional mod-

els, this method is similar to that detailed by Blanz

and Vetter (Blanz and Vetter, 1999). We use a Taylor

series expansion of the l

-norm of the pixel difference

between an input image and the rendered Morphable

Model to ﬁnd the parameters that minimise this differ-

ence. To improve the accuracy of the ﬁtting a set of

delineated feature points on the two-dimensional im-

age are also matched to their corresponding points on

the Morphable Model using the l

-norm of their sepa-

rating distance when projected onto the image plane.

The result of the ﬁtting operation is a set of vectorised

shape and colour parameters p that describes the face

contained in the two-dimensional input image as a

three-dimensional Morphable Model. Figure 1 shows

an example of results of ﬁtting a Morphable Model

using this technique.

Figure 1: An example of a Three-dimensional Morphable

Model ﬁtted to a face image. The image on the top left is

the original photograph. The image on the top right shows

the results of the ﬁtting, rendered in-situ. The images on the

bottom row are of the same model rendered under neutral

lighting conditions from different angles.

5 AGEING METHOD

We applied the face-ﬁtting method outlined in the pre-

vious section to the photographs in the training set, to

produce a set of 3D models of each individual at mul-

tiple age-points. We now use this training set to create

an ageing model.

5.1 Age Prototypes

Prototype face-models were created for each age-

stratum by averaging the parameters over all faces in

the stratum.

∑

(2)

where

are the parameters of the averaged face

model of all the faces in the stratum s. Here p

the vector of parameters for the i

face model in the

stratum and m is the number of faces in the stratum.

A linear transform is deﬁned between from stra-

tum j to stratum k we take as,

t =

−

ˆa

− ˆa

(3)

where ˆa

and ˆa

are the average ages of the individu-

als within strata j and k respectively. An input f

stratum j is aged towards the age group of stratum k

by moving it in the direction of the vector t and mul-

tiplying t by the desired number of years.

= f

+ (a

− a

)t (4)

where f

is the set of model parameters at the target

age, a

and a

are ages of the input face and the target

age respectively. Clearly this transform is most valid

if the target age is within the range of years of the

target stratum k.

5.2 Individualised Linear Transform

It is well known that faces do not age in an identi-

cal manner. In order to generate an ageing trajectory

for an unseen individual we exploit the relationship

between appearance and ageing trajectory. For each

individual in the data-set a linear ageing path is de-

ﬁned as a vector from one sample face in the start-

ing stratum to another in the target stratum containing

the end age. If no suitable pair of sample faces can

be found the individual is excluded from the data-set.

We denote s,e as the start and end ages of the trans-

form respectively, and p

and q

as the parameters of

the face models of the i

individual taken from the

start and end strata respectively. We deﬁne a single

linear ageing function such that the j

parameter of

the face model of the individual i at time t is,

f(t)

= t.a

i, j

+ b

i, j

(5)

where a and b are sets of weights and a

i, j

and b

i, j

are

the j

weights for the i

individual in the training

set. a deﬁnes the gradient of the path in ℜ

and b the

parameters of the face at time t = 0. These are deﬁned

as,

a =

q − p

e − s

, b = p − sa (6)

VISAPP 2009 - International Conference on Computer Vision Theory and Applications

342

These functions can be parametrised using a

and

to describe the ageing function f

for the i

indi-

vidual. A new ageing path for an unseen individual

can be created using a linear weighted sum of the pa-

rameters of the ageing functions for each individual

in the training set.

∑

= 1 (7)

where ρ

are a set of weights relating the unseen in-

dividual to the ageing path of the i

individual in the

data-set. The ρ

’s sum to one, so that that function

does not add a scaling factor to the ageing path.

As in (Lanitis et al., 2002) the weighting ρ is

deﬁned using the probability distribution of the PCA

space of the face model.

ρ(p

, p

) = e

−

∑

in, j

−p

i, j

)

2σ

(8)

where p

and p

are the parameters of the input and

face model respectively. p

in, j

is the j

parameter

of the input face model. σ

is the variance of the PCA

space in the j

dimension.

This is similar to the method by (Park et al.,

2008). Equation (7) can be combined with equa-

tion (1) to derive their method. Ours differs in that

the weights are based on the PDF of the Morphable

Model rather than linear interpolation.

5.3 Partial Least Squares Ageing

The data-set of parameters contains a signiﬁcant

amount of information that is not relevant to ageing.

Any statistical analysis needs to separate those factors

related to ageing from those that are not related either

explicitly or implicitly.

Partial Least Squares (Wold, 1966) also known as

a Projection to Latent Structures is a statistical distri-

bution similar to Principle Components Analysis that

describes mean centred data as a weighted linear com-

bination of basis vectors. Unlike PCA, which ﬁnds

directions of maximum variance in the data, PLS at-

tempts to describe a set of dependent variables from a

set of predictors. It works by extracting a set of latent

vectors that decompose both the dependent and the

independent matrices in such a manner as to explain

as much of their covariance as possible.

We take the parameters of the face models in the

data-set f

and use them to build the matrix X =

, f

, . . . , f

]

such that each row contains the pa-

rameters of an individual face model. We deﬁne

Y = [age

, age

, . . . , age

]

where age

is the corre-

sponding ages to the i

face. The rows of both X

and Y are then mean centred and scaled by the inverse

standard deviation

As described by (Abdi, 2007), we aim to de-

compose the independent variables as X = T P

with

T = I. T is the score matrix and P is the loading

matrix. We estimate Y as

Y = T BC

. The diagonal

matrix B holds the regression weights, and C is the

weight matrix of the dependent variables. See (Abdi,

2007) for further details on what these mean in prac-

tice. The columns of T are the latent vectors that form

an exact decomposition of X but only an approxima-

tion to Y . The decomposition is found using an iter-

ative algorithm whereby, each iteration, a latent vec-

tor is found that maximizes the covariance between X

and Y and is then subtracted from both. The propor-

tion of variance explained by this vector is found by

dividing the sum of squares of the residuals by the the

sum of squares of the input matrices X and Y .

PLS, like PCA, can be truncated such that a

smaller number of basis vectors are found that ap-

proximately span the space of X . We found that the

ﬁrst 6 latent vectors explained 56.3% of the variance

and showed little improvement in accuracy thereafter.

So we trucated the PLS space to 6 latent vectors.

We separated the parameters into two compo-

nents; the components most related to ageing and a

remainder. As the data used to train the PLS model

has been converted to Z-scores by centring the data

on the mean and scaling by the standard deviation,

we must convert the parameters of the input face f to

Z-scores also. We denote the Z-score converted face

f. The parameters of a face model in PCA space

are related to the parameters of the face in PLS space

f ≈ gP. Since the loading matrix P is not gener-

ally orthogonal in PLS regression, g is approximated

using least squares regression,

g = (P

−1

f (9)

The PCA face model parameters can be recovered

from the PLS space as

= gP and converted from

Z-scores to the original PCA parameter space using

σ +

In general the recovered

f, so we compute the

residual r as

f = gP + r. Ageing is performed using

the Individualised Linear ageing Transform described

earlier on the PLS model parameters (g) instead of the

PCA model parameters (p). After the face is aged the

residuals r are added back in.

The results of ageing a face model using these

methods is shown in ﬁgure 2.

5.4 Quantitative Evaluation

In order to determine the comparative effectiveness

between different methods of ageing we used the Ma-

VISUAL FACIAL AGEING USING PLS - Visual Ageing of Human Faces in Three Dimensions using Morphable Models

and Projection to Latent Structures

343

Figure 2: Examples of aged face images. The ﬁsrt column

shows the original model, the second the same individual at

the target age, the remaining columns show, left to right, the

original model aged using, Prototyping, Individual Linear

transforms and the PLS method.

halanobis distance to measure the similarity between

the aged face model and a face model captured by

ﬁtting the Morphable Model to a image of the same

individual at the target age. We used leave-one-out

cross-validation to evaluate the methods. Figure 1

shows the results of ageing using the prototyping, in-

dividualised linear and PLS methods. We can clearly

see that the ‘Individual Linear’ method gives an im-

provement in accuracy over the ‘Prototyping’ method

with a lower average error and the PLS method shows

a marked improvement over both.

Table 1: RMSE between shape and colour parameters of

aged face model and a known ground-truth model for each

individual in the data-set. With 93 subjects for each method.

Ageing Method RMSE Standard Deviation

Prototyping 8.86 1.84

Individual Linear 8.69 1.92

PLS 7.4 1.4

5.5 Perceptual Evaluation

Quantitative measures may miss ageing cues that hu-

man raters would be able to detect. We performed a

series of tests with human raters to evaluate the ability

of the methods to produce images of the required age.

Each user was shown a single image of a rendered

face model at a time and asked to estimate the age of

the face shown. The age is selected from a range be-

tween 5 and 30 to the nearest year. The stimuli are

a selection of mid-child faces aged to student age by

the three-methods, prototyping, individualised linear

and PLS, together with the rendered face-models of

the individuals at the source and target age. The im-

ages were presented with uniform lighting and pose

on a black background and in random order. No pe-

ripheral details such as hair were on display, limit-

ing ageing cue to those in the face. The images were

presented on public website which generated a signif-

icant amount of trafﬁc, with an average of 105 age

estimations per image, and just under 5000 for each

ageing method being trialled.

Table 2 shows the mean perceived age in years

for the face models aged by the different methods, as

well as the mean ages of the rendered models of the

original face models. Table 3 shows the mean age

difference in years between the perceived age of the

individual after the ageing method is applied and the

target age the algorithm was attempting to recreate.

We can see that all the methods succeed in ageing the

faces towards the target age, but vary in how much

they age the face model. The PLS method achieved

the closest results to the target age of all the age-

ing methods. The original student and mid-child age

groups showed a signiﬁcant error implying that some

age related information was lost by the ﬁtting process.

Table 2: Mean (µ) and standard deviation (σ) of the human

rated ages for faces ages by each method.

Ageing Method µ σ Count

Prototyping 17.048 6.7605 5090

Individual Linear 16.801 6.8449 5092

PLS 17.115 6.6780 4987

Student 17.026 5.9044 6205

Mid Child 12.762 6.0626 4678

Table 3: Mean (µ) and standard deviation (σ) of the error in

years in human rated ages for faces ages by each method.

Ageing Method µ σ Count

Prototyping -3.6614 6.1888 4596

Individual Linear -3.8674 6.1688 4646

PLS -3.5643 6.1098 4551

Student -3.3737 5.4273 5855

Mid Child 6.2135 6.0815 4678

6 CONCLUSIONS

We have described a method of ageing 3D Morphable

Models by a method based on Projection to Latent

Structures or Partial Least Squares. This method

shows an improvement over the others tested both

in quantitative measures, in terms of similarity to a

known ground-truth, and in perceptual evaluation by

human raters. Due to its reliance on face-ﬁtting meth-

ods the success of this method depends on the qual-

ity of the face model produced in the ﬁtting stage.

VISAPP 2009 - International Conference on Computer Vision Theory and Applications

344

Improved ﬁtting techniques or a database of three-

dimensional scans of the same person over several

year, would improve the accuracy of these ageing

methods. Other authors have used Quadratic and Cu-

bic functions (Lanitis et al., 2002) in two-dimensions,

or non-linear Kernel methods such as Support Vec-

tor Regression (Scherbaum et al., 2007) in three-

dimensions, so an obvious extension is to examine

non-linear individualised ageing paths.

ACKNOWLEDGEMENTS

This work was supported by Unilever PLC and the

EPSRC. We would also like to thank David Perrett

for providing the 2D photographic data-set, and for

his advice on conducting perceptual evaluation.

REFERENCES

Abdi, H. (2007). Partial least square regression (pls re-

gression). In In N.J. Salkind (Ed.): Encyclopedia of

Measurement and Statistics., pages 740–744. Thou-

sand Oaks (CA): Sage.

Baker, S. and Matthews, I. (2002). Lucas-kanade 20 years

on: A unifying framework: Part 1. Technical Re-

port CMU-RI-TR-02-16, Robotics Institute, Carnegie

Mellon University, Pittsburgh, PA.

Blanz, V. and Vetter, T. (1999). A morphable model for

the synthesis of 3d faces. In SIGGRAPH ’99: Pro-

ceedings of the 26th annual conference on Computer

graphics and interactive techniques, pages 187–194,

New York, NY, USA. ACM Press/Addison-Wesley

Publishing Co.

Gandhi, M. R., Levine, M. D., and G, M. R. N. (2004). A

method for automatic synthesis of aged human facial

images. Technical report, Masters thesis, McGill Uni-

versity,2004. 1.

Hussein, H. K. (2002). Towards realistic facial modeling

and re-rendering of human skin aging animation. In

SMI ’02: Proceedings of the Shape Modeling Inter-

national 2002 (SMI’02), page 205, Washington, DC,

USA. IEEE Computer Society.

Hutton, T. J., Buxton, B. F., Hammond, P., and Potts, H.

W. W. (2003). Estimating average growth trajecto-

ries in shape-space using kernel smoothing. Medical

Imaging, IEEE Transactions on, 22(6):747–753.

Lanitis, A., Taylor, C. J., and Cootes, T. F. (2002). Toward

automatic simulation of aging effects on face images.

IEEE Trans. Pattern Anal. Mach. Intell., 24(4):442–

455.

Mark, L. and Todd, J. (1983). The perception of growth

in three dimensions. Perception and Psychophysics,

33(2):193–196.

Park, U., Tong, Y., and Jain, A. K. (2008). Face recogni-

tion with temporal invariance: A 3d aging model. In

FGR06.

Pittenger, J. and Shaw, R. (1975). Aging faces as viscol-

elastic events: Implications for a theory of nonrigid

shape perception. J. Experimental Psychology: Hu-

man Perception and Performance, 1(4):374–382.

Pittenger, J., Shaw, R., and Mark, L. (1975). Perceptual

information for the age level of faces as a higher or-

der invariant of growth. J. Experimental Psychology:

Human Perception and Performance, 5(3):478–493.

Ramanathan, N. and Chellappa, R. (2006). Modeling age

progression in young faces. In IEEE Computer Vision

and Pattern Recognition or CVPR, pages I: 387–394.

Rowland, D. A. and Perrett, D. I. (1995). Manipulating fa-

cial appearance through shape and color. IEEE Com-

puter Graphics and Applications, 15(5):70–76.

Scandrett, C. M., Solomon, C. J., and Gibson, S. J. (2006).

A person-speciﬁc, rigorous aging model of the human

face. Pattern Recogn. Lett., 27(15):1776–1787.

Scherbaum, K., Sunkel, M., Seidel, H.-P., and Blanz, V.

(2007). Prediction of individual non-linear aging tra-

jectories of faces. In The European Association for

Computer Graphics, 28th Annual Conference, EURO-

GRAPHICS 2007, volume 26 of Computer Graphics

Forum, pages 285–294, Prague, Czech Republic. The

European Association for Computer Graphics, Black-

well.

Suo, J., Min, F., Zhu, S., Shan, S., and Chen, X. (2007). A

multi-resolution dynamic model for face aging simu-

lation. In IEEE Computer Vision and Pattern Recog-

nition or CVPR, pages 1–8.

Tiddeman, B., Burt, M., and Perrett, D. I. (2001). Proto-

typing and transforming facial textures for perception

research. IEEE Computer Graphics and Applications,

21(5):42–50.

Tiddeman, B., Stirrat, M., and Perrett, D. I. (2005). To-

wards realism in facial image transformation: Results

of a wavelet mrf method. Comput. Graph. Forum,

24(3):449–456.

V. Bruce, M. Burton, T. D. (1989). Further experiment on

the perception of growth in three dimensions. Percep-

tion and Psychophysics, 46(6):528–536.

Wold, H. (1966). Estimation of principal components and

related models by iterative least squares. Multivariate

Analysis, pages 391–420.

Xu, Z., Chen, H., and Zhu, S.-C. (2005). A high resolu-

tion grammatical model for face representation and

sketching. In CVPR ’05: Proceedings of the 2005

IEEE Computer Society Conference on Computer Vi-

sion and Pattern Recognition (CVPR’05) - Volume 2,

pages 470–477, Washington, DC, USA. IEEE Com-

puter Society.

VISUAL FACIAL AGEING USING PLS - Visual Ageing of Human Faces in Three Dimensions using Morphable Models

and Projection to Latent Structures

345