![](bg2.png)
face detection, facial feature localization, face mod-
eling, and face tracking in order to realize a fully
automatic, real-time face replacement system. We
achieved a plausible performance in spite of conti-
nuity and short execution time constraints caused by
real-time video processing. We used Active Appear-
ance Model (AAM) (Cootes et al., 2001) to model the
face, therefore, our study does not put a constraint on
similarity of pose angles as in (Bitouk et al., 2008).
In (Blanz et al., 2004), the performance of the sys-
tem strictly depends on manual initialization. Also,
3D modeling of faces requires more complicated im-
plementation issues. On the other hand, our study
presents a fully automatic way of swapping faces
successfully using a simpler approach. Additionally,
the work in (Gross et al., 2006) achieves face de-
identification using AAM in images. The main differ-
ence of this work from ours is that they used AAM to
produce different appearances by changing model pa-
rameters to de-identify faces, whereas, we use AAM
to obtain face contour in different images, align and
swap them for de-identification.
The rest of the paper is organized as follows; Sec-
tion 2 introduces related work and Section 3 explains
a detailed description on the proposed method. Exper-
imental results and discussions are given in Section 4.
Finally, Section 5 concludes the paper.
2 RELATED WORK
There are both 2D and 3D studies related to face
swapping, face replacement or face de-identification.
The work in (Blanz et al., 2004) presents a sys-
tem that exchanges faces across large differences in
viewpoint and illumination in images. It allows user
to replace faces in images for a hair-style try-on. They
fit a morphable 3D model to both input and target face
images by estimating shape, pose and the direction of
the illumination. The 3D face reconstructed from the
input image is rendered using pose and illumination
parameters obtained by the target image. This system
requires manual initialization for accurate alignment
between the model and the faces images.
The work in (Gross et al., 2006) introduces a
framework for de-identifying facial images. The ma-
jority of the privacy protection schemes currently
used in practice, rely on ad-hoc methods such as pixe-
lation or blurring of the face, they, instead, combine a
model-based face image parameterization with a for-
mal privacy protection model. They change the iden-
tifying points on the face according to the protection
model. They utilize AAM in order to model the faces
and find the identity information on them.
The method proposed in (Bitouk et al., 2008) au-
tomatically replaces faces in images. They construct a
large library of face images which are extracted from
images obtained by the internet using a face detection
software and aligned to a common coordinate system.
Their replacement is composed of three stages: First
they detect a face on the input image and align it to
the coordinate system in order to find candidate faces
which are similar to the input face in terms of pose
and appearance. Then, they adjust pose, lighting, and
color properties of the candidate face images accord-
ing to the input image, and perform swapping with
the input image. They rank the resulting face replace-
ments according to a match distance and display the
top ranked ones as results.
3 METHODOLOGY
3.1 System Overview
We propose a face swapping mechanism in video se-
quences by both combining some known computer
vision techniques, such as face detection and AAM,
and providing a face alignment procedure. We used
modified census transform (MCT) feature-based face
detection (Kublbeck and Ernst, 2006) and Active Ap-
pearance Model (AAM) to locate and model the face
and its attributes. For the face alignment procedure,
we use piecewise-affine warping. The basic steps of
our face swapping approach are shown in Fig. 2.
Mainly, there are two independent execution mech-
anisms; offline and online processes. The offline pro-
cess is a one time procedure to build an AAM used
during the online process. The AAM is built from a
set of landmarked faces and this model is loaded each
time the real-time system is started.
Figure 2: System overview.
The online process is executed each time the sys-
tem starts and the internal steps including face de-
tection and tracking, AAM fitting, face alignment,
face swapping and post processing are sequentially
performed. That is, results of the detection step are
used for the construction of initial shape required in
AAM modeling step and alignment through warping
REAL-TIME FACE SWAPPING IN VIDEO SEQUENCES - Magic Mirror
49