![](bg3.png)
Figure 2: Training process consists of face-based PCA
creation, patch-based DCM creation, and facial
appearance error-to-eyes position updating DCM creation.
points of the eyes P by an affine transformation
function
via a parameter
. The initial position
of two inner corner points of the eyes
P
0
(it is
probably occluded) is roughly detected, and K
initializations particles are randomly generated
around
P
0
:
{}
o
PP P=+Δ
. The particle weight
is
measured with its coincidence with the learned
eigenspace
U
(Initial weights of all particles are
equal). Further, to suppress the effects of
illumination differences between different facial
appearances, a illumination
normalization scheme A
l
is performed.
3.2 Local-based Occlusion Recovery
The posterior density, (|,,)Pf bI
for recovery
stage is modeled by a Markov network embedded
with the proposed DCM algorithm. Different from
the common patch-based Markov network
approaches (Freeman et al., 2000; Liu et al., 2001;
Sudderth et al., 2003) that selects the recovered
patches from the training database, the current
approach recovers patches by other non-occluded
patches via the DCM transformation.
Learning Patch Pair l-to-m DCM
[]
TTT
lm
UU
(Fig. 2). For each patch
, {1, 2,..., }lm M∈ (each f is
composed of M patches), the l-m patch pairs of N
training facial appearances
{()}
P are used to train
the combined principal space
[]
TTT
lm
UU
and the
DCM transformation from l to m by Eq. (1) and Eq.
(3), respectively.
Occlusion Detection (Fig. 1.b). The confidence of
visibility of patch l is written as c
l.
which is directly
proportional to the difference between the original
local texture detail of patch l and the reconstructed
texture by the bidirectional DCM transforms from l
to m and from m back to l, where the patch m is one
of neighbor patches of l.
Occlusion Recovery (Fig. 1.b). We solve the
defined Markov network by the nonparametric belief
propagation method (NBP) (Sudderth et al., 2003),
but the recovery order is from non-occluded patches
to the occluded patches sorted based on their
confident values, i.e. the c-value.
3.3 Global-based Face Alignment
After the recovery, the higher-weighted particles are
chosen to form the distribution of
(, | )
bf
′′ ′
in
the face-based measurement step of the probabilistic
propagation stage (Fig. 1.c.1), where only these
correctly aligned ones will be treated as the updating
initializations in the following steps of re-
randomization. According, the summarization of the
current recovered results,
{(), }fP
′
, is the mean of
these particles,
1
[]
N
ii
i
Ef f
=
==
∑
.
Learning the Position-facial Appearance DCM
[]
TTT
fP
UU
ΔΔ
(Fig. 2). Each training image I generates
N’ perturbed facial appearances, {( )}
PP+Δ , by
disturbing elements of the manually labeled position
P. Subsequently,
'NP
Δ of N training images and
their corresponding facial appearance difference
generated by
U
are used to train the DCM
combined principal space
[]
TTT
fP
UU
ΔΔ
and the DCM
transformation from
to
PΔ
, respectively.
Face Alignment. The drift step (Fig. 1.c.2) updates
positions
{}PP P
+Δ from the given facial
differences,{}
based on the combined space
[]
TTT
fP
UU
ΔΔ
in order to form the transition
probability,
(, | , )
bb
ξ
′
. Finally, a diffuse step
(Fig. 1.c.3) is done on these higher-weighted
particles to generate several copies and shift them to
the neighbors of the updated position,
{}
P
′′
+Δ
.
The new set of particles then forms the distribution
of
(, | )
bf
for the following step.
4 EXPERIMENTAL RESULTS
The performance of the proposed recovery system
was evaluated by performing a series of
experimental trials using training and testing
databases comprising 100 and 50 facial images,
respectively, where specific facial feature regions of
(, ) {1, , }lm M∀∈…
2
M×
Local Patch-based
DCM Creation :
DCM:
[]
TTT
m
l
UU
ˆ
FF
Δ= −
Δ
A
g
: Geometric normalization
A
l
: Illumination normalization
S: Samp ling of P
Pr: Projection to eigenspace
Re: Reconstruction based on eigenspace
L
M
: Sep arate as M local p atches
1
{}
iii
P
=
+
N
1
{()}
iii
FP
=
N
m-patch
N
'
11
{{ }}
N
ii jji
IPP
==
++Δ
N
'
11
{{ ( )} }
NN
ii jji
PP
==
+Δ
'
N×
'
11
ˆ
{{ ( )} }
NN
ii jji
FP P
=
+Δ
'NN
[]
TTT
Fp
UU
ΔΔ
Facial Appearance and Eye
Position DCM Creation :
DCM:
Global Face-based
PCA Creation:
Eigenspcae:
[]
U
l-patch
S
A
g
A
l
A
g
A
l
Pr+
Re
1
1
N
1
N
1
1
1
1
()m
()l
L
M
NON-PARAMETRIC BAYESIAN ALIGNMENT AND RECOVERY OF OCCLUDED FACE USING DIRECT
COMBINED MODEL
497