A Uniﬁed Spectral Embedding for Shape Correspondence

Zizhao Wu, Ruyang Shou and Xinguo Liu

State Key Lab of CAD&CG, Zhejiang University, HangZhou, China

Keywords:

Geometry Processing, Shape Correspondence, Spectral Embedding.

Abstract:

Spectral embedding, as one of shape representative techniques, takes hold of many researchers’ attention

in ﬁeld of shape correspondence. One of the biggest challenges of spectral correspondence method is that

embeddings of different shapes need to be aligned in the embedding space in order to eliminate sign ﬂip and

ordering ambiguity of their eigenfunctions, before seeking for correspondence. In this paper, we introduce

a spectral correspondence method by embedding shapes in a uniﬁed space simultaneously. In the uniﬁed

embedding space, the sample points of the same shape with small intrinsic distances, and from different

shapes with high similarity, are close to each other. Our uniﬁed embedding can be used for correspondence

directly, without need of alignment. Furthermore, the uniﬁed embedding captures both the spatial arrangement

and the feature similarity. Shape correspondence is achieved with such embedding by minimizing an objective

function. Results show the efﬁciency of our method.

1 INTRODUCTION

Shape correspondence (also called shape matching)

is an important prerequisite to many geometry pro-

cessing applications such as statistical shape model-

ing, shape morphing, deformation transfer, shape reg-

istration, and sequential meshes analysis, etc. Existed

shape correspondence methods could be roughly cat-

egorized into rigid and non-rigid cases, regarding to

the types of transformations they involve. In this pa-

per, we suppose the shapes to be isometric, we make

such restriction based on the observation that many

real world deformations, such as articulated shapes in

different poses, preserve pairwise geodesic distances

up to minor errors, which approximately satisﬁes the

deﬁnition of isometry.

Spectral correspondence method as it represents

the shape in a spectral representation, is an impor-

tant tool in ﬁnding non-rigid shape correspondence.

It mainly comprises two key ingredients: represent-

ing shapes using spectral embeddings and matching

the shapes based on their embeddings. To repre-

sent an individual shape using spectral embedding,

some methods usually ﬁrst construct an afﬁnity ma-

trix based on the geometric information of the shape,

and then perform eigen decomposition on the afﬁn-

ity matrix that will produces low dimensional embed-

ding whose entirety is also called the spectrum. While

as some researchers (Jain and Zhang, 2006; Mateus

et al., 2008) pointed out, directly utilizing the embed-

dings to ﬁnd correspondences between shapes causes

fallacious inaccurate results, because there may ex-

ist sign ﬂip and ordering ambiguity in their spectral

representations. To address this issue, several authors

(Jain and Zhang, 2006; Mateus et al., 2008) employed

an additional alignment step, using either brute force

search or greedy approach, which is a reasonable way

but a complicated and exhausting task.

In this paper, we introduce a novel framework

for shape correspondence. We simultaneously embed

the target shapes in one uniﬁed space. The uniﬁed

embedding encodes both the spatial arrangement and

feature similarity as they are captured in the uniﬁed

afﬁnity matrix, where the spatial arrangement is pre-

served for sample points of each shape and simulta-

neously the sample points from different shapes with

high similarity are close to each other. Given such

embedding, the matching results can be achieved by

minimizing an objective function, which is expected

to be solved in polynomial time.

Comparing to state-of-the-art shape correspon-

dence methods based on spectral embedding, our

main contributions and advantages are summarized as

follows:

• Our method takes advantages of the uniﬁed rep-

resentation of spectral embedding for different

shapes, which doesn’t need a further alignment

step as done by some traditional spectral methods.

• To our knowledge, we are the ﬁrst to encode both

Wu Z., Shou R. and Liu X..

A Uniﬁed Spectral Embedding for Shape Correspondence.

DOI: 10.5220/0004278700940099

In Proceedings of the International Conference on Computer Graphics Theory and Applications and International Conference on Information

Visualization Theory and Applications (GRAPP-2013), pages 94-99

ISBN: 978-989-8565-46-4

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

the spatial arrangement and feature similarity in

one embedding space. The uniﬁed spectral em-

bedding is experimented suitable for shape corre-

spondence.

2 RELATED WORK

Descriptor based Approaches. Dealing with the

non-rigid shapes, the appealing descriptor should pre-

serve intrinsic (e.g., geodesic) distances which are in-

variant to inelastic deformations of the shape. Gal

et al. (Gal et al., 2007) extended the shape context

to the non-rigid cases. Elad and Kimmel (Elad and

Kimmel, 2001) developed multidimensional scaling

(MDS) to represent shape in a low-dimensional em-

bedding space and compared them using Euclidean

distance.

However, descriptors based on geodesic distances

are very sensitive to topological noise, since a small

modiﬁcation on the connectivity of the mesh will

change geodesic distances dramatically over a large

part of the shape. Spectral method however is stable

under small changes in the intrinsic metric, as some

authors proved (Jain and Zhang, 2006; Sun et al.,

2009; Ovsjanikov et al., 2010). Among them, the

Heat Kernel Signature (HKS) is a very efﬁcient tool

in extracting features and comparing shapes at differ-

ent scales. Sun et al. (Sun et al., 2009) developed the

HKS which inherits many properties of the heat ker-

nel including isometric invariance, multi-scale geo-

metric information encoding, and robustness to small

perturbations.

Due to the strong properties of the HKS, we em-

ployed it as one of the components of our descrip-

tor measuring the feature similarity between a pair of

shapes.

Spectral Embedding. Besides the HKS, other spec-

tral embedding methods have been well studied.

Since the afﬁnity matrix encodes pairwise geometric

invariants which can also be preserved in the spec-

tral embedding space, in order to describe articulated

motion, Jain and Zhang (Jain and Zhang, 2006) pro-

posed to use geodesic distance to construct the ma-

trix. Mateus et al. (Mateus et al., 2007) took use of

LLE (Roweisand Saul, 2000) and Laplace embedding

(Mateus et al., 2008).

The embeddings of different shapes, obtained

through techniques mentioned above, are incompat-

ible and cannot be used to generate correspondences

directly, as their proposer pointed out the reasons that:

• There exists a sign ﬂip problem when computing

eigenvectors (Av = λv ⇔ A(−v) = λ(−v)).

• There exists an unreliable ordering of its eigen-

values due to large algebraic multiplicity of an

eigenvalue and numerical instabilities in calcula-

tion process.

To deal with the sign ﬂip and the eigenfunction

reordering problems. Jain and Zhang (Jain and

Zhang, 2006) suggested to align the embeddings

by minimizing a cost function. Sahillio˘glu and

Yemez (Sahillio˘glu and Yemez, 2010) aligned the

spectral embeddings using the the same way as (Jain

and Zhang, 2006), and found initialized correspon-

dences by solving a bipartite graph matching prob-

lem. Mateus et al. (Mateus et al., 2008) proposed

to ﬁnd an orthogonal transformation that best aligns

pairwise embeddings, followed by introducing an un-

supervised clustering and the EM algorithm for seek-

ing correspondences of embeddings.

3 ALGORITHM OVERVIEW

In this paper, we focus on the shape matching be-

tween two approximately isometric shapes. We de-

note by S and T the discrete version of both shapes

which are differentiable 2-manifolds in R

, indicat-

ing the source and target shape respectively. Further-

more, we sample a few feature points on them based

on the importance sampling technique (Hilaga et al.,

2001), the sets of sample points of S and T are des-

ignated as K

and K

. By embedding the shapes into

a d-dimensional metric space, we use U

∈ R

and

∈ R

to represent the sets of embeddings of sam-

ple points K

and K

in the embedding space, re-

spectively. In this respect, the innovative point of our

framework is that we achieve U

and U

at once by

simultaneously embed K

and K

into a uniﬁed em-

bedding space. Our ﬁnal goal is to ﬁnd a matching

function φ that maps U

to U

. φ should be mean-

ingful and accurate, at the same time cover the em-

bedding as much as possible, and can be efﬁciently

determined.

Figure 1 illustrates the pipeline of our 3-stage

framework:

In the “Afﬁnity Matrices” stage, we introduce the

spatial afﬁnities which represent spatial constraints of

sample points of each shape and the similarity afﬁni-

ties which represent similarity constraints of sample

points between different shapes. These afﬁnities form

submatrices that constitute the uniﬁed afﬁnity matrix.

We will show it in details in Section 6.

In the “Uniﬁed Embedding” stage, we perform

eigen decomposition on the uniﬁed matrix and ob-

tain the embeddings U

and U

. Since the uniﬁed

afﬁnity matrix are composed of the spatial afﬁnities

AUnifiedSpectralEmbeddingforShapeCorrespondence

Figure 1: Three main stages of the proposed method. We ﬁrst construct the spatial afﬁnities and similarity afﬁnities for both

input shapes, then achieve uniﬁed embedding through eigen decomposition of uniﬁed afﬁnity matrix built in the way of using

spatial and similarity matrices as its block submatrices. Finally, the matching result is obtained by minimizing an objective

function based on the uniﬁed embedding coordinates.

and the similarity afﬁnities, our uniﬁed embedding in-

trinsically possesses the spatial and similarity proper-

ties of signiﬁcance. Both properties are applicable for

shape matching. The spatial arrangement maintains

global consistency of correspondences and the simi-

larity constraints improve matching accuracy. We will

offer a detailed discussion in the next section.

In the “Feature Matching” stage, we determine

the optimal matching φ by minimizing an objective

function with regard to the uniﬁed embedding, which

will be particularized in Section 5.

4 THE UNIFIED SPECTRAL

EMBEDDING

To formulate properties of the embedding mentioned

in Section 3, inspired from the work (Torki and El-

gammal, 2010), for the source shape S , we denote the

spatial afﬁnity matrix P, with the entry p

measures

the spatial distances of i-th and j-th sample point of

S . Similarly, we deﬁne Q with entry q

for the tar-

get shape T . We next deﬁne the similarity afﬁnity

matrix for S as R with respect to T , the entry r

mea-

sures similarity between i-th sample point on S and

j-th on T . The construction of these afﬁnities will be

discussed in Section 6.

Suppose we are given the matrices above, then

the embeddings U

and U

are determined aiming

at minimizing the following objective function

argmin

∑

i, j

−U

∑

i, j

−U

∑

i, j

−U

∑

i, j

−U

(1)

where U

(i ≤ |K

|) denotes the embedding represen-

tation of i-th sample point on shape S . It can be seen

from the objective function that the ﬁrst two terms en-

code the spatial arrangement of two point sets, which

guarantees that two points will be close to each other

in the embedding space if their corresponding afﬁn-

ity entry is relatively high. The last two terms aiming

at keeping points pairs of different point sets close to

each other if value of their corresponding similarity

afﬁnity entry is large.

We simplify the Equation 1 by introducing a ma-

trix U stacks the desired embedding coordinates,

U =



)

, (U

)



, . . . ,u

, u

, . . . ,u

and a uniﬁed afﬁnity matrix A deﬁned as

A =



P R



let a

denotes the element on i-th row and j-th col-

umn of A, the Equation 1 can then be simpliﬁed to

argmin

∑

i, j

−U

, (2)

where U

is i-th element of U.

Note that the Equation 2 can then be rewritten in

the following form

argmin

LU),

where L is the Laplacian of the matrix A, i.e., L = D−

A, and D is the diagonal matrix deﬁned as D

∑

Since each element of U is a multi-dimensional rep-

resentation, according to (Belkin and Niyogi, 2003),

the above objective function reduces to

∗

= arg min

DU=I

tr(U

LU), (3)

the constraint U

DU = I removes the arbitrary scal-

ing. The uniﬁed embedding can be reached through

a generalized eigen decomposition Lu = λDu. We

make use of the bottom d nonzero eigenvectors to

form the optimal solution of U.

GRAPP2013-InternationalConferenceonComputerGraphicsTheoryandApplications

5 FEATURE MATCHING

In this Section, we model the matching problem as a

Linear Assignment Problem, which can be solved in

polynomial time.

Let φ : K

→ K

denote the bijective mapping

function representing the correspondencebetween K

and K

, we achieve the matching results by minimiz-

ing the following matching cost

argmin

∑

ψ(U

φ(i)

), (4)

where ψ(a, b) is the cost function measuring similar-

ity between the embeddings a and b, which in practice

we use L

distance.

Similar to (Jiang and Yu, 2009), we rewrite the

Equation 4 in a succinct matrix form. Let X be the

assignment matrix and C be the cost matrix related to

cost function ψ respectively. Thus, we have



1, φ(i) = j

0, otherwise

= ψ(U

Without loss of generality we assume that |K

| ≥

|, and let tr denotes the trace of a matrix, then

Equation 4 can be solved by

min f(X) = tr(C

X), (5)

subject to

= 1

≤ 1

X ∈ {0, 1}

|×|K

where 1

denotes a column vector of size n with en-

tries equal to 1, X ∈ {0, 1}

|×|K

denotes that the

matching between the candidate pair (i,j) is either

true or false. For the |K

| = |K

| case we force each

row and each column of X containing exactly a sin-

gle one, that means all sample points from source

shape should be matched into target sample points

and vise versa. For more general case where there

exists min(|K

|, |K

|) injections in the matching re-

sult, we allow some undetected matchings, which is

represented as zero elements in X1

or X

In our implementation we employed the lp solve

(lp solve, ) to solve this linear assignment problem.

6 AFFINITIES

6.1 Spatial Arrangement Afﬁnity

The spatial afﬁnity should reﬂect the spatial arrange-

ment of the sample points of each shape, which means

in our case two sample points are close to each other

in the embedding space if the intrinsic distance be-

tween them is small. In order to attain bending in-

variant representation, we use geodesic distances to

deﬁne the relationships between the sample points of

each shape. We further apply a Gaussian kernel ﬁl-

tering on the geodesic distances so as to remove scale

relevance. In summary, for shape S, we construct the

spatial afﬁnity matrix P using the following deﬁnition

= exp

−d

, K

)

2σ

where d

(a, b) is the geodesic distance between sam-

ple points a and b, K

is i-th sample point of shape

S , σ

= w

max

is the Gaussian kernel width with re-

spect to the maximal geodesic distance d

max

among

sample points, with w

usually set to 0.005. We also

construct the spatial afﬁnity matrix Q for T similarly.

6.2 Feature Similarity Afﬁnity

The similarity afﬁnity matrix should reﬂect the fea-

ture similarity of embeddings from different shapes.

In this work, we adopt the HKS (Sun et al., 2009)

as our choice with consideration to its strong proper-

ties as mentioned previously. However, due to the ge-

ometry discretization errors and sampling distortion,

it’s hard to preserve the consistency of signatures in

the diffusion embedding space. Moreover, there also

exists ambiguity problem when dealing with intrinsi-

cally symmetric shapes of the HKS. In order to mea-

sure the feature similarity following a continuity con-

strained way, we employed a geodesic ﬁeld and com-

bine it with HKS. Hence, our similarity afﬁnity matrix

is constructed through the following steps

• Calculate the HKS for each shape, based on which

we then detect anchor correspondences.

• Compute the geodesic ﬁeld using the anchors.

Construct new signature as combination of the

HKS and the geodesic ﬁeld.

• Compute optimized matrix through singular value

decomposition (SVD) on primal afﬁnity matrix

measured on Gaussian kernel on squared Eu-

clidean distance in the signature space.

We follow the work proposed by (Sun et al., 2009)

to calculate the HKS, and extract the anchor corre-

spondences using the method of (Ovsjanikov et al.,

2010).

Given a set of anchors on shape S , denoted by

, ··· , κ

, k < |K

|, for each sampling point we es-

tablish the geodesic ﬁeld deﬁned as

, κ

)

max

, ··· ,

, κ

)

max

AUnifiedSpectralEmbeddingforShapeCorrespondence

where d

max

is the maximal geodesic distance among

sampling points, K

is i-th feature point of shape S .

The combination of the HKS and geodesic ﬁeld is

a straightforward dimensional concatenation, that’s to

say our new signature can be written as



, λy



, (6)

where f

is the combined signature for i-th sample

point of the shape and h

is the HKS. We believe that

a balanced weight λ = 1 is ideal for most cases.

Now we can compute feature similarity afﬁnity

R measured on combined signatures. The afﬁnity

should exclude most of features, while at the same

time, avoid to make any hard decision, i.e., a zero-one

permutation matrix is not the one we pursuit.

To address this issue, according to (Scott and

Longuet-Higgins, 1991), we ﬁrst deﬁne an afﬁnity

matrix

R, whose entry is given as

˜r

= exp

−k f

− f

2σ

where we set the width of Gaussian kernel σ = 0.005.

Then we run the singular value decomposition of the

afﬁnity matrix

R, which we express it in the form

R =

, where

U and

V are orthogonal matrices,

and

S is a non-negativediagonal matrix. Next we con-

vert the matrix

S into an identity matrix E by replac-

ing the singular values of

S by ones, and obtain an op-

timized matrix R

∗

, where each element indi-

cates the extent of pairing between two sample points

of different shapes. The optimized matrix R

∗

corre-

lates best with

R as is shown in (Scott and Longuet-

Higgins, 1991).

Finally, we use the optimized matrix R

∗

to con-

struct the similarity afﬁnity matrix R by setting the

negative values to zero.

7 RESULTS

Time Evaluation. We implemented our framework

in C++ and evaluated the performance on a com-

puter with Intel Core

2 Quad processor and 4GB

of RAM. The running time of our pipeline mainly

depended on the construction of the spatial and sim-

ilarity afﬁnity matrices. For a mesh with about

6000 vertices, these preprocessing stages totally spent

about one minute, with the computation of HKS tak-

ing about more than half of the duration. Fortu-

nately, ﬁnding that the complexity of HKS can be

reduced through a multi-resolution optimization ap-

proach (Vaxman et al., 2010), we consider employing

it in future to extend capability of our framework to

extremely large meshes.

Table 1: Time consumption of the alignment using brute

force search. #f denotes the number of sample points, k

represents the dimension of embeddings. Times are given

in seconds.

#f k = 3 k = 4 k = 5 k = 6

100 0.11 0.72 6.8 73.2

200 0.21 1.8 16.25 180

300 0.38 2.5 20.0 208

400 0.42 2.99 31.6 367.8

Table 2: The matching distortion of our method in com-

parison with (Sahillio˘glu and Yemez, 2010) on the TOSCA

nonrigid world dataset (Bronstein et al., 2008).

Sahillio˘glu et al. Our method

pair D

avg

max

avg

max

Cat 0.032, 0.117 0.024, 0.167

Michael 0.031, 0.119 0.035, 0.165

Victoria 0.014, 0.094 0.021, 0.117

In contrast with the state-of-the-art spectral based

techniques, our method doesn’t entail an alignment

step to eliminate sign ﬂips and orderings between em-

beddings of different shapes. The alignment is quite

time consuming (Jain and Zhang, 2006; Mateus et al.,

2007), for two k-dimensional embeddings, there are

k! possible cases to be veriﬁed. Table 1 shows the

time consumption by using exhaustive search. Al-

though the author of (Mateus et al., 2007) addressed

this problem by searching for an orthogonal transfor-

mation, still the algorithm is low performance.

Matching Results Evaluation. We evaluated match-

ing qualities using triangular meshes from the

TOSCA nonrigid world dataset (Bronstein et al.,

2008) and human bodies generated using the SCAPE

model (Anguelov et al., 2005). We sampled about

200 feature points on each shape that originally have

about 3K to 20K vertices.

We used the average matching error D

avg

and

the maximal matching error D

max

, as deﬁned by

Sahillio˘glu and Yemez (Sahillio˘glu and Yemez, 2011)

to quantify the matching distortion. The quantiﬁed

distortions are provided in Table 2, and visualized

matching results are shown in Figure ??, where we

can observe that our method achieves good results

with less time-consuming.

8 CONCLUSIONS

We proposed a uniﬁed spectral embedding for shape

correspondence. The embedding is achieved through

eigen decomposition on a uniﬁed afﬁnity matrix com-

posed of the spatial afﬁnities and the similarity afﬁni-

GRAPP2013-InternationalConferenceonComputerGraphicsTheoryandApplications

Figure 2: Some experimental results by our method tested on shapes from the TOSCA nonrigid world dataset (Bronstein

et al., 2008).

ties, which means that the obtained embedding en-

codes both the spatial arrangement and the feature

similarity. Comparing with other existing spectral

techniques, our embeddings of differentshapes can be

directly used as shape descriptors for correspondence,

without necessity alignment between them.

In this paper, we only consider isometric shapes

matching in this paper because the construction of

afﬁnity matrices are based on isometric invariant geo-

metric properties. In the future we plan to explore the

possibilities for rigid matching and partial matching

of our framework.

REFERENCES

Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers,

J., and Davis, J. (2005). Scape: shape completion and

animation of people. ACM Trans. Graph., 24(3):408–

416.

Belkin, M. and Niyogi, P. (2003). Laplacian eigenmaps

for dimensionality reduction and data representation.

Neural Comput., 15(6):1373–1396.

Bronstein, A., Bronstein, M., and Kimmel, R. (2008).

Numerical geometry of non-rigid shapes. Springer-

Verlag New York Inc.

Elad, A. and Kimmel, R. (2001). Bending invariant repre-

sentations for surfaces. In CVPR (1), pages 168–174.

Gal, R., Shamir, A., and Cohen-Or, D. (2007). Pose-

oblivious shape signature. IEEE Trans. Vis. Comput.

Graph., 13(2):261–271.

Hilaga, M., Shinagawa, Y., Komura, T., and Kunii, T. L.

(2001). Topology matching for fully automatic simi-

larity estimation of 3d shapes. In SIGGRAPH, pages

203–212.

Jain, V. and Zhang, H. (2006). Robust 3d shape correspon-

dence in the spectral domain. In Proc. of Shape Mod-

eling International, pages 118–129.

Jiang, H. and Yu, S. X. (2009). Linear solution to scale and

rotation invariant object matching. In CVPR, pages

2474–2481.

lp solve. http://sourceforge.net/projects/lpsolve.

Mateus, D., Cuzzolin, F., Horaud, R. P., and Boyer, E.

(2007). Articulated shape matching by robust align-

ment of embedded representations. In IEEE Workshop

on 3D Representation for Recognition (3DRR 2007).

IEEE Computer Society Press.

Mateus, D., Horaud, R. P., Knossow, D., Cuzzolin, F., and

Boyer, E. (2008). Articulated shape matching using

laplacian eigenfunctions and unsupervised point reg-

istration. In Proceedings of the IEEE Conference on

Computer Vision and Pattern Recognition.

Ovsjanikov, M., M´erigot, Q., M´emoli, F., and Guibas, L. J.

(2010). One point isometric matching with the heat

kernel. Comput. Graph. Forum, 29(5):1555–1564.

Roweis, S. and Saul, L. (2000). Nonlinear dimensional-

ity reduction by locally linear embedding. Science,

290(5500):2323–2326.

Sahillio˘glu, Y. and Yemez, Y. (2010). 3d shape corre-

spondence by isometry-driven greedy optimization. In

CVPR, pages 453–458.

Sahillio˘glu, Y. and Yemez, Y. (2011). Coarse-to-ﬁne com-

binatorial matching for dense isometric shape cor-

respondence. Comput. Graph. Forum, 30(5):1461–

1470.

Scott, G. L. and Longuet-Higgins, H. C. (1991). an algo-

rithm for associating the features of two images. In

Proc. Biol Sci.

Sun, J., Ovsjanikov, M., and Guibas, L. (2009). A concise

and provably informative multi-scale signature based

on heat diffusion. In Eurographics Symposium on Ge-

ometry Processing (SGP).

Torki, M. and Elgammal, A. M. (2010). One-shot multi-set

non-rigid feature-spatial matching. In CVPR, pages

3058–3065.

Vaxman, A., Ben-Chen, M., and Gotsman, C. (2010). A

multi-resolution approach to heat kernels on discrete

surfaces. ACM Trans. Graph., 29(4).

AUnifiedSpectralEmbeddingforShapeCorrespondence