MOTION CAPTURE OF AN ANIMATED SURFACE

VIA SENSORS’ RIBBONS

Surface Reconstruction via Tangential Measurements

Nathalie Sprynski

CEA-LETI, MINATEC Campus, Grenoble, France

Bernard Lacolle, Luc Biard

Laboratoire Jean Kuntzmann, Universit

´

e Joseph Fourier, Grenoble, France

Keywords:

Motion capture, Micro-sensors, Surface reconstruction, Curve reconstruction, Hermite interpolation.

Abstract:

This paper deals with the motion capture of physical surfaces via a curve acquisition device. This device is a

ribbon of sensors, named Ribbon Device, providing tangential measurements, allowing to reconstruct its 3D

shape via an existing geometric method. We focus here on the problem of reconstructing animated surfaces,

from a ﬁnite number of curves running on these surfaces, acquired with the Ribbon Device. This network

of spatial curves is organized according a comb structure allowing to adjust these curves with respect to a

reference curve, and then to develop a global C1 reconstruction method based on the mesh of ribbon curves

together with interpolating transversal curves. Precisely, at each time position the surface is computed from

the previous step by an updating process.

1 INTRODUCTION

We are concerned with the reverse engineering prob-

lem of re–constructing animated physical surfaces

from tangential data. These tangential data are pro-

vided by embedded sensors (micro–accelerometers

and micro–magnetometers) along a curve represented

by a ribbon. This problem is not a dual interpola-

tion or approximation problem (Hoschek, 1983) as

the tangential data are not localized in space. Appro-

priate methods for the reconstruction of planar and

spatial curves from such tangential information have

been developed in (Sprynski et al., 2007) and have

been validated by a real-time demonstrator: a ribbon–

like device, denoted Ribbon Device – see Figure 1

– equipped with 32 micro–sensors. See also (Hoshi

and Shinoda, 2008) for a prototype of sensing device,

analogous to a rectangular grid of linear instrumented

segments, providing thus elementary geometry.

The deep novelty of such capture/reconstruction

approaches is to deal with purely orientation data.

Furthermore, notice that all previous related works

only consider static surfaces. We are thus concerned

in this paper with the motion capture of a surface

in deformation from a network of spatial curves run-

ning on the surface, obtained with the Ribbon Device.

Precisely, by placing the Ribbon Device on a physi-

cal surface at regular intervals, the surface is divided

into a system of patches, which can be then ﬁlled by

interpolating Coons processes (Coons, 1964; Coons,

1974; Farin, 2002). See also (Peters, 1990; Sarraga,

1987; Shirman and S

´

equin, 1987) for construction

processes of smooth surfaces from given boundary

curve data. The shape of the surface is thus essentially

modeled by these spatial curves forming a character-

istic mesh of the surface.

Applications are countless, ranging from medical

applications (e.g. determining shape and curvature of

the spinal column), to aerodynamic applications (e.g.

acquiring the shape and deformation of a wing).

This document is organized as follows. The sen-

sors, the acquisition tool and the process used for sur-

face capture are detailed in Section 2. Section 3 fo-

cuses on the algorithms developed to solve the prob-

lem, which ﬁrst consists in reconstructing the ribbon

curves, and then the interpolating surface. Finally,

Section 4 deals with the validation and the implemen-

tation.

421

Sprynski N., Lacolle B. and Biard L..

MOTION CAPTURE OF AN ANIMATED SURFACE VIA SENSORS’ RIBBONS - Surface Reconstruction via Tangential Measurements.

DOI: 10.5220/0003330604210426

In Proceedings of the 1st International Conference on Pervasive and Embedded Computing and Communication Systems (PECCS-2011), pages

421-426

ISBN: 978-989-8425-48-5

Copyright

c

2011 SCITEPRESS (Science and Technology Publications, Lda.)

2 ACQUISITION STEPS

2.1 The Acquisition Tool

What is targeted here is to introduce new kinds of in-

strumented materials. We think for example about

plastic or textile ribbons or surfaces, which will be

equipped with arrays of sensors in order to gain some

new properties. The alliance between instrumented

materials and mathematical algorithms will allow ma-

terials to be able to access some knowledge about

their own shape, introducing what we could call pro-

prioceptive materials. The approach for building such

materials is presented.

2.1.1 Sensors

First step is to provide angular adequate information

for surface computation. For that purpose, the follow-

ing microsensors arre used.

• Microaccelerometers are able to provide the angle

between the sensor and the vertical (as long as the

sensor is quite static).

• Micromagnetometers are able to provide angular

information with the earth magnetic ﬁeld (when

no magnetic pertubations around them occur).

These sensors have to be combined as a biaxial or a

triaxial way, in order obtain the exact orientation of

the measure point in the space (Fontaine et al., 2003).

Five different measure from sensors are necessary to

have the exact orientation (three accelerometers in a

triaxial organization, and two magnetometers in a bi-

axial way). This study allows the creation of the pro-

totype.

2.1.2 The Ribbon

Our new generation ribbon has been developed with

the considerations above. The ribbon is equipped with

a set of sixteen 3D microaccelerometers, alternating

with a set of sixteen 2D micromagnetometers (AMR

type sensors from Honeywell or similar). They are

mounted on a ﬂexible PCB ribbon. The distance be-

tween the sensors is nearly 25 mm. Such arrangement

of the sensors allows gaining complete tangential in-

formation (not exactly at each sensor location, but for

a set of two adjacent sensors). The sensors are read

via a SPI serial bus, which allows a lot of sensors (see

Figure 1). This ribbon is also easy to use thanks to its

Bluetooth connection to the host computer. Finally, a

software driver has been designed which allows to se-

quentially read all sensor values at different sampling

Figure 1: Ribbon of sensors able to provide 3D tangential

data.

rates. In fact, the current ribbons are essentially proto-

types that have been used for demonstration and val-

idation purpose. Some technological issues are cur-

rently being addressed concerning sensor embedding

in ribbons.

• Connections: we want to connect wires to a pos-

sibly great number of sensors. Solutions already

exist at the die scale, but not at larger scales (up

to meters). Dedicated connection technologies are

currently being studied.

• Reliability: as the ribbon is planned to be ﬂexible,

most existing technologies do not apply. We are

also studying various solutions to get such relia-

bility.

Our ﬁnal goal is to be able to produce ribbons and sur-

faces embedding possible large sensors areas, while

keeping the advantage of low cost sensors.

2.1.3 The Ribbon Curve

The ribbon described above is able to provide its own

shape. More precisely, methods have been developed

to reconstruct curves via data from the ribbon, the

curve being denoted ribbon curve (see Appendix A).

Data are tangential data at sensors’ positions, and dis-

tances between sensors along the ribbon. Let us no-

tice that we do not have any information concerning

absolute positions of any points of the ribbon, such

that the ribbon curves reconstructed are unique up to

their starting point.

2.2 Surface Acquisition Process

We have to acquire a surface via a ribbon of sensors,

thus we have to reconstruct a surface from a ﬁnite

number of ribbon curves laying on it. As it does not

exist an intrinsic parameterization for surfaces (con-

trary to the curves with the arc-length parameter), we

will keep the linear organization of sensors, and the

ribbons are then a natural way to acquire surfaces.

The question is thus as follows : how can we orga-

nize curves on a surface in order to know it ? Two

PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems

422

kinds of organization appear. The ﬁrst one is a mesh

of ribbons, where sensors are on the intersections.

The surface is known by two families of curves in

two complementary directions, so that we have a ten-

sorial topology of the surface. This problem is well

posed but this acquisition system is not really easy to

use (how can we put it on non-developable surfaces?

How can we put ribbons to cross exactly on sensors’

positions?). So, in a more general case, we consider a

second acquisition system, which consists in one fam-

ily of ribbons in the same direction.

More precisely, we ﬁx ribbons in the same direc-

tion on a mobile surface. Figure 2 shows an exam-

ple of acquisition process with four parallel ribbons

(in red), with sensors represented in black squares.

In that case, ribbon curves are reconstructed inde-

pendently, thus we have to ﬁx their relative position

with each other: we use an additional ribbon in the

transversal direction, denoted reference ribbon link-

ing all starting points A

k

: thus we have a comb struc-

ture of ribbons to acquire a surface.

Figure 2: Surface acquisition process with 4 ribbons.

These ribbons give at each sensors’ point tangen-

tial information (two vectors at sensors’ position: t

τ

k

tangential vector of the ribbon curve, b

τ

k

binormal

vector of the ribbon curve in the tangent plane (ribbon

seen as a surface, equivalent to the surface to acquire

at these points) (see Figure 3).

Figure 3: Tangential data given by sensors : unit tangent

vector t

τ

k

and unit binormal vector b

τ

k

.

When the surface is moving, ribbons are following

the surface deformations. The surface is thus known

via the moving curves laying on it. The ﬁgure 5-left

illustrates the process.

The physical surface is then described with the

ﬂow of the tangential constraints (given by sensors

on ribbons), with length constraints along the rib-

bons. The sensors are organized in a comb structure,

with tangential data in both directions, but length con-

straints in only one direction (the ribbon curves direc-

tion).

3 SURFACE RECONSTRUCTION

As the surface acquisition process – with a comb

structure – induces a tensorial structure on the physi-

cal surface, we consider the following reconstruction

strategy.

At each time position :

1- we reconstruct the 3D ribbon curves from sensors

data and length constraints,

2- these ribbon curves are adjusted according the

comb structure,

3- from which we deduce the transversal 3D curves

from sensors’ data, but without any length infor-

mation,

4- ﬁnally, the surface is ﬁlled by a standard cubic

Coons process.

The main strategy is thus to follow the ani-

mated/deformed ribbons.

Step 1 has been validated in case of static recon-

struction but could be time expensive. So, for real

time reconstruction, the 3D ribbon curves at time po-

sition τ+∆τ are deduced from the 3D ribbon curves at

time position τ by an iterative/minimisation process.

Precisely, Step 1 splits into two phases.

1a- Initialization – At time position τ = 0, the 3D

initial ribbon curves are reconstructed from the

method described in Appendix A.

1b- Iterative step – The 3D ribbon curves at time po-

sition τ + ∆τ are deduced by an updating process

from the tangential data of sensors at time τ + ∆τ

and the sensors’ position at time τ.

Notice that each ribbon curve is reconstructed up

to an arbitrary starting point, in Steps 1a and 1b.

These ribbon curves are then adjusted in Step 2 by

using the reference ribbon curve. Finally, the whole

reconstruction process will produce a C

1

surface.

3.1 Ribbon Curves Reconstruction

As Step-1a is detailed in Appendix A, we now focus

on the iterative Step-1b. Assume we have N + 1 rib-

MOTION CAPTURE OF AN ANIMATED SURFACE VIA SENSORS' RIBBONS - Surface Reconstruction via

Tangential Measurements

423

bons (one of them being the reference ribbon curve),

each of these ribbons being equipped with n + 1 sen-

sors numbered from 0 to n.

Considering any of these ribbons, each segment

ribbon curve r

τ+∆τ

k

(s) between sensor k and sensor

k + 1, at time position τ + ∆τ, is modeled, as a cubic

Hermite curve (see Appendix B)

r

τ+∆τ

k

(t) = H[p

τ+∆τ

k

, p

τ+∆τ

k+1

, t

τ+∆τ

k

, t

τ+∆τ

k+1

;s

k

, s

k+1

](t),

(1)

where :

⋄ points p

τ+∆τ

k

, p

τ+∆τ

k+1

are the unknown positions of

sensors k and k + 1 at time position τ + ∆τ,

⋄ vectors t

τ+∆τ

k

, t

τ+∆τ

k+1

are the (updated) tangential

information along the curve provided by sensors

k and k + 1 at time τ + ∆τ,

⋄ s

k

, s

k+1

are the arc length positions of sensors k

and k + 1 along the ribbon.

Then, the length constraints between sensors along

the ribbon provide additional relations

∫

s

k+1

s

k

∥

d

dt

r

τ+∆τ

k

(t)∥dt = s

k+1

− s

k

, (2)

for k = 0, ..., n − 1 , at each time τ + ∆τ. Expand-

ing these integrals yield non linear constraints. Fur-

thermore, as equations (2) do not have a unique so-

lution, we consider the supplementary minimization

constraints for k = 0, ..., n − 1

min

∫

s

k+1

s

k

∥

d

2

dt

2

r

τ+∆τ

k

(t)∥

2

dt . (3)

Finally, as constraints (2) and (3) allow to de-

termine uniquely one of the two unknown position

points of each segment ribbon curve, the method for

the ribbon curve reconstruction proceeds iteratively as

follows for each curve.

– Choose point p

τ

0

as starting point p

τ+∆τ

0

of the rib-

bon curve.

– For each segment ribbon curve r

τ+∆τ

k

(s), k =

0, ..., n − 1, compute the ending point p

τ+∆τ

k+1

from

relations (1) with the updated vectors t

τ+∆τ

k

, t

τ+∆τ

k+1

,

and the (previously computed) starting point

p

τ+∆τ

k

, from constraints (2) and (3). This mini-

mization step is initialized with the previous point

p

τ

k+1

.

The Hermite deﬁnition (1) insures that the recon-

structed ribbon curves will be C

1

. Moreover, the en-

ergy constraint (3) on each segment ribbon curve in-

sures to get a smooth ribbon curve among the family

of curve solutions, i.e., C

1

cubic splines. Notice that

these C

1

splines are not the classical ones as the initial

data are tangent directions instead of 3D points.

3.2 Comb Structure Updating

As each ribbon curve is reconstructed up to an arbi-

trary starting point in the previous steps, we are now

faced to adjust these curves according the reference

ribbon curve.

This step is based on the two following points.

- The reference ribbon curve is reconstructed up to

(the reference) point A

τ

1

= A

1

at each time position

τ, see Figure 2. So that the whole surface will be

reconstructed with respect to that point.

- Then, each “orthogonal” ribbon curve is trans-

lated in order its starting point match with the cor-

responding point A

τ

k

.

3.3 Transversal Curves

At this point, we know at each time position the sen-

sor’s 3D position on each ribbon curve together with

its associated transversal tangential information: the

binormal vector, see Figure 3.

Figure 4: Future physical testing framework device.

Denoting more precisely by p

τ

k, j

the 3D position

of sensor k on the ribbon curve j and by b

τ

k, j

the as-

sociated unit binormal vector, at each time position τ,

we consider the cubic Hermite curve (see Appendix

B) joining sensors k on ribbon curves j and j + 1

x

τ

k, j

(t) (4)

=H[p

τ

k, j

, p

τ

k, j+1

, λ

τ

k, j

b

τ

k, j

, λ

τ

k, j+1

b

τ

k, j+1

;t

j

,t

j+1

](t),

with t

j

= j −1 and where λ

τ

k, j

are positive coefﬁcients

associated with sensor k of the ribbon curve j at time

position τ.

Then, considering the C

1

spline curve x

τ

k

(t), com-

posed of segment curves x

τ

k, j

(t), the following mini-

mization energy

min

∫

N−1

0

∥

d

2

dt

2

x

τ

k

(t)∥

2

dt , (5)

leads to determine unique values for coefﬁcients λ

τ

k, j

,

producing smooth C

1

Hermite interpolating transver-

sal curves.

PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems

424

Figure 5: Left: animated analytical surface – Right: the reconstructed animated surface.

3.4 Surface Filling

At this point, the physical surface is recon-

structed/modeled by a network of two families of “or-

thogonal” C

1

spline curves meeting at sensors’ posi-

tion, delimiting thus a set of n(N −1) curvilinear rect-

angles.

Each of these curvilinear rectangles is then ﬁlled

by a partially bi-cubically blended Coons process

(Coons, 1974), producing a G

1

global surface.

Notice that this whole reconstruction process al-

lows to faster refresh the display of the reconstructed

surface, by only considering the previous network of

“orthogonal” curves as a mesh approximating the sur-

face.

4 CONTROL AND VALIDATION

The validation of such methods requires to compare

the physical deformed surfaces and the reconstructed

shapes at each time position, and clearly, a visual con-

trol is not sufﬁcient. We describe here the experimen-

tal device under development for this purpose.

4.1 Experimental Device

We are currently developing a physical testing frame-

work surface (see Figure 4), animated and controlled

by a mechanical device. The surface motion is ac-

quired by an optical system, providing an external re-

liable control.

By placing ribbons of sensors on this testing

framework surface, according the process described

in Section 2.2, we will acquire a network of animated

spatial curves on this surface, allowing to compare the

reconstructed surface by our process with the “optical

surface” reconstructed from the external acquisition

optical system.

Furthermore, the acquisition process of moving

surfaces is intricate and requires a precise methodol-

ogy. While the surface is moving, the physical ribbon

will not keep an intrinsic position on the surface and

will slip on the surface. Precisely, it is proved that

these ribbons actually follow geodesic curves on the

physical surface. Thus, when the surface is deformed,

the ribbons slip in order to remain a geodesic on the

surface. This is not a blocking point for our method

as we need only distance information of the starting

points of the curves, the ribbons remaining ﬁxed at

the origin.

4.2 Computed Examples

The animated analytical surface described in Fig-

ure 5-left is reconstructed using our process, see Fig-

ure 5-right.

An error is estimated between the animated

analytical model of the surface and the re-

constructed animated surface at some time po-

sition. Precisely, at each time position τ,

the error is computed at sensors’ position as

E(τ) =

100

(n + 1)N L

rib

N

∑

j=1

n

∑

k=0

∥

˜

p

τ

k, j

− p

τ

k, j

∥

where

˜

p

τ

k, j

and p

τ

k, j

are respectively the sensor’s posi-

tion on the physical surface and on the reconstructed

surface, and where L

rib

is the ribbon’s length.

Figure 5 exhibits an example with a mean error

equal to 1.68%, comprised in the interval [1.18, 2.57],

on a sample of 120 time positions.

5 CONCLUSIONS

A strategy for the acquisition of animated surfaces via

a ribbon of sensors has been developed in this paper.

The method is based on a comb structure of the recon-

structed ribbon curves according a reference curve.

MOTION CAPTURE OF AN ANIMATED SURFACE VIA SENSORS' RIBBONS - Surface Reconstruction via

Tangential Measurements

425

This approach allows to adjust the network of the re-

constructed curves running on the surface and to get

a “coherent” mesh interpolating the animated surface

by an updating process from the previous time posi-

tion. The method has been tested on an animated an-

alytical surface.

Then, in order to validate the method in the gen-

eral case of a physical animated surface, a testing

framework surface is under development, so that, per-

formance comparisons with real physical data have

not been realized yet. Anyway, the ﬁrst experiments

have highlighted some practical difﬁculties concern-

ing the implementation of the acquisition process. Es-

sentially, we have to ensure a permanent smooth con-

tact between the ribbon and the animated surface. So-

lutions (i.e., smoother and more ﬂexible ribbons,...)

are on progress.

REFERENCES

Coons, S. (1964). Surfaces for computer aided design.

Technical Report, M.I.T.,1964, Available as AD 663

504 from the National Technical Information service,

Springﬁeld, VA 22161.

Coons, S. (1974). Surface patches and b-spline curves.

Computer Aided Geometric Design, In R. Barnhill

and R. Riesenfeld editors, Academic Press.

Farin, G. (2002). Curves and Surfaces for CAGD - Fifth

Edition. Academic Press.

Fontaine, D., David, D., and Caritu, Y. (2003). Sourceless

human body motion capture. In Proc. Smart Objects

Conference (SOC’03).

Hoschek, J. (1983). Dual b

´

ezier curves and surfaces. Com-

puter Aided Geometric Design, North Holland, pages

147–156.

Hoshi, T. and Shinoda, H. (2008). 3d shape measuring

sheet utilizing gravitational and geomagnetic ﬁelds. In

Proceeding of the SICE Annual Conference 2008. The

University Electro-Communications, Japan.

Nielson, G. (2004). ν-quaternion splines for the smooth

interpolation of orientations. IEEE Transactions on

visualization and computer graphics, 10(2):224–229.

Peters, J. (1990). Local smooth surface interpolation: A

classiﬁcation. Computer Aided Geometric Design,

7:191–195.

Sarraga, R. F. (1987). G1 interpolation of generally unre-

stricted cubic b

´

ezier curves. Computer Aided Geomet-

ric Design, 4:23–39.

Shirman, L. A. and S

´

equin, C. H. (1987). Local surface in-

terpolation with b

´

ezier patches. Computer Aided Ge-

ometric Design, 4:279–295.

Sprynski, N., Lacolle, B., David, D., and Biard, L. (2007).

Curve reconstruction via a ribbon of sensors. In Pro-

ceeding of the 14th IEEE International Conference on

Electronics Circuits and Systems. ICECS, Marrakech,

Maroc.

APPENDIX

A: Initial Reconstructed Curves. A 3D curve

C(s) = (x(s), y(s), z(s)), parameterized with respect to

its arc-length s satisfy |C

′

(s)| ≡ 1, so that the deriva-

tive curve C

′

(s) is a curve lying on the unit sphere.

Initial data are unit tangential vectors at points with

assigned arc length parameters. The methodology is

thus as follows.

• First, we interpolate data using cubic splines on

the sphere, leading to the derivative curve C

′

(s).

• Then, by integration we get a solution for C(s).

Cubic splines on the unit sphere (see (Nielson, 2004))

are an extension of the usual B-splines in the euclid-

ian space. The main differences are the following.

⋄ The evaluation of the control polygon of cubic

splines on the spherical space requires to solve a non

linear system through an iterative algorithm.

⋄ The usual De Casteljau algorithm, based on linear

interpolations, has to be replaced by the spherical in-

terpolation

Sler p(a, b, t) =

sin((1 −t)θ)a + sin(tθ)b

sin(θ)

,

where a and b are two unit vectors, θ the angle be-

tween vectors a and b, and t ∈ [0, 1].

It is proved in (Sprynski et al., 2007) that this con-

struction is invariant under rotations and scaling, and

that these spherical splines minimize a combination

of the curvature κ

1

, the torsion κ

2

, and the variations

of the curvature, precisely

min

∫

(κ

′2

1

+ κ

2

1

(κ

′2

1

+ κ

2

2

)),

which gives physical sense to the reconstruction.

B: Cubic Hermite Interpolation. Given spatial

points p

0

and p

1

associated with tangent vectors t

0

and t

1

, together with two parameters α

0

and α

1

(α

0

<

α

1

), there exists a unique cubic spatial parametric

curve r(t) such that

r(α

0

) = p

0

, r(α

1

) = p

1

, r

′

(α

0

) = t

0

, r

′

(α

1

) = t

1

.

Precisely, r(t) is deﬁned by

r(t) = H

0

(

ˆ

t)p

0

+ H

1

(

ˆ

t)p

1

+ (α

1

− α

0

)H

2

(

ˆ

t)t

0

+ (α

1

− α

0

)H

3

(

ˆ

t)t

1

,

with

ˆ

t =

t−α

0

α

1

−α

0

and where functions φ

j

are the cu-

bic Hermite polynomials (Farin, 2002) on the interval

[0, 1], and r(t) and is denoted shortly by

r(t) = H[p

0

, p

1

, t

0

, t

1

;α

0

, α

1

](t).

PECCS 2011 - International Conference on Pervasive and Embedded Computing and Communication Systems

426