![](bg4.png)
the pair of markers. Besides to provide more informa-
tion to the motion estimation (both markers and the
area between them), the binding of markers heuristics
helps the motion of the pair of markers to follow the
motion of the border that crosses the region between
them in the previous frame.
3.2 Spatio-temporal Gradient
Improvement
The watershed from markers applied to in Step 4
of the framework described in the beginning of this
Section uses the markers provided by the binding of
markers heuristics. Since the first proposal of the wa-
tershed from propagated markers, the watershed oper-
ator was always applied directly to the gradient com-
puted from the frame to be segmented at instant i.
Let s be the input sequence and s(i) the frame to be
segmented. If s(i) ∈ Fun[E, K] (i.e., a grayscale im-
age), the morphological gradient was computed by
the classical way. If s(i) ∈ Fun[E, C] (i.e., a color
image), some metrics were applied to compute the
color gradient (F. C. Flores, A. M. Polid´orio and R.
A. Lotufo, 2006; F. C. Flores, A. M. Polid´orio and
R. A. Lotufo, 2004). Anyway, the gradient compu-
tation was an intra-frame process, computed by using
just the information contained in the frame of interest.
It was not exploited any temporal information in the
gradient computation.
The spatio-temporal gradient improvement con-
sists in a 3-D morphological processing of the input
sequence. Instead to compute the intra-frame gradient
in Step 4 each time the watershed is applied to, the
spatio-temporal gradient is computed at once, at the
beginning of the watershed from propagated mark-
ers framework, before Step 1. Let s be an image se-
quence. The spatio-temporal gradient tg ∈ Fun[F, K]
is given by,
tg =
∇
c
B
3D
(s), if s is a color sequence,
∇
B
3D
(s), otherwise,
where B
3D
is a 3-D s.e. such as the examples in Sec-
tion 2.
As stated above, the spatio-temporal gradient tg
is a function from F to K, i.e., it is a grayscale se-
quence. The watershed transform could be applied to
tg but, as discussed in the Introduction, it is an expen-
sive process, propagates interframe error and requires
an inspection in the whole segmentation sequence ev-
ery time it is applied to. Even when the sequence was
sampled, in previous experiments, with two or three
consecutive frames, the interframe error worsened the
segmentation result.
The solution is to sample just the i-th frame of the
spatio-temporal gradient when it is desired to segment
the i-th frame of the input sequence. The sample is
an image in Fun[E, K] but contains spatio-temporal
information obtained in the spatio-temporal gradient
computation. Since it is just a single frame, the inter-
frame error propagation does not occur.
Let tg ∈ Fun[F, K] be a spatio-temporal gradient.
The gradient g ∈ Fun[E, K] where the watershed from
markers will be applied to in Step 4 of the watershed
from propagated markers is given by
g = ς
i
(tg),
where i is the index to the i-th frame to be segmented.
4 EXPERIMENTAL RESULTS
Two experiments were done in order to illustrate the
gains of productivity and segmentation quality pro-
vided by the application of the spatio-temporal gradi-
ent to the binding of markers framework. First exper-
iment shows the gains achievedto the first 150 frames
of Foreman sequence. The second experiment shows
the experimental results to the first 150 frames of Car-
phone sequence.
Both experiments apply an benchmark proposed
in (F. C. Flores and R. A. Lotufo, 2008) to do a quan-
titative assessment of interactive object segmentation
to image sequences. The benchmark evaluates sev-
eral measurements such as quality of segmentation
and time spent to complete the segmentation task. In
that work, several applications of the binding of mark-
ers supported by the Lucas-Kanade (B. Lucas and T.
Kanade, 1981) motion estimator were tested and com-
pared to. The application that presented the best over-
all results was the one with parameters m = w = 10.
These parameters and the cited motion estimator were
used in all experiments done with Foreman and Car-
phone sequences in this paper.
The experiment, for each sequence, consists in to
compare several applications of the binding of mark-
ers supported by the spatio-temporal gradient to the
manual segmentation and to the simple binding of
markers application (the one proposed in (F. C. Flores
and R. A. Lotufo, 2007)). Table 1 shows the quanti-
tative results for both sequences. Four 3-D s.e. were
tested to compute the spatio-temporal gradients: B
5
,
B
6
, B
17
and B
26
(see Section 2). When the s.e. used
to compute the spatio-temporal gradient were B
5
or
B
6
, the chosen connectivity for the watershed opera-
tor was 4 (the cross s.e.). When the spatio-temporal
gradient was computed using B
17
or B
26
, the chosen
connectivity for the watershed was 8 (the box s.e.).
The manual segmentations were used as the ground-
truth segmentations. The application of the spatio-
temporal gradient is denoted in Table 1 by ∇
B
3D
and
WATERSHED FROM PROPAGATED MARKERS IMPROVED BY THE COMBINATION OF SPATIO-TEMPORAL
GRADIENT AND BINDING OF MARKERS HEURISTICS
167