REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT
BOUNDARIES
Padraig Corcoran and Adam Winstanley
National Centre for Geocomputation, John Hume Building, National University of Ireland Maynooth, Ireland
Keywords: Segmentation, Texture Boundary Response, Gabor Filters.
Abstract: Texture is a spatial property and thus any features used to describe it must be calculated within a
neighbourhood. This process of integrating information over a neighbourhood leads to what we will refer to
as the texture boundary response problem, where an unwanted response is observed at object boundaries.
This response is due to features being extracted from a mixture of textures and/or an intensity edge between
objects. If segmentation is performed using these raw features this will lead to the generation of unwanted
classes along object boundaries. To overcome this, post processing of feature images must be performed to
remove this response before a classification algorithm can be applied. To date this problem has received
little attention with no evaluation of the alternative solutions available in the literature of which we are
aware. In this work we perform an evaluation of known solutions to the boundary response problem and
discover separable median filtering to be the current best choice. An in depth evaluation of the separable
median filtering approach shows that it fails to remove certain parts or types of object boundary response.
To overcome this failing we propose two alternative techniques which involve either post processing of the
separable median filtered result or an alternative filtering technique.
1 INTRODUCTION
Segmentation is probably one of the most important
and fundamental tasks in computer vision. Despite
the vast literature and literally hundreds of
algorithms, the problem of segmentation still
remains unsolved
(Pantofaru and Hebert, 2005). If each
object in a given scene was of uniform intensity then
segmentation would be trivial but this is not the
case, a given scene will not only contain regions of
uniform intensity but also regions of uniform
texture. Attempting segmentation using solely
intensity features ignoring the presence of texture
will lead to over segmentation due to false edges
generated by the intensity variation within texture. A
common approach to reduce the number of false
edges is to smooth the image prior to segmentation
in an attempt to remove these false edges due to
texture while maintaining edges due to object
boundaries (Deng and Liu, 2003). When working
with intensity based images, a large number of
objects will have similar average intensity values,
thus smoothing will not only remove edges due to
texture but also edges due to object boundaries
leading to under segmentation. Therefore when
attempting to derive accurate segmentation of
intensity images it is important to model the texture
within these images and integrate it with intensity
features in an intelligent manner, so boundaries
between objects which have similar average
intensity values but different textures can be
detected (Corcoran and Winstanley, 2006).
In this paper we focus on the task of extracting
useful texture features from a given image. Texture
is a spatial property and therefore any features used
to describe it must be calculated over a
neighbourhood. A points neighbourhood which is
located across an object boundary may contain two
or more different textures and/or a large intensity
edge giving an unwanted response along the
boundary which we call the texture boundary
response. If segmentation is derived using these raw
features false segments will appear along these
objects boundaries. To overcome this failing it is
common to apply some form of post-processing to
the raw feature images removing the unwanted
response before segmentation is attempted.
Although this is a necessary step in any texture
feature extraction process it has received little
attention and thus we believe is poorly understood
363
Corcoran P. and Winstanley A. (2007).
REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT BOUNDARIES.
In Proceedings of the Second International Conference on Computer Vision Theory and Applications - IFP/IA, pages 363-368
Copyright
c
SciTePress
with no evaluation of existing solutions available in
literature, of which we are aware.
Figure1: Schematic of overall system within which each
boundary response removal technique is evaluated.
In an effort to provide a better understanding of
the boundary response problem, the different types
of responses which may occur in texture feature
images are described. We evaluate all know
solutions to remove these with separable median
filtering being the most accurate. An in-depth
evaluation of separable median filtering shows it
fails to remove certain types or parts of object
boundary responses. To overcome this we propose
two new techniques. The first operates as a post
processing technique to the separable median
filtering and the second is a separate filtering
technique. Evaluation is performed by judging the
effectiveness of the boundary response removal
techniques with respect to the improvement in
segmentation achieved once they have been applied.
A schematic of the system within which these
techniques are implemented is shown in Figure 1.
In the second section of this paper we present the
texture boundary response problem in more detail
and evaluate existing solutions to the problem. In the
third section we show the different types of object
boundary responses which may occur and the result
after median filtering has been applied. We detail
alternative solutions to removing these boundary
responses which overcome the failings of previous
solutions. Results of our proposed techniques are
presented in section 4. Finally, in section 5 we draw
conclusions and discuss future work.
2 TEXTURE BOUNDARY
RESPONSE PROBLEM AND
EXISTING SOLUTIONS
A texture boundary response can be either negative
or positive relative to neighbourhood values. An
image taken from the Berkeley segmentation dataset
(Martin, Fowlkes et al., 2001) is shown in Figure 2.
Features of a low central spatial frequency with high
spatial resolution extracted from this image using a
Gabor filter
(Clausi and Jernigan, 2000) is shown in
Figure 3. This image displays a negative response at
object boundaries relative to neighbourhood values.
Figure 2: Example of image taken from the Berkeley
segmentation dataset.
This negative response is due to the fact that the
Gabor filter in this case is designed to respond to
low spatial frequency. The image contains a number
of nearly uniform intensity areas and thus this
specific Gabor filter will respond strongly to such
areas but will not respond to areas of high spatial
frequency such as an object boundary which has a
large intensity edge and therefore high spatial
frequency. The opposite of this effect can also occur
where a positive response at object boundaries
relative to neighbourhood values is displayed. A
number of authors are under the impression that the
texture boundary response is always a response that
is greater then its neighbouring values (Kruizinga
and Petkov, 1999; Grigorescu, Petkov et al., 2002;
Jobanputra and Clausi, 2006) but this is not the case.
Within the literature there exist a number of
solutions that attempt to tackle the object boundary
response problem. We will now review and evaluate
each of these in turn. The first technique we discuss
was initially employed in (Shao and Forstner, 1994)
Ima
g
e
Gabor Filter Ban
k
Locally Adaptive Robust
Anisotropic Diffusion
Separable Median
Filtering
Second Boundary
Response Removal
Technique
Gaussian Smoothing
Calculation of Single
Gradient Magnitude
Watershed Transfor
m
Segmentation
VISAPP 2007 - International Conference on Computer Vision Theory and Applications
364
and later used in (Martin, Fowlkes et al., 2004). This
solution is not applied to the actual feature space.
First the gradient magnitude of a given feature
image is calculated which gives a double peak effect
at all boundary locations where the texture feature
extraction responds positively or negatively with
respect to its neighbourhood. This gradient
magnitude image is then smoothed with a large
enough Gaussian kernel converting the two peaks
into a single peak. Since this technique does not try
to eliminate the response to object boundaries both
intensity and texture boundaries will be detected.
This would be an undesirable property if the goal of
an algorithm is to detect only texture boundaries
with the aim of later integrating with a model which
detects only intensity boundaries.
Figure 3: Features extracted from Figure 2 exhibit the
boundary response problem
.
A recent paper by Jobanputra (Jobanputra and
Clausi, 2006) tackles the boundary response problem
by choosing a set of texture features which give a
smoothed step response at object boundaries for a
given dataset. If segmentation is then run at a high
enough scale the boundary response values will be
assigned to classes either side of the boundary. This
approach is not data and model independent and it is
difficult to prove that a given feature extraction
algorithm will always give a smoothed step edge at
object boundaries for a given data type. Also since
we may only choose features which give a smoothed
step edge this limits the texture features which may
be used therefore reducing class separability
Another approach to tackle the boundary
response problem is to perform separable 2-D
median filtering of the feature images. Median
filtering is a smoothing technique which can
preserve discontinuities in a step function (Lim,
1989). It is robust to noise or outliers having a size
less then half the size of the median filter used. Thus
any median filter used to remove object boundary
responses must be at least twice the width of any
object boundary response if it is to be removed.
From the above discussion, the separable median
filtering approach of (O'Callaghan and Bull 2005)
represents the current best solution to the boundary
response problem. It is data and model independent,
can remove boundary responses that are either
negative or positive in relation to neighbourhood
values and also removes responses which are due to
a pure intensity edge. In the following section we
will perform a detailed evaluation of the different
types of boundary responses that may occur and the
results after this separable median filtering approach
has been applied in an attempt remove them.
3 TYPES OF OBJECT
BOUNDARY RESPONSES AND
MEDIAN FILTERING
When a window extracting texture features moves
across the boundary between two objects one of a
number of responses may occur. The first is a
response which is similar to a smoothed step edge as
shown in Figure 4(a). The result after applying a
median filter with greater extent then twice the
width of the boundary response is shown in Figure
4(b). Median filtering fails to remove such a
boundary response.
(a) (b)
Figure 4: A smoothed step like boundary response is
shown in (a) and the result post median filtering in (b).
A second type of boundary response which may
occur is a response which is positive with respect to
neighbouring values. An example such a response is
shown in Figure 5(a) and the result after applying a
median filter with greater extent than twice the
width of the boundary response is shown in Figure
5(b). Although median filtering removes the part of
the boundary response which is positive with respect
to all neighbouring values, it fails to remove the
section of the response which resembles a smoothed
step edge.
Other forms of boundary response which may
occur include a response which is negative with
respect to neighbouring values. No response to a
boundary between two objects which have similar
REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT BOUNDARIES
365
texture properties and respond equally to the texture
feature extraction algorithm. This is represented in
two dimensions by a straight horizontal line. A
positive or negative response relative to similar
neighbouring values on both sides, this is
represented in two dimensions by a straight
horizontal line containing a region of relative
positive or negative values. A pure intensity
boundary usually results in this form of boundary
response.
In all the above forms of texture boundary
responses, separable median filtering will remove
the section of the response which is positive or
negative with respect to neighbouring values on both
sides. It will fail to remove a section of the response
if it contains values which are between the two
levels on either side as shown in Figures 4 and 5.
This property of median filtering presents the
problem of how to perform segmentation using these
features without the generation of unwanted
segments along object boundaries, given that
separable median filtering will not remove the entire
boundary response. To achieve this we propose two
solutions, the first involves post-processing of the
median filtered images, and the second involves a
separate filtering technique. We will discuss each of
these in turn now.
(a) (b)
Figure 5: A boundary response which is positive with
respect to neighbouring values is shown in (a) and the
result post median filtering in (b).
The first approach we propose is to perform
segmentation at a greater spatial scale then the
extent of the sections of boundary response which
remain post separable median filtering. The
boundary responses remaining after separable
median filtering will in general be significantly
smaller then the scale of the window used in texture
feature extraction. Also the section of the boundary
response remaining will already resemble a step
edge at a higher scale due to the fact that it will
contain continuously increasing or decreasing
values. In fact it could be described as a smooth step
edge containing some noise. These two facts permit
the use of smoothing with a small Gaussian kernel
relative to the scale used in feature extraction. The
effect is to produce features represented at a spatial
scale where unwanted segments along boundaries
will not appear in the segmentation result. One
drawback of this method is that Gaussian smoothing
will always introduce a loss in boundary
localization. Figure 6 (a) shows the result of this
technique applied to Figure 3. An alternative
approach would be to perform the segmentation
algorithm at a higher scale, but this would lead to
under-segmentation if all boundaries did have
similar absolute differences.
(a) (b)
Figure 6: The results of both texture boundary response
removal techniques.
The second approach we propose to tackle the
boundary response problem involves the processing
of the feature image with a new filtering technique.
This technique takes as input two parameters; t the
threshold size which is the maximum size of an
object for it to be considered an object boundary
response and
θ
the orientation of the texture feature
extraction algorithm. We first detail how this
method is implemented in 1-D and then extend it to
2-D. The 1-D method is implemented in two steps:
1) A 1-dimensional context window of length t
is aligned around a given point of interest
where the window contains that point and
minimizes the sum of absolute difference
between that point and the two boundary
points of the window. This step aligns the
window for a given point with the
neighbourhood to which it is most similar.
2) Then the point of interest is assigned the
boundary value of this window from which it
is most dissimilar.
These two steps are performed on every point in the
dataset. All boundary responses will be replaced
with step edges where the boundary responses cross
the midway point between the two uniform regions
on either side. If the two uniform regions either side
of a boundary response have values of 0 and 1, the
VISAPP 2007 - International Conference on Computer Vision Theory and Applications
366
step edge returned by the algorithm will be located
where the boundary response crosses the value of
0.5. We are working on the assumption that this is
the optimal point where the step edge should occur.
An illustration of this process applied to a data point
in a one-dimensional dataset is shown in Figures 7.
The result of the algorithm applied to Figure 4(a)
and Figure 5(a) is shown in Figure 8(a) and (b)
respectively.
Figure 7: Proposed texture boundary response removal
technique is performed to a point of interest which is a
member of a boundary response.
To extend this technique to two dimensions we first
apply the 1-D method in a direction parallel to the
direction of the feature extraction and then again in a
direction orthogonal to this. An Example of this
method applied to the Gabor feature image in Figure
3 is shown in Figure 6(b). We can see that this
method removes the boundary responses while
maintaining boundary localization and only suffers a
small drop in image detail.
(a) (b)
Figure 8: Results of the second proposed algorithm for
removing boundary responses applied to Figure 4(a) and
Figure 5(a) shown in (a) and (b) respectively
.
Following feature extraction all feature images are
smoothed by a non-linear locally adaptive diffusion
process. The techniques is similar to that used by
Black (Black and Sapiro 1999), except instead of
using the local median absolute deviation of grey
values we use the local median value of gradient
magnitude values. Also we calculate the edges at the
same scale as the feature extraction algorithm not at
the pixel level as done by Black.
4 TEXTURE BOUNDARY
RESPONSE REMOVAL AND
EVALUATION
To perform evaluation we judge the effectiveness of
our boundary response removal techniques with
respect to the improvement in segmentation
achieved once the boundary responses have been
removed. To perform segmentation we use the
marker controlled watershed transform (Soille,
2002). To prevent over-segmentation the gradient
magnitude image is first filtered using a marker
function, in this case the H-minima transform, to
remove all irrelevant minima (Soille, 2002). Figure 9
shows an example of segmentation achieved using
this algorithm.
Evaluation is performed using data from the
Berkeley segmentation dataset (Martin, Fowlkes et
al., 2001). For each of the images in this dataset 5 to
10 ground truths from different individuals are
available. For quantitative comparison of a single
segmentation result to a set of corresponding ground
truths the Normalized Probabilistic Rand (NPR)
index is used (Unnikrishnan, Pantofaru et al., 2005).
This index can be used to measure the relative
accuracy for various algorithms at producing a
useful segmentation for a given image. The greater
the index score, the greater the performance.
Figure 9: Segmentation performed by applying the
watershed transform.
For evaluation 200 images from the Berkeley
dataset are used and this is split into 100 training and
100 test images. Using the training set, the scale of
segmentation is optimized by varying the h-minima
value. The separable Median filtering of
(O'Callaghan and Bull, 2005) followed by
smoothing approach slightly outperforms the second
technique on the training dataset.
Using both boundary response removal
techniques optimized on the training dataset we
evaluate on the test dataset. On the 100 images in the
test dataset the separable median filtering followed
by smoothing technique achieved an average NPR
index value of 0.48, while the second boundary
Replacement
value
Point of
interest
t
Most dissimilar
window
boundary value
Context
window
aligned
REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT BOUNDARIES
367
response removal technique received an average
score of 0.44. The slight decrease in performance for
the second technique relative to the first is probably
due to the fact that the second technique is not as
robust to noise as the first.
5 CONCLUSIONS
The aim of this paper was to provide researchers in
the area of segmentation with a better understanding
of the texture boundary response problem. Prior to
this we could not find any work stating the different
forms of boundary responses which may occur and
how best to remove them. An evaluation of all
current solutions to removing these texture boundary
responses show separable median filtering to be the
current best solution. We analyzed the result of
applying separable median filtering to all possible
boundary responses and showed that it does not
remove all or parts of certain responses.
Two alternative techniques which overcome this
failing were proposed and evaluated. The first
technique is robust to noise but suffers from a loss in
boundary localization. The second technique gives
the optimal solution in a noise free environment but
is not so robust to noise. Using quantitative
evaluation the first approach of extracting edges at a
greater scale then the scale of the boundary
responses remaining after separable median filtering
was shown to perform best. This result does not
mean the second technique is redundant. If a feature
extraction algorithm which produces noise free
images could be found this technique could be used
to produce the optimal solution. This is based on the
assumption that all boundary responses should be
replaced with step edges where the boundary
response crosses the midway point between the two
uniform regions on either side. Future work will
attempt to evaluate whether this is the case.
If useful segmentation is to be produced, both
texture and intensity features must be extracted and
integrated in an intelligent manner. Future work will
also focus on the extraction of useful intensity
features and how best to integrate them with the
texture features discussed in this paper.
REFERENCES
Black, M. and G. Sapiro (1999). Edges as outliers:
anisotropic smoothing using local image statistics.
International conference on scale-space theories in
computer vision, Corfu, Greece. 259- 270.
Clausi, D. A. and M. E. Jernigan (2000). Designing Gabor
filters for optimal texture separability. Pattern
Recognition 33(11): 1835-1849.
Corcoran, P. and A. Winstanley (2006). Using Texture
to Tackle the Problem of Scale in Land- Cover
Classification. International conference on object
based image analysis, Salzburg.
Grigorescu, S. E., N. Petkov, et al. (2002). Comparison of
texture features based on Gabor filters. Image
Processing, IEEE Transactions on 11(10): 1160-1167.
Jobanputra, R. and D. A. Clausi (2006). Preserving
boundaries for image texture segmentation using grey
level co-occurring probabilities. Pattern Recognition
39(2): 234-245.
Kruizinga, P. and N. Petkov (1999). Nonlinear operator
for oriented texture. Image Processing, IEEE
Transactions on 8(10): 1395-1407.
Lim, J. S. (1989). Two-dimensional signal and image
processing, Prentice Hall.
Martin, D., C. Fowlkes, et al. (2001). A database of human
segmented natural images and its applications to
evaluating segmentation algorithms and measuring
ecological statistics. International conference on
computer vision, Vancouver. 416-423.
Martin, D. R., C. C. Fowlkes, et al. (2004). Learning to
detect natural image boundaries using local
brightness, color, and texture cues. IEEE Transactions
on Pattern Analysis and Machine Intelligence 26(5):
530-549.
O'Callaghan, R. J. and D. R. Bull (2005). Combined
morphological-spectral unsupervised image
segmentation. IEEE Transactions on Image Processing
14(1): 49-62.
Pantofaru, C. and M. Hebert (2005). A comparison of
image segmentation algorithms. Robotics institute,
Carnegie Mellon university, Pittsburgh, PA.
Technical Report No. CMU-RI-TR-05-40.
Shao, J. and W. Forstner (1994). Gabor wavelets for
texture edge extraction. ISPRS commission III
symposium on spatial information from digital
photogrammetry and computer vision, Munich,
Germany. 745-752.
Soille, P. (2002). Morphological image analysis:
principles and applications, Springer-Verlag Berlin
and Heidelberg GmbH & Co. K.
Unnikrishnan, R., C. Pantofaru, et al. (2005). A measure
for objective evaluation of image segmentation
algorithms. IEEE conference on computer vision and
pattern recognition, workshop on empirical
evaluation methods in c omputer vision, San
Diego.
VISAPP 2007 - International Conference on Computer Vision Theory and Applications
368