REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT

BOUNDARIES

Padraig Corcoran and Adam Winstanley

National Centre for Geocomputation, John Hume Building, National University of Ireland Maynooth, Ireland

Keywords: Segmentation, Texture Boundary Response, Gabor Filters.

Abstract: Texture is a spatial property and thus any features used to describe it must be calculated within a

neighbourhood. This process of integrating information over a neighbourhood leads to what we will refer to

as the texture boundary response problem, where an unwanted response is observed at object boundaries.

This response is due to features being extracted from a mixture of textures and/or an intensity edge between

objects. If segmentation is performed using these raw features this will lead to the generation of unwanted

classes along object boundaries. To overcome this, post processing of feature images must be performed to

remove this response before a classification algorithm can be applied. To date this problem has received

little attention with no evaluation of the alternative solutions available in the literature of which we are

aware. In this work we perform an evaluation of known solutions to the boundary response problem and

discover separable median filtering to be the current best choice. An in depth evaluation of the separable

median filtering approach shows that it fails to remove certain parts or types of object boundary response.

To overcome this failing we propose two alternative techniques which involve either post processing of the

separable median filtered result or an alternative filtering technique.

1 INTRODUCTION

Segmentation is probably one of the most important

and fundamental tasks in computer vision. Despite

the vast literature and literally hundreds of

algorithms, the problem of segmentation still

remains unsolved

(Pantofaru and Hebert, 2005). If each

object in a given scene was of uniform intensity then

segmentation would be trivial but this is not the

case, a given scene will not only contain regions of

uniform intensity but also regions of uniform

texture. Attempting segmentation using solely

intensity features ignoring the presence of texture

will lead to over segmentation due to false edges

generated by the intensity variation within texture. A

common approach to reduce the number of false

edges is to smooth the image prior to segmentation

in an attempt to remove these false edges due to

texture while maintaining edges due to object

boundaries (Deng and Liu, 2003). When working

with intensity based images, a large number of

objects will have similar average intensity values,

thus smoothing will not only remove edges due to

texture but also edges due to object boundaries

leading to under segmentation. Therefore when

attempting to derive accurate segmentation of

intensity images it is important to model the texture

within these images and integrate it with intensity

features in an intelligent manner, so boundaries

between objects which have similar average

intensity values but different textures can be

detected (Corcoran and Winstanley, 2006).

In this paper we focus on the task of extracting

useful texture features from a given image. Texture

is a spatial property and therefore any features used

to describe it must be calculated over a

neighbourhood. A points neighbourhood which is

located across an object boundary may contain two

or more different textures and/or a large intensity

edge giving an unwanted response along the

boundary which we call the texture boundary

response. If segmentation is derived using these raw

features false segments will appear along these

objects boundaries. To overcome this failing it is

common to apply some form of post-processing to

the raw feature images removing the unwanted

response before segmentation is attempted.

Although this is a necessary step in any texture

feature extraction process it has received little

attention and thus we believe is poorly understood

363

Corcoran P. and Winstanley A. (2007).

REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT BOUNDARIES.

In Proceedings of the Second International Conference on Computer Vision Theory and Applications - IFP/IA, pages 363-368

 SciTePress

with no evaluation of existing solutions available in

literature, of which we are aware.

Figure1: Schematic of overall system within which each

boundary response removal technique is evaluated.

In an effort to provide a better understanding of

the boundary response problem, the different types

of responses which may occur in texture feature

images are described. We evaluate all know

solutions to remove these with separable median

filtering being the most accurate. An in-depth

evaluation of separable median filtering shows it

fails to remove certain types or parts of object

boundary responses. To overcome this we propose

two new techniques. The first operates as a post

processing technique to the separable median

filtering and the second is a separate filtering

technique. Evaluation is performed by judging the

effectiveness of the boundary response removal

techniques with respect to the improvement in

segmentation achieved once they have been applied.

A schematic of the system within which these

techniques are implemented is shown in Figure 1.

In the second section of this paper we present the

texture boundary response problem in more detail

and evaluate existing solutions to the problem. In the

third section we show the different types of object

boundary responses which may occur and the result

after median filtering has been applied. We detail

alternative solutions to removing these boundary

responses which overcome the failings of previous

solutions. Results of our proposed techniques are

presented in section 4. Finally, in section 5 we draw

conclusions and discuss future work.

2 TEXTURE BOUNDARY

RESPONSE PROBLEM AND

EXISTING SOLUTIONS

A texture boundary response can be either negative

or positive relative to neighbourhood values. An

image taken from the Berkeley segmentation dataset

(Martin, Fowlkes et al., 2001) is shown in Figure 2.

Features of a low central spatial frequency with high

spatial resolution extracted from this image using a

Gabor filter

(Clausi and Jernigan, 2000) is shown in

Figure 3. This image displays a negative response at

object boundaries relative to neighbourhood values.

Figure 2: Example of image taken from the Berkeley

segmentation dataset.

This negative response is due to the fact that the

Gabor filter in this case is designed to respond to

low spatial frequency. The image contains a number

of nearly uniform intensity areas and thus this

specific Gabor filter will respond strongly to such

areas but will not respond to areas of high spatial

frequency such as an object boundary which has a

large intensity edge and therefore high spatial

frequency. The opposite of this effect can also occur

where a positive response at object boundaries

relative to neighbourhood values is displayed. A

number of authors are under the impression that the

texture boundary response is always a response that

is greater then its neighbouring values (Kruizinga

and Petkov, 1999; Grigorescu, Petkov et al., 2002;

Jobanputra and Clausi, 2006) but this is not the case.

Within the literature there exist a number of

solutions that attempt to tackle the object boundary

response problem. We will now review and evaluate

each of these in turn. The first technique we discuss

was initially employed in (Shao and Forstner, 1994)

Ima

Gabor Filter Ban

Locally Adaptive Robust

Anisotropic Diffusion

Separable Median

Filtering

Second Boundary

Response Removal

Technique

Gaussian Smoothing

Calculation of Single

Gradient Magnitude

Watershed Transfor

Segmentation

VISAPP 2007 - International Conference on Computer Vision Theory and Applications

364

and later used in (Martin, Fowlkes et al., 2004). This

solution is not applied to the actual feature space.

First the gradient magnitude of a given feature

image is calculated which gives a double peak effect

at all boundary locations where the texture feature

extraction responds positively or negatively with

respect to its neighbourhood. This gradient

magnitude image is then smoothed with a large

enough Gaussian kernel converting the two peaks

into a single peak. Since this technique does not try

to eliminate the response to object boundaries both

intensity and texture boundaries will be detected.

This would be an undesirable property if the goal of

an algorithm is to detect only texture boundaries

with the aim of later integrating with a model which

detects only intensity boundaries.

Figure 3: Features extracted from Figure 2 exhibit the

boundary response problem

A recent paper by Jobanputra (Jobanputra and

Clausi, 2006) tackles the boundary response problem

by choosing a set of texture features which give a

smoothed step response at object boundaries for a

given dataset. If segmentation is then run at a high

enough scale the boundary response values will be

assigned to classes either side of the boundary. This

approach is not data and model independent and it is

difficult to prove that a given feature extraction

algorithm will always give a smoothed step edge at

object boundaries for a given data type. Also since

we may only choose features which give a smoothed

step edge this limits the texture features which may

be used therefore reducing class separability

Another approach to tackle the boundary

response problem is to perform separable 2-D

median filtering of the feature images. Median

filtering is a smoothing technique which can

preserve discontinuities in a step function (Lim,

1989). It is robust to noise or outliers having a size

less then half the size of the median filter used. Thus

any median filter used to remove object boundary

responses must be at least twice the width of any

object boundary response if it is to be removed.

From the above discussion, the separable median

filtering approach of (O'Callaghan and Bull 2005)

represents the current best solution to the boundary

response problem. It is data and model independent,

can remove boundary responses that are either

negative or positive in relation to neighbourhood

values and also removes responses which are due to

a pure intensity edge. In the following section we

will perform a detailed evaluation of the different

types of boundary responses that may occur and the

results after this separable median filtering approach

has been applied in an attempt remove them.

3 TYPES OF OBJECT

BOUNDARY RESPONSES AND

MEDIAN FILTERING

When a window extracting texture features moves

across the boundary between two objects one of a

number of responses may occur. The first is a

response which is similar to a smoothed step edge as

shown in Figure 4(a). The result after applying a

median filter with greater extent then twice the

width of the boundary response is shown in Figure

4(b). Median filtering fails to remove such a

boundary response.

(a) (b)

Figure 4: A smoothed step like boundary response is

shown in (a) and the result post median filtering in (b).

A second type of boundary response which may

occur is a response which is positive with respect to

neighbouring values. An example such a response is

shown in Figure 5(a) and the result after applying a

median filter with greater extent than twice the

width of the boundary response is shown in Figure

5(b). Although median filtering removes the part of

the boundary response which is positive with respect

to all neighbouring values, it fails to remove the

section of the response which resembles a smoothed

step edge.

Other forms of boundary response which may

occur include a response which is negative with

respect to neighbouring values. No response to a

boundary between two objects which have similar

REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT BOUNDARIES

365

texture properties and respond equally to the texture

feature extraction algorithm. This is represented in

two dimensions by a straight horizontal line. A

positive or negative response relative to similar

neighbouring values on both sides, this is

represented in two dimensions by a straight

horizontal line containing a region of relative

positive or negative values. A pure intensity

boundary usually results in this form of boundary

response.

In all the above forms of texture boundary

responses, separable median filtering will remove

the section of the response which is positive or

negative with respect to neighbouring values on both

sides. It will fail to remove a section of the response

if it contains values which are between the two

levels on either side as shown in Figures 4 and 5.

This property of median filtering presents the

problem of how to perform segmentation using these

features without the generation of unwanted

segments along object boundaries, given that

separable median filtering will not remove the entire

boundary response. To achieve this we propose two

solutions, the first involves post-processing of the

median filtered images, and the second involves a

separate filtering technique. We will discuss each of

these in turn now.

(a) (b)

Figure 5: A boundary response which is positive with

respect to neighbouring values is shown in (a) and the

result post median filtering in (b).

The first approach we propose is to perform

segmentation at a greater spatial scale then the

extent of the sections of boundary response which

remain post separable median filtering. The

boundary responses remaining after separable

median filtering will in general be significantly

smaller then the scale of the window used in texture

feature extraction. Also the section of the boundary

response remaining will already resemble a step

edge at a higher scale due to the fact that it will

contain continuously increasing or decreasing

values. In fact it could be described as a smooth step

edge containing some noise. These two facts permit

the use of smoothing with a small Gaussian kernel

relative to the scale used in feature extraction. The

effect is to produce features represented at a spatial

scale where unwanted segments along boundaries

will not appear in the segmentation result. One

drawback of this method is that Gaussian smoothing

will always introduce a loss in boundary

localization. Figure 6 (a) shows the result of this

technique applied to Figure 3. An alternative

approach would be to perform the segmentation

algorithm at a higher scale, but this would lead to

under-segmentation if all boundaries did have

similar absolute differences.

(a) (b)

Figure 6: The results of both texture boundary response

removal techniques.

The second approach we propose to tackle the

boundary response problem involves the processing

of the feature image with a new filtering technique.

This technique takes as input two parameters; t the

threshold size which is the maximum size of an

object for it to be considered an object boundary

response and

the orientation of the texture feature

extraction algorithm. We first detail how this

method is implemented in 1-D and then extend it to

2-D. The 1-D method is implemented in two steps:

1) A 1-dimensional context window of length t

is aligned around a given point of interest

where the window contains that point and

minimizes the sum of absolute difference

between that point and the two boundary

points of the window. This step aligns the

window for a given point with the

neighbourhood to which it is most similar.

2) Then the point of interest is assigned the

boundary value of this window from which it

is most dissimilar.

These two steps are performed on every point in the

dataset. All boundary responses will be replaced

with step edges where the boundary responses cross

the midway point between the two uniform regions

on either side. If the two uniform regions either side

of a boundary response have values of 0 and 1, the

VISAPP 2007 - International Conference on Computer Vision Theory and Applications

366

step edge returned by the algorithm will be located

where the boundary response crosses the value of

0.5. We are working on the assumption that this is

the optimal point where the step edge should occur.

An illustration of this process applied to a data point

in a one-dimensional dataset is shown in Figures 7.

The result of the algorithm applied to Figure 4(a)

and Figure 5(a) is shown in Figure 8(a) and (b)

respectively.

Figure 7: Proposed texture boundary response removal

technique is performed to a point of interest which is a

member of a boundary response.

To extend this technique to two dimensions we first

apply the 1-D method in a direction parallel to the

direction of the feature extraction and then again in a

direction orthogonal to this. An Example of this

method applied to the Gabor feature image in Figure

3 is shown in Figure 6(b). We can see that this

method removes the boundary responses while

maintaining boundary localization and only suffers a

small drop in image detail.

(a) (b)

Figure 8: Results of the second proposed algorithm for

removing boundary responses applied to Figure 4(a) and

Figure 5(a) shown in (a) and (b) respectively

Following feature extraction all feature images are

smoothed by a non-linear locally adaptive diffusion

process. The techniques is similar to that used by

Black (Black and Sapiro 1999), except instead of

using the local median absolute deviation of grey

values we use the local median value of gradient

magnitude values. Also we calculate the edges at the

same scale as the feature extraction algorithm not at

the pixel level as done by Black.

4 TEXTURE BOUNDARY

RESPONSE REMOVAL AND

EVALUATION

To perform evaluation we judge the effectiveness of

our boundary response removal techniques with

respect to the improvement in segmentation

achieved once the boundary responses have been

removed. To perform segmentation we use the

marker controlled watershed transform (Soille,

2002). To prevent over-segmentation the gradient

magnitude image is first filtered using a marker

function, in this case the H-minima transform, to

remove all irrelevant minima (Soille, 2002). Figure 9

shows an example of segmentation achieved using

this algorithm.

Evaluation is performed using data from the

Berkeley segmentation dataset (Martin, Fowlkes et

al., 2001). For each of the images in this dataset 5 to

10 ground truths from different individuals are

available. For quantitative comparison of a single

segmentation result to a set of corresponding ground

truths the Normalized Probabilistic Rand (NPR)

index is used (Unnikrishnan, Pantofaru et al., 2005).

This index can be used to measure the relative

accuracy for various algorithms at producing a

useful segmentation for a given image. The greater

the index score, the greater the performance.

Figure 9: Segmentation performed by applying the

watershed transform.

For evaluation 200 images from the Berkeley

dataset are used and this is split into 100 training and

100 test images. Using the training set, the scale of

segmentation is optimized by varying the h-minima

value. The separable Median filtering of

(O'Callaghan and Bull, 2005) followed by

smoothing approach slightly outperforms the second

technique on the training dataset.

Using both boundary response removal

techniques optimized on the training dataset we

evaluate on the test dataset. On the 100 images in the

test dataset the separable median filtering followed

by smoothing technique achieved an average NPR

index value of 0.48, while the second boundary

Replacement

value

Point of

interest

Most dissimilar

window

boundary value

Context

window

aligned

REMOVING THE TEXTURE FEATURE RESPONSE TO OBJECT BOUNDARIES

367

response removal technique received an average

score of 0.44. The slight decrease in performance for

the second technique relative to the first is probably

due to the fact that the second technique is not as

robust to noise as the first.

5 CONCLUSIONS

The aim of this paper was to provide researchers in

the area of segmentation with a better understanding

of the texture boundary response problem. Prior to

this we could not find any work stating the different

forms of boundary responses which may occur and

how best to remove them. An evaluation of all

current solutions to removing these texture boundary

responses show separable median filtering to be the

current best solution. We analyzed the result of

applying separable median filtering to all possible

boundary responses and showed that it does not

remove all or parts of certain responses.

Two alternative techniques which overcome this

failing were proposed and evaluated. The first

technique is robust to noise but suffers from a loss in

boundary localization. The second technique gives

the optimal solution in a noise free environment but

is not so robust to noise. Using quantitative

evaluation the first approach of extracting edges at a

greater scale then the scale of the boundary

responses remaining after separable median filtering

was shown to perform best. This result does not

mean the second technique is redundant. If a feature

extraction algorithm which produces noise free

images could be found this technique could be used

to produce the optimal solution. This is based on the

assumption that all boundary responses should be

replaced with step edges where the boundary

response crosses the midway point between the two

uniform regions on either side. Future work will

attempt to evaluate whether this is the case.

If useful segmentation is to be produced, both

texture and intensity features must be extracted and

integrated in an intelligent manner. Future work will

also focus on the extraction of useful intensity

features and how best to integrate them with the

texture features discussed in this paper.

REFERENCES

Black, M. and G. Sapiro (1999). Edges as outliers:

anisotropic smoothing using local image statistics.

International conference on scale-space theories in

computer vision, Corfu, Greece. 259- 270.

Clausi, D. A. and M. E. Jernigan (2000). Designing Gabor

filters for optimal texture separability. Pattern

Recognition 33(11): 1835-1849.

Corcoran, P. and A. Winstanley (2006). Using Texture

to Tackle the Problem of Scale in Land- Cover

Classification. International conference on object

based image analysis, Salzburg.

Grigorescu, S. E., N. Petkov, et al. (2002). Comparison of

texture features based on Gabor filters. Image

Processing, IEEE Transactions on 11(10): 1160-1167.

Jobanputra, R. and D. A. Clausi (2006). Preserving

boundaries for image texture segmentation using grey

level co-occurring probabilities. Pattern Recognition

39(2): 234-245.

Kruizinga, P. and N. Petkov (1999). Nonlinear operator

for oriented texture. Image Processing, IEEE

Transactions on 8(10): 1395-1407.

Lim, J. S. (1989). Two-dimensional signal and image

processing, Prentice Hall.

Martin, D., C. Fowlkes, et al. (2001). A database of human

segmented natural images and its applications to

evaluating segmentation algorithms and measuring

ecological statistics. International conference on

computer vision, Vancouver. 416-423.

Martin, D. R., C. C. Fowlkes, et al. (2004). Learning to

detect natural image boundaries using local

brightness, color, and texture cues. IEEE Transactions

on Pattern Analysis and Machine Intelligence 26(5):

530-549.

O'Callaghan, R. J. and D. R. Bull (2005). Combined

morphological-spectral unsupervised image

segmentation. IEEE Transactions on Image Processing

14(1): 49-62.

Pantofaru, C. and M. Hebert (2005). A comparison of

image segmentation algorithms. Robotics institute,

Carnegie Mellon university, Pittsburgh, PA.

Technical Report No. CMU-RI-TR-05-40.

Shao, J. and W. Forstner (1994). Gabor wavelets for

texture edge extraction. ISPRS commission III

symposium on spatial information from digital

photogrammetry and computer vision, Munich,

Germany. 745-752.

Soille, P. (2002). Morphological image analysis:

principles and applications, Springer-Verlag Berlin

and Heidelberg GmbH & Co. K.

Unnikrishnan, R., C. Pantofaru, et al. (2005). A measure

for objective evaluation of image segmentation

algorithms. IEEE conference on computer vision and

pattern recognition, workshop on empirical

evaluation methods in c omputer vision, San

Diego.

VISAPP 2007 - International Conference on Computer Vision Theory and Applications

368