WINDOW DETECTION FROM TERRESTRIAL

LASER SCANNER DATA

A Statistical Approach

Haider Ali and Robert Sablatnig

Vienna University of Technology

Pattern Recognition and Image Processing Group (PRIP)

Favoritenstrasse 9/183-2, A-1040 Vienna, Austria

Gerhard Paar

Joanneum Research

Institute of Digital Image Processing

A-8010 Graz, Austria

Keywords:

Windows detection, Applied statistics, Deformation analysis, Facade segmentation and ROI.

Abstract:

This paper proposes a window detection system using applied statistics and image based methods from Ter-

restrial Laser Scanners which can be used for direct application in a deformation measurement system. It

exploits the laser distance information either directly in the laser scanner spherical coordinate space images,

or on segmented planar facade patches, both with the assumption that the laser beam penetrates windows. The

applied statistical method uses basic local features on local distance variations and decides on an adaptive

threshold on the basis of the 1 − Sigma percentile upper limit with P

90% and P

10% produced sample

quartiles of the data for the laser spherical coordinate system image and Q

− Sigma for the ortho images of

segmented 3D facade planes as a location in the order statistics. For window detection the image is binarized

and morphological closing is performed using the derived adaptive threshold. Thereafter we do the contour

analysis and obtain the bounding rectangles positions that directly form the window segments in the image.

We compare the window detection results on the laser spherical coordinate system image with those on ortho

images of segmented 3D facades. The system provides a windows detection rate of more than 85% with a

processing time of less than a minute in a typical 360 degree laser scan image.

1 INTRODUCTION

Deformation processes in highly populated areas,

stemming from underground construction work, un-

stable and wet underground, as well as earthquakes

and other similar disasters, are increasing, not only

in alpine areas. Monitoring of such events is a com-

plex task, the more so if a large urban area is involved

(Deffontaines et al., ). It requires to establish non-

stationary sensor systems several times on similar

viewpoints, and re-detect the same set of points of in-

terest for each measurement epoch (Erol et al., 2004).

The order of measurements can be predeﬁned either

as individual measurements or as repetitive measure-

ments (each hour at same minute, each day at same

hour, each month at same day). Such stable points of

interest can be found automatically around windows,

which in such way form salient candidates for an au-

tomatic point-of-interest selection process. The pur-

pose of window detection as described in this paper

is therefore to automatically establish a data base of

points suitable for deformation monitoring (e.g. from

the corners of the detected windows) on one hand, and

on the other hand re-detect them for repeated mea-

surement to enable a robust tracking of the same point

over time.

1.1 Related Work

The aim of this work is the detection of windows in

the original and segmented building facades from Ter-

restrial Laser Scanner Data. (Bauer et al., 2003) pre-

393

Ali H., Sablatnig R. and Paar G. (2009).

WINDOW DETECTION FROM TERRESTRIAL LASER SCANNER DATA - A Statistical Approach.

In Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, pages 393-397

DOI: 10.5220/0001786303930397

 SciTePress

sented a robust approach for detection and partition

of planar surfaces in dense 3D point clouds from fa-

cades. A feature based building segmentation algo-

rithm for an object dependent 3D generalization is

developed by (Frank and Sester, 2004). (Lerma and

Biosca, 2005) presented an automatic approach to ob-

tain planar surfaces on scanned monuments and re-

duce the data volume. The proposed algorithm extract

planar surfaces and reduce non relevant data points

based on the clustering techniques. An automatic

system for the segmentation and extraction of pla-

nar parts using RANSAC is developed by (Boulaassal

et al., 2007). (Mayer and Reznik, 2006) proposed an

approach to determine the 3D position of windows by

plane sweeping for building facades interpretation in

multiple images. Automatic marker-free registration

of Terrestrial Laser Scans using reﬂectance features is

presented by (Bohm and Becker, 2007). A modeling

process for 3D object representation by cell decompo-

sition for building reconstruction at different scales is

presented by (Becker and Haala, 2007). (Pu and Vos-

selman, 2007) presented an approach for automatic

extraction of windows from terrestrial point clouds.

They ﬁrst segment the laser points in planar segments

and then apply two detection strategies for two differ-

ent classes (covered and non-covered with curtains)

of windows. The system is based on different seg-

mentation algorithms and retrieves potential building

features like (doors, walls, windows, etc.) to recog-

nize buildings but operates only on frontal views of

the buildings.

The system we provide covers more general build-

ing views. The ﬁrst option directly operates on 3D

data points and does not involve any 3D segmentation

technique. The second option uses 3D segmentation

of planar surface patches and thereafter applies win-

dow detection. Both options are evaluated on a well

known data set, they perform signiﬁcantly better than

the system proposed by (Pu and Vosselman, 2007) in

terms of time, accuracy and robustness. Robust win-

dow detection can be either applied to the laser spher-

ical coordinate system image or to ortho images of the

segmented 3D facades.

1.2 Overview

The laser scanning system provides ordered 3D point

clouds in a dedicated image structure. It can be shown

that the measured distance signiﬁcantly changes in the

window regions of the facade, either by window pene-

tration, or by (occasional) reﬂection as show in Figure

It is therefore near at hand to exploit these changes

for window detection.

Figure 1: An example of opened window and occasional

reﬂection (The reﬂection of other building in the window).

The system works optionally directly in the spher-

ical coordinate system laser-scanner distance image

or on a digital surface model on segmented planar

patches of the facades that were gained by a clustering

and indexing using an iterative parameter estimation

method (RANdom SAmple Consensus, RANSAC).

Local variations in these data structures are analyzed

by calculating the differences of the distances be-

tween two adjacent pixels. An adaptive threshold is

applied to identify candidate pixels for window re-

gions. Morphological operations and contour anal-

ysis lead to rectangular regions that are direct bound-

ing boxes around the segmented windows. The pla-

nar segmentation facilitates this process, since ob-

ject dimensions can be directly derived from the pla-

nar ortho image and surface models, the calculated

rectangles are parallel with the local coordinate axes,

and they can therefore directly be used for points-of-

interest output on their corners. The global workﬂow

of the system involves 3D data analysis and facade

segmentation in order to identify window segments is

presented in Figure 2.

Figure 2: Pipeline for window detection from a 3D laser

spherical coordinate system image, or optionally using or-

tho images from 3D Facade segmentation which facilitates

the window detection process.

2 FACADE SEGMENTATION

We collect 3D point clouds using a long-range laser

scanner (LPM-2k by Riegl Laser Measurement Sys-

tems) with an operating range of 10m-300m. The

laser scanner is based on the time-of-ﬂight method,

for each single measurement a burst of several hun-

dred laser pulses are emitted. The reﬂected re-

VISAPP 2009 - International Conference on Computer Vision Theory and Applications

394

turn pulses are analyzed by a Digital Signal Pro-

cessor (DSP) to compile a single distance measure-

ment. Several measurement modes (”ﬁrst target”,

”last target”, ”strongest target”) can be selected for

the method used by the DSP. These data points are

stored in an image structure in a spherical coordinate

system, containing, for each measured point, its exact

spherical coordinates, its distance from the laser scan-

ner origin, as well as optional RGB texture values.

Plane segmentation iteratively selects a random set of

10 − 20 points in a local vicinity within this image.

Whenever consistency to a regression plane is within

a given threshold for such a sample, this plane is a

candidate for a facade patch. Additional constraints

apply such as near-horizontal normal vector, mini-

mum number of additionally plane - compliant pix-

els within the bounding box of the sample, and min-

imum/maximum angle to other intersecting already

identiﬁed planar patches. Iterative reﬁnement of the

plane is established by a region-growing approach.

This solution leads to the results as shown in Figure

Figure 3: Top: Laser RGB texture (shading corrected) in

spherical coordinate system, Bottom: Plane (colored) seg-

mentation using RANSAC.

3 WINDOW DETECTION

Window detection system is a schematic process of

3D data acquisition, analysis and morphological op-

erations to detect window segments in laser spherical

coordinate system image and ortho distance images

of segmented 3D facades.

3.1 Statistical Analysis of Distances

For the selection of window candidates pixels in the

original spherical coordinate laser distance image, the

distribution of the local distances variation is obtained

D = dx

− dx

(1)

with D the used difference of local distance, dx

the

distance at laser image coordinate (x

,y), and dx

the

distance at point (x

+ 1, y) for each laser image row

A conﬁdence interval of 1 − Sigma percentile up-

per limit of the mean with P

90% and P

10%

produced sample quantiles of the data is used as an

adaptive threshold value to identify regions of inter-

est (ROIs) as given by (2).

T = P

+ (P

− P

)/2 (2)

In contrast to the direct laser images, the facade

segmentation provides planes that cover more frontal

views used as input to the window detection system

for further processing. An adaptive threshold upper

limit value is decided on the basis of the third and

ﬁrst quartiles (Q

, Q

), 75% and 25% respectively as

given by (3).

T = Q

− (Q

− Q

)/2 (3)

T can be directly used for binarization. An exam-

ple of such an image masking is presented in Figure

Figure 4: RGB image of urban scene in spherical laser scan-

ner coordinates, overlaid by a binary mask from threshold-

ing that resembles window candidate pixels.

3.2 Morphological Operations and

Window Bounding Boxes

The image is binarized using the adaptive threshold

value. A morphological higher order operation (clos-

ing) is performed using a 3 ∗ 3 structuring element.

The closing rejoins the splitted windows due to

the change of distance on facade and window re-

gions. An example of closing is presented in Figure 5.

Figure 5: Closing joins the separate portion of the windows

to be considered as a single window for the detection frame-

work, examples from facades ortho images.

An Open Source connectivity preserving contour

analysis method (ﬁnd contours) is used to ﬁnd and re-

trieve the window regions in the binary image. There-

after Opening and connected components labeling

Window Detection from Terrestrial Laser Scanner Data - A Statistical Approach

395

leads to window contours and their up-right bound-

ing rectangles are calculated. As the windows are of

rectangular shape, we determine the area of the con-

tour and the bounding rectangle and decide that R is a

valid rectangle around a window, if

− A

< (0.5 ∗ A

) (4)

with A

being the Rectangle area, and A

the con-

tour area (4). In experiments we found 0.5 an appro-

priate threshold value to identify window segments.

Determining an adaptive threshold value is subject to

further research.

4 EVALUATION AND RESULTS

We have evaluated the system on the laser spherical

coordinate system image as well as ortho distance im-

ages of segmented 3D facades images of a database

acquired from different viewpoints in urban environ-

ment. Ground truth was determined manually. To

evaluate the window detection rate we counted a true

positive, if the window is correctly detected in the

data-analysis phase, false negative if an existing win-

dow was not detected and false positive, if a win-

dow was detected where there is actually no window

present. The dataset contains a total of 400 windows

being 322 contained in laser spherical coordinate im-

ages and 78 in the segmented facades ortho images.

The detection rate is provided in the Table 1.

Table 1: Robust Windows Detection Evaluation Results on

laser spherical coordinate system image and ortho distance

images of segmented 3D facades

Database(400) Laser Images(322) Ortho Images(78)

True Positive 283 65

False Negative 39 13

False Positive 10 0

A comparison in terms of window size con-

sistency between the windows detected with and

without the optional facade segmentation is given in

Figures [6] and [7].

Figure 6: (a), (b) Examples for Window Segments detected

by the system on the laser spherical coordinate system im-

age.

Figure 7: (a), (b) Examples for Robust Window Segments

detected by the system on the ortho images of segmented

3D facades.

5 CONCLUSIONS

In this paper we have presented an image-based win-

dow detection system to measure 3D objects for ur-

ban deformation monitoring applications. The defor-

mation analysis system gathers accurate 3D measure-

ments on unique points of the building surface in dif-

ferent epochs and compares them across various time

spans. It is therefore fed with 3D coordinates (i.e.

bounding boxes) of the detected windows. The win-

dow detection system works directly in the spherical

coordinate image of the laser scanner, or optionally on

ortho distance images of segmented facades, which is

based on RANSAC selection of sample points and it-

erative regression plane reﬁnement. Window detec-

tion relies on a statistical analysis of distance distri-

butions to binarize and segment the images for rect-

angular bounding boxes of windows. Both data sets

reach a detection accuracy of more than 80%. The re-

sults on the segmented facades are superior in terms

of window size consistency and false positive detec-

tions.

Future work will include the integration of the sys-

tem into a deformation monitoring framework (Re-

iterer, 2007), the exploitation of window context in-

formation (e.g. arrangement of windows w.r.t. each

other), and the exploration of other applications such

as thermal facade analysis for emission control, or the

task of city modeling for virtual reality purposes.

ACKNOWLEDGEMENTS

This work was partially funded by the FWF

Austrian research project ”Multi Sensor Deforma-

tion Measurement System Supported by Knowl-

edge Based and Cognitive Vision Techniques”

(P18286-N04), ”

Osterreichische Orient-Gesellschaft

Hammer-Purgstall” and Joanneum Research.

VISAPP 2009 - International Conference on Computer Vision Theory and Applications

396

REFERENCES

Bauer, J., Karner, K., Schindler, K., Klaus, A., and Zach., C.

(2003). Segmentation of building models from dense

3d point-clouds. In 27th Workshop of the Austrian

Association for Pattern Recognition (OeAGM), pages

253–259, Laxenburg, Austria.

Becker, S. and Haala, N. (2007). Reﬁnement of building fa-

cades by integrated processing of lidar and image data.

In ISPRS Conference Photogrammetric Image Analy-

sis (PIA), volume 36, pages 07–12, Munich, Germany.

Bohm, J. and Becker, S. (2007). Automatic marker-free

registration of terrestrial laser scans using reﬂectance

features. In Proceedings of 8th Conference on Optical

3D Measurment Techniques, pages 338–344, Zurich,

Switzerland.

Boulaassal, H., Landes, T., Grussenmeyer, P., and Tar-

sha Kurdi, F. (2007). Automatic segmentation of

building facades using terrestrial laser data. In IS-

PRS Workshop on Laser Scanning 2007 and Silvi-

Laser 2007, page 65, Espoo, Finland.

Deffontaines, B., Fruneau, B., Rudant, J.-P., and Leparmen-

tier, A.-M. Validation of Monitoring vertical defor-

mations due to construction works using SAR radar

interferometry : A case example. EGS - AGU - EUG

Joint Assembly, Nice, France, 6 - 11 April 2003.

Erol, S., Erol, B., and Ayan, T. (2004). A general review

of the deformation monitoring techniques and a case

study: Analysing deformations using gps/levelling.

In International Archives of Photogrammetry Remote

Sensing and Spatial Information Sciences, volume 35,

pages 622–627, United Kingdom.

Frank, T. and Sester, M. (2004). Segmentation of buildings

for 3d generalisation. In Workshop on generalisation

and multiple representation, ICA, commission on map

generalisation, Leicester, United Kingdom.

Lerma, J. and Biosca, J. (2005). Segmentation and ﬁltering

of laser scanner data for cultural heritage. In Inter-

national Archives of Photogrammetry, Remote Sens-

ing and Spatial Information Sciences, volume 36(5),

pages 896–901, Torino, Italy.

Mayer, H. and Reznik, S. (2006). Mcmc linked with im-

plicit shape models and plane sweeping for 3d build-

ing facade interpretation in image sequences. In In-

ternational Archives of the Photogrammetry, Remote

Sensing and Spatial Information Sciences, volume 36,

pages 130–135, Bonn, Germany.

Pu, S. and Vosselman, G. (2007). Extracting windows

from terrestrial laser scanning. In ISPRS Workshop on

Laser Scanning 2007 and SilviLaser 2007, page 320,

Espoo, Finland.

Reiterer, A. (2007). The development of an online

knowledge-based videotheodolite measurement sys-

tem. In Knowledge Based Systems, volume 20(1),

pages 73–85, Amsterdam, Netherlands.

Window Detection from Terrestrial Laser Scanner Data - A Statistical Approach

397