A Statistical Approach
Haider Ali and Robert Sablatnig
Vienna University of Technology
Pattern Recognition and Image Processing Group (PRIP)
Favoritenstrasse 9/183-2, A-1040 Vienna, Austria
Gerhard Paar
Joanneum Research
Institute of Digital Image Processing
A-8010 Graz, Austria
Windows detection, Applied statistics, Deformation analysis, Facade segmentation and ROI.
This paper proposes a window detection system using applied statistics and image based methods from Ter-
restrial Laser Scanners which can be used for direct application in a deformation measurement system. It
exploits the laser distance information either directly in the laser scanner spherical coordinate space images,
or on segmented planar facade patches, both with the assumption that the laser beam penetrates windows. The
applied statistical method uses basic local features on local distance variations and decides on an adaptive
threshold on the basis of the 1 Sigma percentile upper limit with P
90% and P
10% produced sample
quartiles of the data for the laser spherical coordinate system image and Q
Sigma for the ortho images of
segmented 3D facade planes as a location in the order statistics. For window detection the image is binarized
and morphological closing is performed using the derived adaptive threshold. Thereafter we do the contour
analysis and obtain the bounding rectangles positions that directly form the window segments in the image.
We compare the window detection results on the laser spherical coordinate system image with those on ortho
images of segmented 3D facades. The system provides a windows detection rate of more than 85% with a
processing time of less than a minute in a typical 360 degree laser scan image.
Deformation processes in highly populated areas,
stemming from underground construction work, un-
stable and wet underground, as well as earthquakes
and other similar disasters, are increasing, not only
in alpine areas. Monitoring of such events is a com-
plex task, the more so if a large urban area is involved
(Deffontaines et al., ). It requires to establish non-
stationary sensor systems several times on similar
viewpoints, and re-detect the same set of points of in-
terest for each measurement epoch (Erol et al., 2004).
The order of measurements can be predefined either
as individual measurements or as repetitive measure-
ments (each hour at same minute, each day at same
hour, each month at same day). Such stable points of
interest can be found automatically around windows,
which in such way form salient candidates for an au-
tomatic point-of-interest selection process. The pur-
pose of window detection as described in this paper
is therefore to automatically establish a data base of
points suitable for deformation monitoring (e.g. from
the corners of the detected windows) on one hand, and
on the other hand re-detect them for repeated mea-
surement to enable a robust tracking of the same point
over time.
1.1 Related Work
The aim of this work is the detection of windows in
the original and segmented building facades from Ter-
restrial Laser Scanner Data. (Bauer et al., 2003) pre-
Ali H., Sablatnig R. and Paar G. (2009).
In Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, pages 393-397
DOI: 10.5220/0001786303930397
sented a robust approach for detection and partition
of planar surfaces in dense 3D point clouds from fa-
cades. A feature based building segmentation algo-
rithm for an object dependent 3D generalization is
developed by (Frank and Sester, 2004). (Lerma and
Biosca, 2005) presented an automatic approach to ob-
tain planar surfaces on scanned monuments and re-
duce the data volume. The proposed algorithm extract
planar surfaces and reduce non relevant data points
based on the clustering techniques. An automatic
system for the segmentation and extraction of pla-
nar parts using RANSAC is developed by (Boulaassal
et al., 2007). (Mayer and Reznik, 2006) proposed an
approach to determine the 3D position of windows by
plane sweeping for building facades interpretation in
multiple images. Automatic marker-free registration
of Terrestrial Laser Scans using reflectance features is
presented by (Bohm and Becker, 2007). A modeling
process for 3D object representation by cell decompo-
sition for building reconstruction at different scales is
presented by (Becker and Haala, 2007). (Pu and Vos-
selman, 2007) presented an approach for automatic
extraction of windows from terrestrial point clouds.
They first segment the laser points in planar segments
and then apply two detection strategies for two differ-
ent classes (covered and non-covered with curtains)
of windows. The system is based on different seg-
mentation algorithms and retrieves potential building
features like (doors, walls, windows, etc.) to recog-
nize buildings but operates only on frontal views of
the buildings.
The system we provide covers more general build-
ing views. The first option directly operates on 3D
data points and does not involve any 3D segmentation
technique. The second option uses 3D segmentation
of planar surface patches and thereafter applies win-
dow detection. Both options are evaluated on a well
known data set, they perform significantly better than
the system proposed by (Pu and Vosselman, 2007) in
terms of time, accuracy and robustness. Robust win-
dow detection can be either applied to the laser spher-
ical coordinate system image or to ortho images of the
segmented 3D facades.
1.2 Overview
The laser scanning system provides ordered 3D point
clouds in a dedicated image structure. It can be shown
that the measured distance significantly changes in the
window regions of the facade, either by window pene-
tration, or by (occasional) reflection as show in Figure
It is therefore near at hand to exploit these changes
for window detection.
Figure 1: An example of opened window and occasional
reflection (The reflection of other building in the window).
The system works optionally directly in the spher-
ical coordinate system laser-scanner distance image
or on a digital surface model on segmented planar
patches of the facades that were gained by a clustering
and indexing using an iterative parameter estimation
method (RANdom SAmple Consensus, RANSAC).
Local variations in these data structures are analyzed
by calculating the differences of the distances be-
tween two adjacent pixels. An adaptive threshold is
applied to identify candidate pixels for window re-
gions. Morphological operations and contour anal-
ysis lead to rectangular regions that are direct bound-
ing boxes around the segmented windows. The pla-
nar segmentation facilitates this process, since ob-
ject dimensions can be directly derived from the pla-
nar ortho image and surface models, the calculated
rectangles are parallel with the local coordinate axes,
and they can therefore directly be used for points-of-
interest output on their corners. The global workflow
of the system involves 3D data analysis and facade
segmentation in order to identify window segments is
presented in Figure 2.
Figure 2: Pipeline for window detection from a 3D laser
spherical coordinate system image, or optionally using or-
tho images from 3D Facade segmentation which facilitates
the window detection process.
We collect 3D point clouds using a long-range laser
scanner (LPM-2k by Riegl Laser Measurement Sys-
tems) with an operating range of 10m-300m. The
laser scanner is based on the time-of-flight method,
for each single measurement a burst of several hun-
dred laser pulses are emitted. The reflected re-
VISAPP 2009 - International Conference on Computer Vision Theory and Applications
turn pulses are analyzed by a Digital Signal Pro-
cessor (DSP) to compile a single distance measure-
ment. Several measurement modes (”first target”,
”last target”, ”strongest target”) can be selected for
the method used by the DSP. These data points are
stored in an image structure in a spherical coordinate
system, containing, for each measured point, its exact
spherical coordinates, its distance from the laser scan-
ner origin, as well as optional RGB texture values.
Plane segmentation iteratively selects a random set of
10 20 points in a local vicinity within this image.
Whenever consistency to a regression plane is within
a given threshold for such a sample, this plane is a
candidate for a facade patch. Additional constraints
apply such as near-horizontal normal vector, mini-
mum number of additionally plane - compliant pix-
els within the bounding box of the sample, and min-
imum/maximum angle to other intersecting already
identified planar patches. Iterative refinement of the
plane is established by a region-growing approach.
This solution leads to the results as shown in Figure
Figure 3: Top: Laser RGB texture (shading corrected) in
spherical coordinate system, Bottom: Plane (colored) seg-
mentation using RANSAC.
Window detection system is a schematic process of
3D data acquisition, analysis and morphological op-
erations to detect window segments in laser spherical
coordinate system image and ortho distance images
of segmented 3D facades.
3.1 Statistical Analysis of Distances
For the selection of window candidates pixels in the
original spherical coordinate laser distance image, the
distribution of the local distances variation is obtained
D = dx
with D the used difference of local distance, dx
distance at laser image coordinate (x
,y), and dx
distance at point (x
+ 1, y) for each laser image row
A confidence interval of 1 Sigma percentile up-
per limit of the mean with P
90% and P
produced sample quantiles of the data is used as an
adaptive threshold value to identify regions of inter-
est (ROIs) as given by (2).
T = P
+ (P
)/2 (2)
In contrast to the direct laser images, the facade
segmentation provides planes that cover more frontal
views used as input to the window detection system
for further processing. An adaptive threshold upper
limit value is decided on the basis of the third and
first quartiles (Q
, Q
), 75% and 25% respectively as
given by (3).
T = Q
)/2 (3)
T can be directly used for binarization. An exam-
ple of such an image masking is presented in Figure
Figure 4: RGB image of urban scene in spherical laser scan-
ner coordinates, overlaid by a binary mask from threshold-
ing that resembles window candidate pixels.
3.2 Morphological Operations and
Window Bounding Boxes
The image is binarized using the adaptive threshold
value. A morphological higher order operation (clos-
ing) is performed using a 3 3 structuring element.
The closing rejoins the splitted windows due to
the change of distance on facade and window re-
gions. An example of closing is presented in Figure 5.
Figure 5: Closing joins the separate portion of the windows
to be considered as a single window for the detection frame-
work, examples from facades ortho images.
An Open Source connectivity preserving contour
analysis method (find contours) is used to find and re-
trieve the window regions in the binary image. There-
after Opening and connected components labeling
Window Detection from Terrestrial Laser Scanner Data - A Statistical Approach
leads to window contours and their up-right bound-
ing rectangles are calculated. As the windows are of
rectangular shape, we determine the area of the con-
tour and the bounding rectangle and decide that R is a
valid rectangle around a window, if
< (0.5 A
) (4)
with A
being the Rectangle area, and A
the con-
tour area (4). In experiments we found 0.5 an appro-
priate threshold value to identify window segments.
Determining an adaptive threshold value is subject to
further research.
We have evaluated the system on the laser spherical
coordinate system image as well as ortho distance im-
ages of segmented 3D facades images of a database
acquired from different viewpoints in urban environ-
ment. Ground truth was determined manually. To
evaluate the window detection rate we counted a true
positive, if the window is correctly detected in the
data-analysis phase, false negative if an existing win-
dow was not detected and false positive, if a win-
dow was detected where there is actually no window
present. The dataset contains a total of 400 windows
being 322 contained in laser spherical coordinate im-
ages and 78 in the segmented facades ortho images.
The detection rate is provided in the Table 1.
Table 1: Robust Windows Detection Evaluation Results on
laser spherical coordinate system image and ortho distance
images of segmented 3D facades
Database(400) Laser Images(322) Ortho Images(78)
True Positive 283 65
False Negative 39 13
False Positive 10 0
A comparison in terms of window size con-
sistency between the windows detected with and
without the optional facade segmentation is given in
Figures [6] and [7].
Figure 6: (a), (b) Examples for Window Segments detected
by the system on the laser spherical coordinate system im-
Figure 7: (a), (b) Examples for Robust Window Segments
detected by the system on the ortho images of segmented
3D facades.
In this paper we have presented an image-based win-
dow detection system to measure 3D objects for ur-
ban deformation monitoring applications. The defor-
mation analysis system gathers accurate 3D measure-
ments on unique points of the building surface in dif-
ferent epochs and compares them across various time
spans. It is therefore fed with 3D coordinates (i.e.
bounding boxes) of the detected windows. The win-
dow detection system works directly in the spherical
coordinate image of the laser scanner, or optionally on
ortho distance images of segmented facades, which is
based on RANSAC selection of sample points and it-
erative regression plane refinement. Window detec-
tion relies on a statistical analysis of distance distri-
butions to binarize and segment the images for rect-
angular bounding boxes of windows. Both data sets
reach a detection accuracy of more than 80%. The re-
sults on the segmented facades are superior in terms
of window size consistency and false positive detec-
Future work will include the integration of the sys-
tem into a deformation monitoring framework (Re-
iterer, 2007), the exploitation of window context in-
formation (e.g. arrangement of windows w.r.t. each
other), and the exploration of other applications such
as thermal facade analysis for emission control, or the
task of city modeling for virtual reality purposes.
This work was partially funded by the FWF
Austrian research project ”Multi Sensor Deforma-
tion Measurement System Supported by Knowl-
edge Based and Cognitive Vision Techniques”
Osterreichische Orient-Gesellschaft
Hammer-Purgstall” and Joanneum Research.
VISAPP 2009 - International Conference on Computer Vision Theory and Applications
Bauer, J., Karner, K., Schindler, K., Klaus, A., and Zach., C.
(2003). Segmentation of building models from dense
3d point-clouds. In 27th Workshop of the Austrian
Association for Pattern Recognition (OeAGM), pages
253–259, Laxenburg, Austria.
Becker, S. and Haala, N. (2007). Refinement of building fa-
cades by integrated processing of lidar and image data.
In ISPRS Conference Photogrammetric Image Analy-
sis (PIA), volume 36, pages 07–12, Munich, Germany.
Bohm, J. and Becker, S. (2007). Automatic marker-free
registration of terrestrial laser scans using reflectance
features. In Proceedings of 8th Conference on Optical
3D Measurment Techniques, pages 338–344, Zurich,
Boulaassal, H., Landes, T., Grussenmeyer, P., and Tar-
sha Kurdi, F. (2007). Automatic segmentation of
building facades using terrestrial laser data. In IS-
PRS Workshop on Laser Scanning 2007 and Silvi-
Laser 2007, page 65, Espoo, Finland.
Deffontaines, B., Fruneau, B., Rudant, J.-P., and Leparmen-
tier, A.-M. Validation of Monitoring vertical defor-
mations due to construction works using SAR radar
interferometry : A case example. EGS - AGU - EUG
Joint Assembly, Nice, France, 6 - 11 April 2003.
Erol, S., Erol, B., and Ayan, T. (2004). A general review
of the deformation monitoring techniques and a case
study: Analysing deformations using gps/levelling.
In International Archives of Photogrammetry Remote
Sensing and Spatial Information Sciences, volume 35,
pages 622–627, United Kingdom.
Frank, T. and Sester, M. (2004). Segmentation of buildings
for 3d generalisation. In Workshop on generalisation
and multiple representation, ICA, commission on map
generalisation, Leicester, United Kingdom.
Lerma, J. and Biosca, J. (2005). Segmentation and filtering
of laser scanner data for cultural heritage. In Inter-
national Archives of Photogrammetry, Remote Sens-
ing and Spatial Information Sciences, volume 36(5),
pages 896–901, Torino, Italy.
Mayer, H. and Reznik, S. (2006). Mcmc linked with im-
plicit shape models and plane sweeping for 3d build-
ing facade interpretation in image sequences. In In-
ternational Archives of the Photogrammetry, Remote
Sensing and Spatial Information Sciences, volume 36,
pages 130–135, Bonn, Germany.
Pu, S. and Vosselman, G. (2007). Extracting windows
from terrestrial laser scanning. In ISPRS Workshop on
Laser Scanning 2007 and SilviLaser 2007, page 320,
Espoo, Finland.
Reiterer, A. (2007). The development of an online
knowledge-based videotheodolite measurement sys-
tem. In Knowledge Based Systems, volume 20(1),
pages 73–85, Amsterdam, Netherlands.
Window Detection from Terrestrial Laser Scanner Data - A Statistical Approach