UAV Technology Integration for Remote Sensing Image Analysis

Marco Piragnolo and Francesco Pirotti

CIRGEO, Interdepartmental Research Center of Geomatics, University of Padua, Viale dell'Università,

16 35020 Legnaro, Italy

1 RESEARCH PROBLEM

In this paper, we focus on a multilevel remote

sensing framework to integrate information obtained

through UAV images, satellite images from Sentinel

I and II, radiometric analysis, and spatial

information in order to derive informative maps to

be used for educated decision support. The goal is

the answer to the following questions:

1. Could the classic techniques of remote

sensing be used to extract suitable land use

maps – suitable in terms of classification

accuracy – also for the very high resolution

UAV images?

2. Which methods are optimal to analyze UAV

images and which benefits could be achieved

through the use of more sophisticated

techniques, such as the integration of multi-

source spatial data to add to the feature

vector?

3. Could information from UAV images be

merged with data from satellite images in the

same area, in order to achieve better results?

2 OUTLINE OF OBJECTIVES

Based on the above research questions, the specific

objectives of this project are:

1. To test standard classification methods of

remote sensing to UAV multispectral images.

2. To integrate spatial and morphological

information of objects to the machine

learning methods applied for classification

3. To test advanced classifiers to UAV only and

to UAV and satellite data integrated together.

3 STATE OF THE ART

In the last years there was a growing demand for

innovative tools to monitor geomorphological

aspects for environmental analyses, land use,

fragmentation of habitats and risk assessment

(Piragnolo et al., 2014a; Piragnolo et al., 2014b; Van

Asselen et al., 2013) in particular in rural areas

which, in many cases, have proved to be of strategic

importance to national and regional economy

(Marsden, 2010; van Eupen et al., 2012).

Recently, unmanned aircraft vehicles (UAVs)

have seen great attention from the scientific

community. There are many aspects regarding this

attention, the main one is the prospect to get high-

resolution data “on demand” quickly at a relatively

low cost. The technology in terms of cost and

availability follows the typical development curve:

the prices and weight of the components have

decreased, data accuracy has increased, and all with

a lower power demand, or a constant power and

greater durability of the apparatus as a whole. The

market has come at a point where the cost for the

apparatus, with RGB or multispectral sensors,

becomes accessible to amateur users and to a large

audience. Research fields are cultural heritage,

archaeology, 3D survey, environmental, forestry and

precision agriculture (Berni, 2009; Haarbrink and

Koers, 2006; Herwitz, 2004; Hunt, 2010; Lelong,

2008; Remondino et al., 2011).

Software for image processing is playing a key

role in the diffusion of UAV technology. Since the

accuracy of the positioning systems and orientation

is not comparable to the classical systems of aerial

photogrammetry, software would compensate this

limit with a massive use of image matching and

structure from motion (SfM) techniques. These

techniques, coupled with computer vision

algorithms, have led to the development of various

software for photogrammetric processing available

with commercial licenses and Open Source licenses

(Remondino, 2012). Several authors (Grenzdörffer

et al., 2008; Sona et al., 2014) have reviewed these

new technology and they have reported some

problems in photogrammetric, radiometric aspects

and data size:

1. Photogrammetric problems concern the

limited size and quality of the sensor in the

Piragnolo, M. and Pirotti, F.

UAV Technology Integration for Remote Sensing Image Analysis.

In Doctoral Consortium (DCGISTAM 2016), pages 12-19

camera mounted on the UAV; i.e. missing

information regarding the internal orientation,

distortion of frames, overlapping of frames,

low precision of GPS-INS, high number of

ground control point (GCP) required.

2. Radiometric problems are related to image

interpretation, correct use of radiometric

information, new techniques for the

processing of Multispectral Data and

calculation of derived index (Honkavaara et

al., 2012, Torres-Sanchez et al., 2014).

3. Sensors with high spatial and temporal

resolution produce massive data size which

increases exponentially (Zaslavsky, 2013).

Data size and processing time can be related

to the Big Data paradigm: Big Data not only

relates to physical storage, but also to the

velocity of acquisition and variability of

number of files, tables, records and

processing time (Singh, 2012).

Photogrammetric techniques will be used to

obtain the basic data. The evaluation and

improvement of the accuracy of the

photogrammetric survey will be studied marginally

as it has to be taken into account to provide the

spatial error budget. In literature, many authors have

proposed new frameworks, GIS environments and

objects algorithms in order to solve problems of size

and scalability of dataset (Baumann, 2014; Lin et al.,

2013; Peña et al., 2013; Zhao and He, 2009).

Radiometric analyses for segmentation and

classification for GIS environment are the issues that

will be considered in this study.

4 METHODOLOGY

The issues that will be considered are related to

analysis in GIS environment thus with full spatial

support like image interpretation, spectral

information, the calculation of derived indices and

the integration of other spatial data (data fusion).

UAV data will be collected in test areas where

ground information is acquired from experts

assigning agricultural classes depending on crop

type and yield. These data will be analysed in order

to understand whether the classic techniques of

remote sensing could be applied - i.e. minimum

distance, maximum likelihood algorithms (Richards,

2006) and spectral angle mapping SAM (Kruse,

1993) – to correctly return the class of the area.

Whether new techniques are necessary and which

benefits could be achieved through the use of more

advanced techniques, such as the integration of

spatial data to increase the number of features

describing significantly the phenomena, which we

want to model. The integration of information

obtained through photogrammetric methods and

remote sensing, such as Sentinel-2 data, might

improve the quality of derived products such as land

use maps. The accuracies of the classification

methods will be evaluated by weighing both the

feature information from the reflectance from the

spectral bands (optical information), and the

information on the spatial proximity between classes

or morphological information of the objects; spatial

and morphological information is the third

dimension obtained by photogrammetric technique

(Dalponte et al., 2008). A first example of feature

vector with elements that will be tested is [b1, b2,

b3, b4, b5, H, P] where bx are the bands of

wavelength increasing from blue to near infrared, H

refers to height from the ground, and P refers to

slope. Standard classifiers and sophisticated

classifiers such as support vector machines (SVM)

(Melgani and Bruzzone, 2004) and Random Forest

(Brieman 2001) will be tested.

Considering the continuous use of multiband

UAV digital images, it is necessary to structure data

and to apply a harmonious management. It is

important to manage the "raw" data, and information

obtained from the various stages of the processing,

to define the standard products; these data must be

kept for further analysis.

5 EXPECTED OUTCOME

5.1 Multilevel Remote Sensing

Framework

The expected outcome is to set a procedure for

classification and relative algorithms for integrating

satellite and UAV data with other spatial

information. The best algorithms in term of

performance could be integrated in a multilevel

remote sensing framework. The framework could

integrate the information obtained through

photogrammetric methods and remote sensing

techniques (Figure 1). A first classification at

smaller scale will be executed on satellite images.

Classification results and accuracies will be

evaluated using a control dataset which consists of

an independent classification. In case of errors a

deeper analysis at larger scale will be necessary, e.g.

using aerial or drones orthopotos.

UAV Technology Integration for Remote Sensing Image Analysis

Figure 1: Multilevel framework.

Figure 2: Classification map of land use produced by

random forest algorithm.

In Figure 2 we present an initial classification of

a test area. It is located at south-east of city of

Padova, in Italian Veneto Region. The classification

is based on Sentinel II images using random

Random Forest algorithm.

Figure 3 shows the UAV image of the test area

flown with a drone. The overlap shows a

disagreement between Urban class of classification

(red pixels) and crops that can be recognized in

UAV orthomosaic.

Figure 3: Testing area was flew by drone.

The final classification will be cross validated

using a ground-truth dataset acquired by a team of

professionals working in the field of land-use maps.

The expected outcome is a robust procedure to

integrate UAV and satellite data to support decision

procedures mainly, but not limited to, the field of

agricultural crop administration.

DCGISTAM 2016 - Doctoral Consortium on Geographical Information Systems Theory, Applications and Management

5.2 UAV Fly Test

Testing area is located in Legnaro inside Campus of

Agripolis of University of Padova, at south-east of

city of Padova, in Italian Veneto Region. It measures

242 meters width, 508 meters height extension, and

area is twelve hectares. It was chosen because it

contains heterogeneous crops, not flat

geomorphology, and ground truth is well known. In

November 2015 eighteen ground control points

(GCP) were put in the area, and the coordinates were

collected with GPS in Real Time Kinematic. The

root mean square error of measures is between 0.008

and 0.011 centimetres. Then the area was flown by

Agency of Veneto Region for payment in

Agriculture (AVEPA), with eBee UAV, Figure 4.

Figure 4: Position of the GCP in the testing area.

Ebee UAV was equipped with three Sensefly

cameras, Red Green Blue (RGB), Near Infrared

(NIR) and multispectral. RGB camera model was

WX. NIR camera model was S110 NIR with three

bands, green with central wavelength at 550 nm, red

with central wavelength at 625 nm, near infrared

with central wavelength at 850 nm. Multispectral

camera model was multiSPEC 4C with four bands,

green with central wavelength at 550 nm, red with

central wavelength at 660 nm, Red edge with central

wavelength at 735 nm, near infrared with central

wavelength at 790 nm. RGB and NIR camera images

had pixel size of 4.5 centimeters. Multispectral camera

images had pixel size of 18 centimeters. All images

were processed with photogrammetric software Agisoft

Photoscan, and then orthorectified. The error calculated

by Photoscan is 0.396 pixel (Table 1). Single band

orthomosaic were exported as GeoTIFF file.

Table 1: GCP errors calculated with Photoscan.

GCP XY

error

(m)

error

(m)

Error

(m)

Proj. Error

(pix)

1 0.0198 0.0002 0.0198 86 0.3340

2 0.0291 -0.0089 0.0304 83 0.3630

3 0.0286 0.0074 0.0295 75 0.3180

4 0.0260 -0.0109 0.0281 92 0.4170

5 0.0156 0.0233 0.0280 106 0.3440

6 0.0331 -0.0307 0.0452 102 0.3180

7 0.0498 -0.0051 0.0500 91 0.4220

8 0.0237 -0.0394 0.0460 109 0.3430

9 0.0193 -0.0069 0.0205 91 0.5160

10 0.0324 0.0783 0.0848 81 0.3980

11 0.0115 -0.0082 0.0141 85 0.4350

12 0.0316 0.0116 0.0336 88 0.3780

13 0.0111 -0.0116 0.0160 116 0.4260

14 0.0480 0.0392 0.0620 84 0.3470

15 0.0267 -0.0550 0.0611 100 0.4100

16 0.0562 0.0613 0.0832 89 0.4590

17 0.0467 -0.0005 0.0467 78 0.3540

18 0.0300 -0.0135 0.0329 45 0.5410

Tot 0.0198 0.0002 0.0456 0.3960

5.3 Classification

In the previous step we have orthorectified nine

bands. Then we have selected seven bands in order to

have continuous spectrum coverage without overlaps

(Table 2), and we uploaded the images in QGis.

Table 2: Bands selected for the classification test.

Band Camera Wavelength nm

Blue RGB 450

Green multiSPEC 4C 550

Red NIR 625

Red multiSPEC 4C 660

Red

Edge

multiSPEC 4C 735

Nir multiSPEC 4C 790

Nir NIR 850

UAV Technology Integration for Remote Sensing Image Analysis

We used Semi-Automatic Classification Plugin

Version 4.9. To test two algorithms, Minimum

Distance and Maximum Likelihood, we chose four

classes that are, 1 - urban, 2 - ploughed land, 3-

crops and 4- vegetation, and we identified regions of

interest (ROI) using the specific tool. Minimum

Distance classification is shown in Figure 5.

Figure 5: Classification with Minimum Distance

algorithm.

In order to asses the classification accuracy a

comparison ROI was created and it was used to

calculate error matrix (Table 3) and Kappa index.

Kappa index for Minimum Distance classfication is

0.64. Then we applied the same procedure for

Maximum Likelihood algorithm. Figure 6 shows the

classification map, and Table 4 shows error matrix.

Kappa index for Maximum likelihood is 0.92.

Figure 6: Classification with Maximum likelihood

algorithm.

DCGISTAM 2016 - Doctoral Consortium on Geographical Information Systems Theory, Applications and Management

Table 3: Error matrix for Minimum Distance

classification.

Reference

Class 1 2 3 4 Tot.

1 32718 4313 0 479 37510

2 10779 389257 2276 0 402312

3 877 53722 32506 29239 116344

4 0 0 6793 50438 57231

Tot. 44374 447292 41575 80156 613397

Table 4: Error matrix for Maximum likelihood

classification.

Reference

Class 1 2 3 4 Tot.

1 42746 306 0 0 43052

2 1438 442206 6342 0 449986

3 0 4610 27994 842 33446

4 190 170 7239 79314 86913

Tot. 44374 447292 41575 80156 613397

5.4 Conclusion

This work is preliminary analysis to explore the

potentiality of Satellite images coupled with UAV

images. We have defined a procedure for integrating

satellite and UAV data, and we have tested two

classic remote sensing algorithms, Minimum

distance and Maximum likelihood with UAV data.

Images were collected with eBee drone, using with

different sensors. Then they were orthorectified and

classified in four classes, urban, ploughed land,

crops and vegetation. The accuracy of classification

was estimated with K index. Maximum Likelihood

got 0.91, while Minimum Distance got 0.64. In

literature Maximum Likelihood algorithm is one of

the most popular classifiers used in remote sensing

from satellite. In this preliminary test with images

from drone, Maximum Likelihood algorithm gives

better result than Minimum Distance classifier. In

Figure 7 we can see two comparisons between the

algorithms and ground truth. On left images,

Minimum Distance algorithm classifies trees as

buildings, while Maximum Likelihood assigns trees

to vegetation class. On right images Minimum

Distance Algorithm produces confused

classification. Maximum Likelihood is more precise,

but it mixes crops and vegetation.

Figure 7: Comparison between classifications obtained

two Minimum Distance and Maximum Likelihood

algorithms.

6 STAGE OF THE RESEARCH

At the moment the research is at initial phase as the

project started a few months ago. In this contribution

we want to present the research question and the

methods which will be tested in the project.

REFERENCES

Baumann, P., 2014. Spatio-Temporal Big Data the

rasdaman approach, p.32. Available at:

UAV Technology Integration for Remote Sensing Image Analysis

http://2014.ogrs-community.org/2014_workshops/Ras

daman/BigDataRasdamanWorkshop.pdf.

Berni, J., Zarco-Tejada, P. J., Suarez, L., Fereres, E., 2009.

Thermal and narrowband multispectral remote sensing

for vegetation monitoring from an unmanned aerial

vehicle. IEEE Transactions on Geoscience and

Remote Sensing, 47, pp.722–738.

Breiman, L., 2001. Random forests. Machine Learning,

45, 5–32. doi:10.1023/A:1010933404324.

Coppa, U., Guarnieri, A., Pirotti, F., and Vettore, A., 2009.

Accuracy enhancement of unmanned helicopter

positioning with low-cost system. Applied Geomatics,

1(3), pp.85–95.

Dalponte, M., Bruzzone, L., Gianelle, D., Member, S.,

2008. Fusion of Hyperspectral and LIDAR Remote

Sensing Data for Classification of Complex Forest

Areas. IEEE Transactions on Geoscience and Remote

Sensing, 46(5), pp.1416–1427.

Grenzdörffer, G., Engel, A., Teichert, B., 2008. The

photogrammetric potential of low-cost UAVs in

forestry and agriculture. International Archives of

Photogrammetry Remote Sensing and Spatial

Information Sciences, 1, pp.1207–1213. Available at:

http://www.isprs.org/proceedings/XXXVII/congress/1

_pdf/206.pdf.

Haarbrink, R., Eisenbeiss, H., 2008. Accurate DSM

production from unmanned helicopter systems.

International Archives of Photogrammetry Remote

Sensing and Spatial Information Sciences, Vol.

XXXVI, pp.1259–1264. Available at: http://www.isp

rs.org/proceedings/XXXVII/congress/1_pdf/214.pdf.

Herwitz, S.R., Johnson, L.F., Dunagan, S.E., Higgins,

R.G., Sullivan, D. V., Zheng, J., Lobitz, B.M., Leung,

J.G., Gallmeyer, B. a., Aoyagi, M., Slye, R.E., Brass,

J. a., 2004. Imaging from an unmanned aerial vehicle:

Agricultural surveillance and decision support.

Computers and Electronics in Agriculture, 44(1),

pp.49–61.

Honkavaara, E., Kaivosoja, J., Mäkynen, J., Pellikka, I.,

Pesonen, L., Saari, H., Salo, H., Hakala, T., Marklelin,

L., Rosnell, T., 2012. Hyperspectral Reflectance

Signatures and Point Clouds for Precision Agriculture

By Light Weight UAV Imaging System. ISPRS

Annals of Photogrammetry, Remote Sensing and

Spatial Information Sciences, I-7(September), pp.353–

358.

Hunt, E.R., Dean Hively, W., Fujikawa, S.J., Linden, D.S.,

Daughtry, C.S.T., McCarty, G.W., 2010. Acquisition

of NIR-green-blue digital photographs from unmanned

aircraft for crop monitoring. Remote Sensing, 2(1),

pp.290–305.

Kruse, F. A., Lefkoff, A.B., Boardman, K.B. Shapiro,

A.T., Barloon, P.J., Goetz, A.F.H., 1993. The Spectral

Image Processing System (SIPS) - Interactive

Visualization and Analysis of Imaging Spectrometer

Data. Remote Sensing of Environment, 44, pp.145–

163. doi:10.1016/0034-4257(93)90013-N.

Lelong, C.C.D., Burger, P., Jubelin, G., Roux, B., Labbé,

S., Baret, F., 2008. Assessment of unmanned aerial

vehicles imagery for quantitative monitoring of wheat

crop in small plots. Sensors, 8(5), pp.3557–3585.

Lin, F.-C., Chung, L.-K., Ku, W.-Y., Chu, L.-R., Chou,

T.-Y., 2013. The Framework of Cloud Computing

Platform for Massive Remote Sensing Images.

Advanced Information Networking and Applications

(AINA), 2013 IEEE 27th International Conference on

pp.621–628. Available at: http://ieeexplore.ieee.org/xp

ls/icp.jsp?arnumber=6531812.

Marsden, T., 2010. Mobilizing the regional eco-economy:

evolving webs of agri-food and rural development in

the UK. Cambridge Journal of Regions, Economy and

Society, 3 (2 ), pp.225–244. Available at: http://cjres.

oxfordjournals.org/content/3/2/225.abstract.

Melgani, F. and Bruzzone, L., 2004. Classification of

hyperspectral remote sensing images with support

vector machines. Geoscience and Remote Sensing,

IEEE Transactions on, 42(8), pp.1778–1790.

Peña, J.M., Torres-Sánchez, J., de Castro, A.I., Kelly, M.,

López-Granados, F., 2013. Weed Mapping in Early-

Season Maize Fields Using Object-Based Analysis of

Unmanned Aerial Vehicle (UAV) Images. PLoS ONE,

8(10), pp.1–11.

Piragnolo, M., Pirotti, F., Vettore, A., Salogni, G., 2014a.

ANTHROPIC RISK ASSESSMENT ON

BIODIVERSITY. ISPRS - International Archives of

the Photogrammetry, Remote Sensing and Spatial

Information Sciences, XL-5/W3, pp.21–26.

doi:10.5194/isprsarchives-XL-5-W3-21-2013.

Piragnolo, M., Pirotti, F., Guarnieri, A., Vettore, A.,

Salogni, G., 2014b. Geo-Spatial Support for

Assessment of Anthropic Impact on Biodiversity.

ISPRS International Journal of Geo-Information, 3,

pp.599–618. doi:10.3390/ijgi3020599.

Remondino, F., Barazzetti, L., Nex, F., Scaioni, M.,

Sarazzi, D., 2011. UAV photgrammetry for mapping

and 3D modeling current status and future

perspectives. International Archives of the

Photogrammetry, Remote Sensing and Spatial

Information Sciences, XXXVIII (September), pp.14–

16.

Remondino, F., Pizzo, S. Del, 2012. Low-cost and open-

source solutions for automated image orientation–a

critical overview. Progress in Cultural Heritage

Preservation, pp.40–54. Available at: http://link.spring

er.com/chapter/10.1007/978-3-642-34234-9_5.

Richards, J. A., Jia, X., 2006. Remote Sensing Digital

Image Analysis: An Introduction. Berlin, Germany:

Springer.

Singh, S., Singh, N., 2012. Big Data analytics. 2012

International Conference on Communication,

Information and Computing Technology (ICCICT),

pp.1–4. Available at: http://ieeexplore.ieee.org/lpdocs

/epic03/wrapper.htm?arnumber=6398180.

Sona, G., Pinto, L., Pagliari, D., Passoni, D., Gini, R.,

2014. Experimental analysis of different software

packages for orientation and digital surface modelling

from UAV images. Earth Science Informatics, 7(2),

pp.97–107.

DCGISTAM 2016 - Doctoral Consortium on Geographical Information Systems Theory, Applications and Management

Torres-Sánchez, J., Peña, J.M., de Castro, a. I., López-

Granados, F., 2014. Multi-temporal mapping of the

vegetation fraction in early-season wheat fields using

images from UAV. Computers and Electronics in

Agriculture, 103, pp.104–113.

Van Asselen, Sanneke, and Peter H. Verburg. 2013. Land

Cover Change or Land-Use Intensification: Simulating

Land System Change with a Global-Scale Land

Change Model. Global Change Biology, 19, pp.3648–

3667. doi:10.1111/gcb.12331.

Van Eupen, M., Metzger, M.J., Pérez-Soba, M., Verburg,

P.H., van Doorn, a., Bunce, R.G.H., 2012. A rural

typology for strategic European policies. Land Use

Policy, 29(3), pp.473–482. Available at:

http://dx.doi.org/10.1016/j.landusepol.2011.07.007.

Zaslavsky, A., Perera, C., Georgakopoulos, D., 2013.

Sensing as a Service and Big Data, in: arXiv Preprint

arXiv:1301.0159. Bangalore.

Zhao, W., Ma, H. and He, Q., 2009. Parallel K-Means

Clustering Based on Map Reduce. Cloud Computing,

5931, pp.674–679. Available at: http://link.springe

r.com/chapter/10.1007/978-3-642-10665-1_71.

UAV Technology Integration for Remote Sensing Image Analysis