Knowledge Models and Image Processing Analysis in Remote

Sensing: Examples of Yakutsk (Russia) and Kaunas (Lithuania)

Sébastien Gadal

and Walid Ouerghemmi

Aix-Marseille Univ, CNRS, ESPACE UMR 7300, Univ Nice Sophia Antipolis, Avignon Univ,

13545 Aix-en-Provence, France

Keywords: Geographic Knowledge, Temporal Analysis, Geographic Ontologies, Spectral Databases, Spatial Modelling,

Simulation, Artificial Intelligence, Image Processing, Remote Sensing.

Abstract: The use of geographic knowledge in remote sensing constitutes one of the fundamental base of the

methodologies of image processing. Image processing, image analysis, and oriented-object recognition are

based on the geographic knowledge. More specifically, the large panel of supervised classifications methods

are one of the main example where geographic knowledge is necessary for both algorithms training and results

validation. Recently, with the coming back of the artificial intelligence (AI) wave, it appears that a large

spectrum of usually employed methodologies in remote sensing and image processing, are one of the main

drivers of AI: machine learning, deep learning are the most effective’s examples. As well as many based

processing algorithms like the Support Vector Machine (SVM) or the Random Forest (RF). However, despite

the constant performances of the methods of calculus; the geographic knowledge’s determines the accuracy

of recognition and classification in image processing and spatial modelling generated. In regard of the fast

seasonal and annual landscape changes in the Arctic climate, and complex urban structures, Yakutsk and

Kaunas cities contribute to the reflexion.

1 INTRODUCTION

The use of geographic knowledge in remote sensing

constitutes one of the fundamental base of the

methodologies of image processing. The efficiency of

the maps generated, oriented objects-recognition,

geographic object extractions and image analysis

depend on the expert knowledge acquired by the

analyst or/and the databases created: oriented-object

spectral databases, object databases, maps, etc (e.g.

Blaschke et al, 2008; Mennis and Guo, 2009). The

accuracy of the model of knowledge implemented in

image processing determines the capacity of

recognition, as well as the results obtained establishes

the quality of the spatial modelling of the landscape

changes for example. Image processing, spatial

modelling and geographic knowledge are

interdependent. The performances of algorithms

depends on the databases characterising the object to

identify (spectral, morphological, etc.), which are

implemented as learning bases and include expert

https://orcid.org/0000-0002-6472-9955

https://orcid.org/0000-0002-2119-7290

knowledge of the geographer (e.g. Huang and Jensen,

1997). These databases are used more and more for

image analysis and interpretation (e.g. Durand et al,

2017; Weber et al, 2018). Algorithms used in

artificial intelligence (AI) applied to remote sensing

applications and more specifically to the field of

geography process analysis, are mostly related to the

image processing methods (Random Forest (RF),

Support Vector Machine (SVM), neural network

(NN), etc.) (e.g. Vopham et al, 2018). Therefore,

there is a strong link between AI, image processing

and analysis, especially with the last advances in

terms of image acquisition and sensors development,

involving an increase in terms of data complexity.

The approaches of spatial and territorial

modelling and simulation base the processing on the

panel of algorithms like the SMA, Markov chains,

cellular automats, etc.). Algorithms used for the

calculus as well as the methods in image processing

remote sensing or in geographic modelling are not

new. The performances of the infrastructures and

282

Gadal, S. and Ouerghemmi, W.

Knowledge Models and Image Processing Analysis in Remote Sensing: Examples of Yakutsk (Russia) and Kaunas (Lithuania).

DOI: 10.5220/0007752202820288

In Proceedings of the 5th International Conference on Geographical Information Systems Theory, Applications and Management (GISTAM 2019), pages 282-288

ISBN: 978-989-758-371-1

architectures of calculus, combined to the

implementations of massive data, and to the

important progress on the formalisation of the

knowledge in databases, are the last advances in AI

and big data fields (e.g. Chi et al, 2016).

One of the issue focus on the object ontologies.

Despise the lake of researches these since the 2000’s,

knowledge models and spatial ontologies are one way

to follow and reinvestigate (e.g. Breen, 2002). The

formalisation of the geographic and social

environments is generally structured on the expert

knowledge (field missions, perception, etc.), the

geographic databases (maps, census, etc.) organised

and structured in GIS and databases (e.g.

Ouerghemmi et al, 2017; Gadal and Ouerghemmi,

2016). Therefore, the processing approaches based on

object ontologies (e.g. spectral, morphological,

geometric characterisations), applies to the

heterogeneity and complexity of urban objects (i.e.

heterogeneous measurements). These objects are

related to: (a) the form aspects (e.g. metric,

geometric, morphology, textures, topology), (b) the

biophysical characteristics (e.g. spectral signatures,

spectral databases), and (c) the semantics (e.g.

geolinguistics modelling based on identification,

description of landscapes, human and societal uses,

territorial knowledge of inhabitants, perception and

conceptualisation of their home space/territory).

2 ISSUES IN URBAN REMOTE

SENSING

2.1 Multiplication of Earth

Observation Systems and

Geographic Databases

The fast grow of the Earth observation sensors

combines the flood of remote sensing, geographic,

environmental, or socioeconomic data. Flood and

massive data characterise the present of the remote

sensing: up to a few thousand spectral bands in

hyperspectral imagery, high temporal repetitiveness

of images taken (Meteosat 3

generation, Sentinel 1

and 2, continuous covering on zone with UAV, etc.).

Storage and heterogeneous massive data processing

as well as the capacities of mutual enrichment by

multi-source processing revealed new

methodological challenges today. If the methods of

fusion, machine learning, and deep learning are today

established, the complexification of the knowledge

structure including a large panel of heterogeneous

data is still challenging.

The developed knowledge models are usually

structured in accordance to data and knowledge

integration in geographic information systems (GIS).

Image-processing methods will rely on artificial

intelligence for the extraction of information and

knowledge in accordance to available geographic

ontologies. The proposed approach is based on the

joint use of knowledge databases and artificial

intelligence for image processing and simulation

(Fig.1).

The increase of the massive data with the use of

hyperspectral data, high resolution centimetres

satellite, airborne images and high temporal

repetitiveness, requires automation of the processing

sequences. For that purpose, artificial intelligence

data processing methods (e.g. machine learning, deep

learning, etc.) are increasingly used in image

processing field.

Figure 1: The use of artificial intelligence based methods

for image processing and management of massive data.

The need of implementing different types of

knowledge to optimise the objects recognition

process (e.g. training, learning and validation of the

results) and to model the geographic and urban

changes constitutes the second methodological block

of the developed approach. For example, based-

knowledge morphological databases using specific

geometric rules were used to recognise and extract

urban objects (Fig. 2).

Figure 2: Implementation of knowledge model for image

processing and objects recognition.

Knowledge Models and Image Processing Analysis in Remote Sensing: Examples of Yakutsk (Russia) and Kaunas (Lithuania)

283

The different sequence of image processing and

modelling are related to the knowledge database for

training the models, creating rules, and validating and

interpreting the results.

2.2 Contribution of Knowledge

Databases in Remote Sensing

2.2.1 Knowledge Database Approaches

Result from the remote sensing image processing

depends of the geographic knowledge formalised in

databases or in ontological rules. Results of

classification (e.g. land cover maps), segmentation or

simulation will have no meaning without expert

knowledge that will serve to both enhancement of the

model training and improvement of the validation

results. The classification accuracy of trees species

identification made by airborne hyperspectral sensors

was enhanced by the expert knowledge training

compared to the learning made from spectral library

of in-situ measurements of the same species (i.e.

increase of Overall accuracy (O.A.) of up to 20%)

(Fig.3) (Ouerghemmi et al, 2018a), the obtained

O.A.(s) were < 25% for 11 vegetation species

identification using the Spectral Angle Mapper

(SAM) classifier. In (Ouerghemmi et al, 2018b), the

usefulness of an expert knowledge for vegetation

mapping was consolidated, and the mapping

performance was enhanced when using a machine

learning classifier jointly with the knowledge

database, O.A. was up to 46% for 8 vegetation species

identification.

Figure 3: Vegetation species mapping over Kaunas city, at

left mapping by spectral library, at right mapping by

validation points of the expert knowledge.

In (Ouerghemmi et al, 2017; Gadal and

Ouerghemmi, 2016), the use of morphological

database for urban objects recognition in addition to

spectral oriented-object database, allows to solve the

misclassifications occurred when spectral database

are only using in the classification (Fig.4). The use of

morphometric databases provides the ability to detect

the socio-economic functions of buildings. The

implementation of other types of knowledge, for

example the ontologies, is helpful in image

processing for mapping and modelling the geographic

changes. Ontological knowledge can be introduced in

the form of rules, or can be used to build specific

simulation scenarios and thus to make the simulation

more reliable. Expert knowledge integration

including a large panel of thematic must be explored

for the future development in image processing

approaches. As well, today, the main issues concerns

essentially the standardisation of the data and

knowledge oriented object databases like the morpho-

spectral databases (e.g. Weber et al, 2018). The

standardisation is one of the key-issue to implement

and integrate the knowledge models in the methods

of image recognition and geographic processing.

Figure 4: (a) Urban objects identification by spectral

library, (b) morphological rectification for

misclassifications compensation.

The introduction of other types of knowledge (e.g.

ontological), could be very helpful in image

processing in terms of mapping and modelling,

indeed, ontological knowledge could be introduced in

the form of rules, or could be used to build specific

simulation scenarios and thus to make the simulation

more reliable. In that context, expert knowledge of

heterogeneous fields can be taken into consideration

for future development of image processing

methodologies. The issues will concern essentially

the management and the standardisation of these data

to make the integration in the processing frame easier

and simpler for the end-user (e.g. Weber et al, 2018).

(a)

(b)

GISTAM 2019 - 5th International Conference on Geographical Information Systems Theory, Applications and Management

284

2.2.2 Artificial Intelligence Processing:

Rules, Training, Databases Building,

and Ontological Recognition

Artificial intelligence is a rising branch of interest,

which is used more and more in remote sensing field.

The goal is to take advantage of the available

knowledge database in the processing step; the

progress of the operations will follow certain

ontologies. This will ensure the system autonomy

thanks to the injection of intelligence into the process.

Autonomous and intelligent systems permit to solve

the different problems of image mapping, objects

recognition, and simulation of different

environmental phenomenon’s. In parallel, the

knowledge database should follow certain structure

and certain standardization directive, so that the

different sources will be converted into ready to use

format for the different algorithms and models.

3 APPLICATIONS OF

KNOWLEDGE MODELS IN

POST-SOVIET URBAN

ENVIRONEMENTS (SOME

RESULTS)

3.1 Kaunas: Morpho-Spectral

Database Learning for the

Recognition of Built-Ups and

Urban Vegetation

In the following section, we present some application

related to the use of different types of knowledge

models in urban remote sensing. The socio-economic

building characterisation, building identification, and

urban vegetation detection in two post-soviet cities:

Kaunas in Lithuania (Baltic) and Yakutsk in the

Eastern Siberia characterised by the Arctic climate.

The city of Kaunas is characterised by a large

types of architectures and urban morphologies related

to the different historical periods since the 15 century.

Coexisting medieval, modern, Russian 19e century,

Art modern, soviet and postmodern architectures and

urban structures. The recognition and characterisation

of the built-ups combines two knowledge’s database:

the spectral database was used for urban classes’

identification, O.A. was up to 82% (Fig. 5), and

geometric rules related to the morphology of the

buildings for urban structures characterisation (Fig.

6). Application are made with an airborne VNIR

Rikola hyperspectral sensor with a GSD of 70 cm

with 16 spectral bands.

Bitumen

Tile

Paint steel in

red

Light paint

steel

Dark paint

steel

Paving

Turf

Coniferous

Deciduous

Figure 5: Recognition based on the spectral database of

urban materials (roof, human-made, vegetation).

Medieval

(Kaunas Castle)

20e century (Art

Modern)

Gothic (end of the

15e century)

16e century

Modern period

(18e-19e century)

Figure 6: Recognition of medieval, reconnaissance and

modern buildings based on the geometric database.

The methodology applied integrates first, a

threshold based segmentation according to the roof

radiometry of the built-ups, and next geometric rules

are used for the extraction of specific buildings or

Knowledge Models and Image Processing Analysis in Remote Sensing: Examples of Yakutsk (Russia) and Kaunas (Lithuania)

285

urban structures. Several functions were identified in

this context, which are individual houses,

administrative building and cultural structures (Gadal

and Ouerghemmi, 2016) (Fig. 6). The proposed

methodology for urban objects extraction and

characterisation was inspired by the research carried

out in (Gadal and Ouerghemmi, 2015), from which,

an airborne image over Toulouse (France) was used

for the study. First, a spectral classification was

carried to extract the urban frame using a SAM

classifier, and second, textural attributes were used at

different bands to extract three socio-economic

classes, which are modern building, mean historical

buildings, and mean residential buildings.

The figure 7 shows examples of the identification

of urban structures using 4 different rules based on

simple and complex geometric attributes, which are

respectively pixel count, [elongation + area],

[elongation + compactness] and [Minimum Bounding

Rectangle (MBR)-fill Ratio + Area] (i.e. respect. Fig

2.a, 2.b, 2.c and 2.d). First rule allows detecting urban

objects depending only on the attribute pixels count,

which could be useful for detecting easily and

roughly some specific structures in terms of size.

Second rule allows extracting the urban structures of

Kaunas city using both elongation and area attributes.

Third rule allows detecting compact and elongated

building corresponding to administrative and

residential building using attributes compactness and

elongation. Last rule permits the identification of

specific morphologies using Objects MBR attribute

and area attribute, most of the identified building

corresponds to religious structures.

Figure 7: Recognition of urban structures using geometric

rules (from the geometric urban oriented-object database).

3.2 Yakutsk: Multi-Temporal Urban

Landscape Changes and

Morpho-Spectral Database

Learning

This section presents some results of multi-temporal

satellite imagery use for environmental change

detection, urban frame identification, and

morphological processing, in this context, a Sentinel

2 multi-temporal images were used over Yakutsk for

inter-annual and inter-annual change detection

estimation of built-up areas within the city (Gadal and

Ouerghemmi, 2019). The used knowledge model

include multi-temporal Sentinel-2A satellite imagery

of several dates, high resolution SPOT 6 satellite

imagery, biophysical indices, validation points of

Google Street + field survey campaign, and

morphological measurements of the different urban

objects. Built-up areas were first extracted using the

merging of different biophysical indices (Fig. 8.a),

with O.A.(s) from 67% to 75%. In second step, built-

up areas change was identified for two dates using

Sentinel-2A imagery, the study revealed an increase

of about 12% in terms of built-ups between 2 years

interval. And finally morphological measurements of

the extracted objects permits to identify main socio-

economic classes of the city (Fig. 8.b), high resolution

SPOT 6 imagery showed encouraging results in terms

of morphological modeling of urban objects. This last

example well illustrates the usefulness of a hybrid

knowledge model for objects identification and

characterisation, nevertheless the artificial

intelligence part need to be further explored to better

automate the processing sequence and to enhance the

identification performance.

The perspectives behind the proposed

methodology is focused on two main axis which are

(a) the creation of a heterogeneous and standardized

knowledge model for application to remote sensing

issues, and (b) the development of a simulation

strategy for urban sprawl based on the use of

knowledge model. Indeed, the goal is to take

advantage of many available heterogeneous sources

of information in the simulation process. The

usefulness of such knowledge model could be found

at different levels of the simulation sequence, it

permits to build coherent simulation scenarios,

includes several biophysical indices (e.g. spectral

signatures, spectral indices) in the simulation model,

and bring some new features to the model like

ontologies and topologies related to the our specific

context and to our specific study zone. The

availability of such knowledge model can enhance the

model coherency and enhance the simulation

GISTAM 2019 - 5th International Conference on Geographical Information Systems Theory, Applications and Management

286

accuracy. Several studies have been already made in

the literature; nevertheless, the used knowledge

model was limited in terms of components.

Figure 8: Upper figure: urban frame identification over

Yakutsk city, lower figure: urban function extraction by

morphological attributes.

4 CONCLUSIONS

In this study, we presented an original

methodological scheme based on the joint use of a

hybrid knowledge model and artificial intelligence

for remote sensing application purpose. Several

application were already made using simple

knowledge model composed of biophysical or

morphological features or both at best. The idea

behind the methodology is to enrich the knowledge

model with several sources of information, which

could be very different at first sight, but will better

model the problem. The main issue will be related to

the standardisation of such sources of information, so

that the joint use of all the available sources could be

easily considered by the end-user.

The field of application related to such knowledge

model could be varied including objects mapping by

remote sensing imagery, objects characterisation and

identification, simulation of different environmental

phenomenon (e.g. urban sprawl, urban degrowth,

change detection, environmental phenomenon’s

management). Such knowledge model, will requires

the introduction of intelligence in both the structuring

procedure of the model and in the processing

sequence, including machine learning, deep learning

and neuronal networks methods. The joint use of

hybrid knowledge model coupled to the development

of intelligent rules and powerful training models will

contribute to the enhancement of image processing

and modelling methods and will constitute a new

exploration field.

ACKNOWLEDGEMENTS

French National Agency through Polar Urban Centers

(PUR, grant number ANR-15-CE22-0006) and

through Hyperspectral Imagery for Urban and

Environmental planning (HYEP, grant number ANR-

14-CE22-0016) supported this research.

SPOT 6 and Sentinel 2 data got the support of the

EQUIPEX GEOSUD (ANR-10-EQPX-20).

REFERENCES

Breen, C., Khan, L., and Ponnusamy. A., 2002. Image

Classification Using Neural Networks and Ontologies.

In Proc. 13th International Workshop on Database and

Expert Systems Applications, co-located with DEXA

2002, pages 98–102, Aix en-Provence, France.

Blaschke, T, Lang, S., and Hay, G.J., 2008. Object-Based

Image Analysis: Spatial Concepts for Knowledge-

Driven Remote Sensing Applications, New York, NY,

USA:Springer-Verlag.

Chi, M., Plaza, A., Benediktsson, J.A., Sun, Z. Shen, J. and

Zhu, Y., 2016. Big Data for Remote Sensing:

Challenges and Opportunities. Proceedings of the

IEEE, vol. 104, no. 11, pp. 2207-2219.

Durand, N. et al, 2007. Ontology-based object recognition

for remote sensing image interpretation, In 19th IEEE

international conference on tools with artificial

intelligence (ictai 2007), Patras, Greece, pp. 472-479.

Huang, X., and Jensen, J. R., 1997. A machine-learning

approach to automated knowledge-base building for

remote sensing image analysis with GIS data.

Photogrammetric Engineering and Remote Sensing, 63,

1185–1194.

Gadal, S., 2012. Geographic Space Ontology, Locus-

Object, and Spatial Data Representation Semantic

Theory. Universal Ontology of Geographic Space:

Semantic Enrichment for Spatial Data, IGI Global,

pp.28-52, 9786-1-4666-032-27-1.

Gadal, S., and Ouerghemmi, W., 2015. Morpho-Spectral

Recognition of Dense Urban Objects by Hyperspectral

Imagery. In ISPRS Geospatial week, 28 Sept – 02 Oct,

La Grande-Motte, France.

Gadal, S., and Ouerghemmi. W., 2016. Morpho-spectral

objects classification by hyperspectral airborne

imagery. In Proceedings of the 8th Workshop on

Hyperspectral Image and Signal Processing: Evolution

in Remote Sensing (WHISPERS), Los Angeles, CA.

Gadal, S., and Ouerghemmi. W., 2018. Multi-Level

Morphometric Characterisation of Built-ups in Siberian

Knowledge Models and Image Processing Analysis in Remote Sensing: Examples of Yakutsk (Russia) and Kaunas (Lithuania)

287

Sub-Arctic Urban Area: Yakutsk. In GEOBIA 2018:

From pixels to ecosystems and global sustainability,

Montpellier, France.

Gadal, S.; Ouerghemmi, W., 2019. Multi-Level

Morphometric Characterization of Built-up Areas and

Change Detection in Siberian Sub-Arctic Urban Area:

Yakutsk. ISPRS Int. J. Geo-Inf., 8, 129.

Mennis, J., and Guo, D., 2009. Spatial data mining and

geographic knowledge discovery—An introduction,

Computers, Environment and Urban Systems, Volume

33, Issue 6, 2009, Pages 403-408, ISSN 0198-9715.

Mozgeris, G. Juodkienė, V., Jonikavičius, D., Straigytė, L.,

Gadal, S. et al., 2018. Ultra-Light Aircraft Based

Hyperspectral and Colour-Infrared Imaging to Identify

Deciduous Tree Species in an Urban Environment.

Remote Sensing, MDPI, 2018, 10 (10).

Ouerghemmi, W., Gadal, S., Mozgeris, G., Jonikavičius,

D., and Weber, C., 2017. Urban objects classification

by spectral library: feasibility and applications. In 2017

Joint Urban Remote Sensing Event (JURSE), Dubai,

pp. 1- 4.

Ouerghemmi, W., Gadal, S., Mozgeris. G., 2018a. Urban

Vegetation Mapping using Hyperspectral Imagery and

Spectral Library. In IEEE International Geoscience and

Remote Sensing Symposium (IGARSS) 2018, Valencia,

Spain. pp.1632-1635.

Ouerghemmi, W., Gadal, S., Mozgeris, G., and

Jonikavičius, D., 2018b. Urban vegetation mapping by

airborne hyperspectral imagery: feasability and

limitations. In WHISPER 2018: 9th Workshop on

Hyperspectral Image and Signal Processing: Evolution

in Remote Sensing, Amsterdam, Netherlands. pp.245-

249.

VoPham, T., Hart, J. E., Laden, F., and Chiang, Y. Y.

(2018). Emerging trends in geospatial artificial

intelligence (geoAI): potential applications for

environmental epidemiology. Environmental health: a

global access science source, 17(1), 40.

Weber, C., Aguejdad, R., Briottet, X., Aval, J., Fabre, et al.,

2018. Hyperspectral Imagery for Environmental

Planning. In Proc. of the IEEE International

Geoscience and Remote Sensing Symposium (IGARSS),

Valencia (E).

Zakharov, M., Filippova, V., Savvinova, A., Pestereva, K.,

and Gadal, S., 2017. Recognition of Landscape

Structures by Toponymical Map and Remote Sensing

Analysis (Khangalassky district, Yakutia). In IX Arctic

Congress in Social Sciences (ICASS IX): "People and

Place", Jun 2017, Umea, Sweden.

GISTAM 2019 - 5th International Conference on Geographical Information Systems Theory, Applications and Management

288