Semantic Segmentation of Crops via Hyperspectral PRISMA

Satellite Images

Manilo Monaco

, Angela Sileo

, Diana Orlandi

, Maria Libera Battagliere

Laura Candela

, Mario G.C.A. Cimino

, Gaetano A. Vivaldi

and Vincenzo Giannico

Italian Space Agency, Matera/Rome, Italy

The Revenue Agency, Matera, Italy

Dept. of Information Engineering, University of Pisa, Pisa, Italy

Dept. of Soil, Plant and Food Sciences, University of Bari “Aldo Moro”, Bari, Italy

laura.candela@asi.it, mario.cimino@unipi.it, gaetano.vivaldi@uniba.it, vincenzo.giannico@uniba.it

Keywords: Crop-Type Mapping, PRISMA, Satellite Hyperspectral Imagery, Convolutional Neural Network.

Abstract: Data from hyperspectral remote sensing are promising to extract and classify crop characteristics, because it

provides accurate and continuous spectral signatures of crops. This paper focuses on data acquired by

PRISMA, a high-resolution hyperspectral imaging satellite. Due to this large data availability, huge training

datasets can be built to feed modern deep learning algorithms. This paper shows a spectral-temporal data

processing based on random forest to perform feature selection, and on two-dimensional convolutional neural

network to carry out classification of crops, exploiting variations in respective phenological phases during the

annual life cycle. The proposed solution is described via a pilot case study, involving a field farmed with olive

groves and vineyards in Apulia, Italy. Moreover, one-dimensional convolutional neural networks are used to

compare classification accuracies. Early results are promising with respect to the literature.

1 INTRODUCTION

Agriculture represents an ideal application domain

for the use of hyperspectral imaging technology due

to several concurrent factors such as high biological

complexity, wide variety of plant growth conditions,

climatic conditions, soil types, and crops.

Data from hyperspectral remote sensing can be

used to extract and classify crop characteristics.

Typically, since it is unstructured data, convolutional

architectures can handle this application effectively

because they work on both the spatial and spectral

dimensions of the data (Bhosle, 2022).

Thanks to its high spectral resolution and the

resulting sensitivity to subtle spectral variations

between ground objects, hyperspectral data have been

used with excellent results to identify crop types and

varieties in order to obtain spatial distribution maps

https://orcid.org/0000-0001-5122-4520

https://orcid.org/0000-0002-4272-7238

https://orcid.org/0000-0002-1031-1959

https://orcid.org/0000-0002-5798-2573

https://orcid.org/0000-0002-9907-3730

and to acquire information on the structural,

biochemical and physiological properties of plants.

However, it should be emphasized that hyperspectral

data with medium to high spatial resolution, which

are typically used for accurate crop classification, are

unsuitable for use at regional- or country-level for this

application purpose, precisely because of their high

spectral and spatial dimensionalities and the resulting

excessive computational workload required.

Therefore, in the near future the identification of

both dimensionality reduction strategies and

methodologies that make use of classifiers that can

speed up "near real-time" processing of hyperspectral

data will be of critical importance for solving

classification problems at regional- or country-level

(Zhang, 2019).

Monaco, M., Sileo, A., Orlandi, D., Battagliere, M., Candela, L., Cimino, M., Vivaldi, G. and Giannico, V.

Semantic Segmentation of Crops via Hyperspectral PRISMA Satellite Images.

DOI: 10.5220/0012705700003696

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 10th International Conference on Geographical Information Systems Theory, Applications and Management (GISTAM 2024), pages 187-194

ISBN: 978-989-758-694-1; ISSN: 2184-500X

187

1.1 Land Use and Land Cover

The task known as Land Use and Land Cover (LULC)

refers to segment, that is, to classify each pixel of a

remote sensing image, or to identify particular objects

or regions within the image. In literature, this task,

and more specifically crop segmentation, is by far the

most likely to be approached with deep learning

techniques. Land cover mapping is considered the

most important descriptor of terrestrial environmental

dynamics (Herold, 2006).

Automated methods represent the least reliable

system, but simultaneously the easiest to scale. In

summary, the research goal in land use and land cover

and in crop segmentation task is to improve the

quality of automated methods, with the aim of

equating their reliability to that of human experts.

Crops’ phenological phases are specific plant

growth stages influenced by climatic conditions.

Satellite remote sensing provides temporal series on

vegetation development with a short revisiting period.

It provides data source for monitoring vegetation

phenology, considering the full spectral information

from multispectral and hyperspectral imagery.

Analysis of satellite images is linked to the seasonal

variation of cultivated surfaces such as the beginning

of the vegetative season (green-up), the peak of the

growing season, and the end of the growing season,

i.e. the senescence, by using spectral bands or

vegetation indices that best describe these changes.

The use of phenology can be employed as a classifier

to map crops. The accuracy of classification is

affected by both the distribution of vegetation cover

within the pixel and the specificity of the spectral-

temporal signatures of different crops to be identified.

The highest accuracy for recognizing different

phenological phases is achieved by using

combinations of bands that fall within the near-

infrared (NIR), red-edge, and shortwave infrared

(SWIR) domains (Abubakar, 2023; Peng, 2023).

In most studies involving the implementation of

automated systems for mapping land use and land

cover, supervised learning models are typically used,

and there are basically three methods for constructing

training datasets: manual field inspection, visual

inspection of remote sensing images, and application

of automated methods. Manual inspections of the area

to be segmented are the most reliable, but also the

most difficult and expensive method. Using an

automated method is less reliable and cheaper, but it

greatly limits the models' ability to learn the actual

distribution of classes in the images, misleading the

models to focus on the dynamics of the automated

method used for labeling. Therefore, in some papers

the authors use a combination of the previous two

methods, instead in other papers labeling is done

completely through visual inspection of the remote

sensing images, which is still a less reliable method

than manual inspection in the field (Victor, 2022).

1.2 Deep Learning Models

In literature, various deep learning architectures are

widely used in the context of land use and land cover

tasks, as well as in agricultural crop segmentation.

Aside from the great popularity of these types of

tasks, the main reason for success lies in the relative

ease of obtaining data, which enables the construction

of sufficiently large training datasets to feed modern

deep learning algorithms. In fact, studies that focus

on these applications usually work with datasets

containing 2000 to 10000 samples that, although

small by deep learning standards, are significantly

larger than datasets used in other precision

agriculture-related applications. Typically, the best

modern deep learning methods, namely convolutional

neural networks (CNNs), recurrent neural networks

(RNNs), and Transformers (neural networks that rely

on the attention mechanism), provide more accurate

results than traditional machine learning methods,

including decision trees, support vector machines,

random forests, and multi-layer perceptrons (Victor,

2022). An early attempt of using modern deep

learning techniques for land use and land cover was

in 2017, when Kussul et al. proposed a custom

architecture consisting of five convolutional layers

and, using a dataset composed of 100000 pixels

labeled through manual field inspection, found that

convolutional neural networks working on two

dimensional axes (2DCNNs) provided more accurate

results than random forests, multi-layer perceptrons,

and convolutional neural networks working on only

one dimensional axis (1DCNNs).

In most works using two-dimensional (2D)

convolutional neural networks to segment crop types,

the authors have built custom network architectures

both in the arrangement and type of layers and in the

number of units per layer, typically preferring

shallower networks with fewer than 10 hidden layers.

In contrast, only a few authors use more common and

well-known architectures typically used in generic

computer vision tasks, such as VGG, ResNet and

UNet (Victor, 2022; Orlandi, 2023). Spatiotemporal

data processing can be accomplished through two

main classes of algorithms: the first is convolutional

neural networks working on three dimensional axes

(3DCNN) (Ji, 2018; Gallo, 2021), while the second is

the use of a convolutional network working on two

GISTAM 2024 - 10th International Conference on Geographical Information Systems Theory, Applications and Management

188

dimensional axes (2DCNN) and cascading a recurrent

neural network (RNN) with LSTM or GRU cells to

process the outputs of the former (Rußwurm, 2020;

Teimouri, 2019). However, the two previous methods

have not been compared with each other.

In generic computer vision tasks, Transformers

architectures (Dosovitskiy, 2021; Orlandi, 2022),

based on the self-attention mechanism, represent the

state of the art on major industry benchmarks, so one

might expect them to be so on satellite image

classification tasks as well. However, in works

comparing a Transformer network with other modern

deep learning methods, the Transformer network does

not always achieve more accurate results (Rußwurm,

2020; Tang, 2022; Sykas, 2022). In this regard, it is

important to point out that typically Transformers

networks require more training data even than

convolutional neural networks (even the ImageNet

dataset that collects over one million labeled images

turns out to be too small). So it is simply possible that

the datasets used in the studies mentioned above are

not large enough for Transformers networks to begin

to take full advantage of their features and thus

outperform CNNs, and LSTMs or GRUs.

This paper shows a spectral-temporal data

processing method based on random forest, followed

by a two-dimensional convolutional network. A

comparison with a mono-dimensional convolutional

network is also provided.

The paper is structured as follows. Section 2

describes the materials and method: overall data

processing workflow is described by Section 2.1,

whereas detailed data preparation and processing are

covered by Section 2.2. In Section 3, experimental

results are shown. Finally, Section 4 draws

conclusions.

2 MATERIALS AND METHOD

2.1 Overall Data Processing Workflow

Figure 1 describes the overall data processing

workflow. On the top left, the hyperspectral cube

represents the data acquired by PRISMA satellite,

processed by the ground segment processing chain to

provide an L2D product, already atmospherically

corrected and geocoded. The 3D-hyperspectral

images are then transformed into a bidimensional

matrix, to feed the random forest model. When used

as a classifier, the random forest model provides

directly the target class (olive/grapevine). In addition,

it provides the most important features in order to

dimensionally reduce the 3D-hyperspectral images

throughout the spectral axis. Each pixel of the

reduced images is then projected onto the spectral-

temporal image. Finally, the spectral-temporal image

is considered as an input of the 2D Convolutional

Neural Network, which in turn provides the target

class. The next section illustrates a detailed pipeline

via a pilot example.

Figure 1: Overall data processing workflow.

Semantic Segmentation of Crops via Hyperspectral PRISMA Satellite Images

189

2.2 Detailed Data Processing

The dataset was generated from five hyperspectral

images acquired by means of the payload of the

PRISMA mission satellite (eoPortal, 2024). This

payload consists of an electro-optical instrument,

based on a pushbroom scanning technique, achieved

via a high spectral resolution imaging spectrometer.

The spectrometer works in the range of the

electromagnetic spectrum 0.4-2.5µm, covering both

the visible and near-infrared band (VNIR 0.4-1.0µm)

and the mid-infrared band (SWIR 0.9-2.5µm),

supplemented with a medium-resolution

panchromatic camera working in the spectral range

0.4-0.7µm (Candela, 2016).

The number of spectral channels is 66 for the

VNIR band and 173 for the SWIR band. However, in

this research only 63 channels were considered for the

VNIR band, due to the non-availability of 3 channels,

and 167 channels for the SWIR band, due to the non-

availability of 3 channels and the partial overlap of

the two spectral bands. Table 1 shows some

geometric characteristics of the images used in the

study, at 2D processing level: Acquisition Time

(Time), Average Solar Zenith (ASZ) Angle in 0-90

deg., Average Observing (AO) Angle in 0-90 deg.,

Average Relative Azimuth (ARA) Angle in 0-90

deg., and Size (pixel).

Table 1: Geometric characteristics of the PRISMA products

used.

Time

ASZ

Angle

ARA

Angle

Size

23/04/2022