Agrinet: A Hyperspectral Image Based Precise Crop Classiﬁcation

Model

Aditi Palit

, Himanshu Dolekar

and Kalidas Yeturu

Department of Computer Science and Engineering,Indian Institute of Technology Tirupati, India

Keywords:

Agriculture, Crop Classiﬁcation, Hyperspectral Imagery, Convolution Neural Network.

Abstract:

Modern smart agriculture utilizes Unmanned Arial Vehicles (UAVs) with hyperspectral cameras to enhance

crop production to address the food security challenges. These cameras provide detailed crop information for

type identiﬁcation, disease detection, and nutrient assessment. However, processing Hyper Spectral Image

(HSI) is complex due to challenges such as high inter-class similarity, intra-class variability, and overlapping

spectral proﬁles. Thus, we introduce the Agrinet model, a convolutional neural network architecture, to han-

dle complex hyperspectral image processing. Our novelty lies in the image pre-processing step of selecting

suitable bands for better classiﬁcation. In tests, Agrinet achieved an impressive accuracy of 99.93% on the

LongKou crop dataset, outperforming the existing methods in classiﬁcation.

1 INTRODUCTION

According to the Food and Agricultural Organization

of the United Nations (FAO), agriculture is vital in

sustaining over 60% of the global population, utiliz-

ing approximately 12% of the world’s land for farm-

ing activities (Bender, 2020). Thus, to feed such a big

population, many advances are made in the agricul-

tural sector by developed and developing countries.

But despite signiﬁcant progress in the ﬁeld, chal-

lenges to food security persist (Rosegrant and Cline,

2003; Mc Carthy et al., 2018). These challenges in-

clude escalating demand for food, the strain on avail-

able land resources, the impact of climate change,

diminishing pollination, outbreaks of plant diseases,

water scarcity, inadequate food distribution systems

(Calicioglu et al., 2019; Pereira, 2017).

Addressing these challenges requires immediate

and proactive measures.

One notable technological advancement in this di-

rection is the widespread use of drones (Veroustraete,

2015). With spectral cameras, drones can capture HSI

that provides valuable insights into crops (Harakan-

nanavar et al., 2022; Mohanty and Salath

e, 2016;

Ahmed and Reddy, 2021). These HSI offer a range of

applications, including crop type classiﬁcation, early

https://orcid.org/0009-0007-7444-2994

https://orcid.org/0009-0005-7487-1663

https://orcid.org/0000-0002-9237-5453

disease detection, assessment of soil mineral and nu-

trient levels, identiﬁcation of water stress in leaves,

analysis of crop phenology, moisture content evalua-

tion, and more (Khan et al., 2018; Ravikanth et al.,

2017). Some models and pre-processing pipelines

have been suggested in the literature to examine the

HSI (Zhong et al., 2020a; Ramirez et al., 2020; Heble

et al., 2018; Mogili and Deepak, 2018; Yamamoto

et al., 2017; Guo et al., 2005; Xu et al., 2020), but

compared to their performance, our model is bet-

ter at selecting more meaningful spectral-spacial fea-

tures. When classiﬁed, HSI presents challenges due

to high inter-class similarity, spectral region overlap,

and high intra-class variability.

As stated above, the classiﬁcation of HSI presents

three critical challenges that need to be addressed:

• High Inter-Class Similarity. Objects within an

image often exhibit similar spectral reﬂectance

properties, leading to overlapping spectral signa-

tures. This similarity makes it challenging to dis-

tinguish between different crops or objects solely

based on spectral proﬁles.

• Overlapping Regions. HSI suffers from the chal-

lenge of overlapping spectral signatures from dif-

ferent classes within the feature space. This over-

lap arises due to the limited number of spectral

bands available.

• High Intra-Class Variability. The concept of

high intra-class variability points to the phe-

562

Palit, A., Dolekar, H. and Yeturu, K.

Agrinet: A Hyperspectral Image Based Precise Crop Classiﬁcation Model.

DOI: 10.5220/0012378400003660

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2024) - Volume 2: VISAPP, pages

562-566

ISBN: 978-989-758-679-8; ISSN: 2184-4321

nomenon where various regions or sub-regions

within a singular class exhibit dissimilar spectral

signatures.

To address these challenges, we introduce the

Agrinet model, a convolutional neural network with

active band selection according to Normalized differ-

ence vegetation index (NDVI), for crop classiﬁcation

using UAV hyperspectral data. To facilitate incre-

mental learning and prevent catastrophic forgetting,

we decompose the multiclass classiﬁer into a binary

classiﬁer (Singh et al., 2020) called binary Agrinet.

Normalized Difference Vegetation Index (NDVI):

is a vegetation index that assesses vegetation by

quantifying the disparity between near-infrared light,

which vegetation reﬂects strongly, and red light,

which vegetation absorbs. NDVI values consistently

range from -1 to +1. Negative values typically indi-

cate the presence of water, while values close to +1

suggest the presence of dense green foliage. Con-

versely, when NDVI is near zero, it indicates the ab-

sence of green leaves, potentially indicating an urban-

ized area. Thus, bands indicating vegetation are only

chosen at the pre-processing step for the model to ex-

tract better spatial-spectral features.

Another aspect of the Agrinet model is that it

doesn’t adhere to an end-to-end convolutional archi-

tecture. The reason to refrain from using end-to-

end convolutional models stems from a desire to pre-

vent unwarranted model bulk. Implementing a sep-

arate pre-processing pipeline proves advantageous in

crafting streamlined models, leading to a reduction in

computational demands.

2 DATASET DESCRIPTION

HSI classiﬁcation models often rely on low-resolution

satellite datasets with limited labeled pixels. To over-

come this, we used the WHU-Hi dataset (Luo et al.,

2018), which consists of high-resolution UAV-borne

HSI. This dataset comprises WHU-Hi-LongKou,

WHU-Hi-HanChuan, and WHU-Hi-HongHu (Zhong

et al., 2020b), offering improved spatial resolution.

UAV datasets are better suited for crop classiﬁcation,

disease detection, and crop phenology proﬁling than

satellite images.

3 AGRINET PIPELINE

Let us consider that we have HSI data of height(H) ×

width(W) × bands(B) along with the ground truth of

Table 1: Layer-based summary of Agrinet architecture.

Layer Output Shape No. of Parameters

Input Layer (11,11,20,1) 0

Conv3D 1 (9,9,18,8) 224

Conv3D 2 (7,7,16,16) 3472

Conv3D 3 (5,5,14,32) 13856

Flatten (11200) 0

Dense 1 (128) 1433728

Dropout 1 (128) 0

Dense 2 (64) 8256

Dropout 2 (64) 0

Output Layer (No. of class) 1430

height(H) × width(W). The dataset can be expressed

as X = [x

, x

. . . x

]

∈ HSI

(H×W×B)

. Thus, it has

H ×W samples associated with L class label per band

based on the ground truth with collectively B bands

(Ahmad et al., 2022). Each sample can be denoted

as (x

, y

) where y

is the label of sample x

. Thus i

sample belongs to j

class.

The raw HSI data is preprocessed and band selec-

tion is applied based on the spectral proﬁle by measur-

ing the NDVI of the bands. Thus, no useful bands are

clipped from the data. Feature extraction is done by

incremental PCA, reducing the dataset’s spectral di-

mensions. The HSI is divided into small 3D patches.

These are divided into train-test sets to feed to the

model to train the model and generate the classiﬁca-

tion map. Incremental principal component analysis

is applied after band selection. The most signiﬁcant N

bands are chosen from B bands where N << B while

maintaining the spatial dimensions. For the HSI data

cube to process with the Agrinet model (Figure1), the

data cube has to be divided into small 3D overlapping

spatial patches of a certain window size of S. Based

on the 3D patches, ground labels are formed consider-

ing the central pixel. This creates the small K 3D HSI

patches where K ∈ HSI

(S×S×N)

centered at location

(m, n) covering (S ×S) spatial window.

The input 3D HSI patches are convolved with a

3D kernel of the Agrinet model, thus calculating the

dot product between the patches and kernel. The ex-

tracted features are subsequently subjected to the relu

activation function, which introduces non-linearity to

the learned representations. The activation value at

position (a, b, c) in the i

layer and j

feature map is

given by u

a,b,c

i, j

φ(

i−1

∑

δ=1

∑

r=−α

∑

s=−β

∑

t=−λ

r,s,t

i, j,δ

∗u

(a+r),(b+s),(c+t)

(i−1),δ

i, j

)

φ represents the activation function. f

i−1

be the num-

ber of 3D feature map at (i − 1)

layer. The depth of

the kernel is represented by v

i, j

and b

i, j

corresponds

to the bias term. To extract the spectral and spatial

features, three convolutional 3D layers are deployed

Agrinet: A Hyperspectral Image Based Precise Crop Classiﬁcation Model

563

Figure 1: Proposed method Overview: The Agrinet model.

Figure 2: Binary Agrinet overview.

as shown in Figure1 so that the model can differ-

entiate between the spectral band and spatial infor-

mation without losing information. The weights are

randomly initialized, and the Adam optimizer is used

with the softmax function. The layer-based network

summary is shown in Table 1.

4 BINARY AGRINET

By decomposing the multi-class crop classiﬁcation

into a binary classiﬁer (Figure 2) for each crop and

leveraging the one-versus-all strategy, the complexity

and computational requirements are reduced, making

it feasible to handle large-scale multi-class classiﬁca-

tion tasks effectively(Lorena et al., 2008).

The Agrinet binary model 2 has a sigmoid at the

output layer, and a binary cross-entropy loss function

is used. The individual classiﬁer is optimized with an

SGD optimizer. For binary Agrinet (Figure 2) only

the training dataset is modiﬁed as per individual label

classiﬁer. On the training set, an individual binary

classiﬁer is built for each class label. Considering

the interest class as the target class or positive class

and other classes as the outlier class or negative class.

This individual binary Agrinet model is trained and

ensembled. The test set data is passed to ensembles

of the binary Agrinet model, the probability score is

calculated, and the maximum of them gives the ﬁnal

predicted class label.

Let n be the number of distinct classes in the

dataset D. let D

and M

be the dataset and binary

classiﬁer corresponding to label i ∈ {1 . . . n} s.t. Now

∀i ∈ 1 . . . n ∃ a binary data set B

corresponding to

each D

s.t

= B

i,p

∪ B

i,n

(1)

i,p

= {(x

, I[y = i]) | x

∈ D ∧I[y = i] = 1} (2)

i,n

= {(x

, I[y = i]) | x

∈ D ∧I[y = i] = 0} (3)

Let the training dataset for the i

client be

where

= B

i,p

∪ B

i,n

(4)

= train(

) (5)

x ∈ D

test

⊆ D and let ρ be the probability conﬁdence

function. Then, the predicted class for x is given by

ˆy = arg max

i∈{1...n}

ρ(M

) (6)

The sample is assigned to the class having the highest

probability score.

5 EVALUATION ANALYSIS

To extract patches in the HSI cube, a window size of

15 × 15 is used. Meanwhile, the learning rate is ﬁxed

at 0.001 throughout the experiments. A total of 30 of

the most signiﬁcant bands are used after the band se-

lection based on agricultural indices like the normal-

ized difference vegetation index (NDVI) and moisture

index using incremental principal component analy-

sis. The size of the training set is 60%, the test data

VISAPP 2024 - 19th International Conference on Computer Vision Theory and Applications

564

Table 2: Performnace of Agrinet on comparison to benchmark models.

Dataset Metric SVM FNEA-OO SVRFMC SSAN SSRN pResNet CNNCRF SSFCN FPGA CNN Agrinet

LongKou OA(%) 94.96 98.59 98.37 94.44 99.02 98.70 98.91 94.60 99.17 97.30 99.93

AA(%) 95.18 97.48 97.41 95.38 99.39 98.88 98.21 95.27 99.30 97.40 99.76

(κ) 0.9345 0.9815 0.9786 0.9279 0.9871 0.9830 0.9857 0.9300 0.9912 0.9647 0.9995

HanChan OA(%) 73.55 88.83 89.86 87.34 91.29 95.32 93.74 94.26 97.45 93.74 99.75

AA(%) 73.46 83.21 84.37 88.25 90.09 92.91 92.69 86.42 97.88 84.93 99.50

(κ) 0.7414 0.8330 0.8435 0.8673 0.8815 0.9212 0.9290 0.8742 0.9747 0.8497 0.9969

HongHu OA(%) 77.61 85.63 86.53 88.63 89.82 93.32 93.95 89.75 97.83 93.95 99.73

AA(%) 71.23 83.13 87.16 84.93 91.68 95.83 94.78 92.03 97.79 86.16 99.31

(κ) 0.6805 0.8590 0.8728 0.8415 0.8910 0.9412 0.9217 0.9219 0.9678 0.8210 0.9969

is 40%, and the validation is 30% of the training data

(Ahmad et al., 2022).

For binary, an Agrinet window size of 11 × 11

is used to extract patches in the hyperspectral image

cube. A total of 20 most signiﬁcant bands are used

after the band selection based on agricultural indices

like NDVI and moisture index using incremental prin-

cipal component analysis.

5.0.1 Analysis on WHU-Hi- LongKou Dataset

For evaluation purposes, we calculate overall accu-

racy (OA), kappa (κ) coefﬁcient, and average accu-

racy (AA) using the confusion matrix obtained from

the assessment. The metric of average accuracy (AA)

provides insight into the classiﬁcation performance

per class. κ coefﬁcient, on the other hand, serves as a

statistical measure that gauges the level of agreement

between the classiﬁcation map and the ground truth

map, relying on mutual information. The accuracy

of Agrinet is compared with outcomes obtained from

models in (Zhong et al., 2020b) , providing a com-

prehensive benchmark against existing techniques as

shown in Table 2. Agrinet’s performance demon-

strates a marginal percentage advantage over the cur-

rent state-of-the-art model across OA, AA, and κ on

the WHU-Hi-LongKou dataset.

We evaluate our binary Agrinet model on

the WHU-Hi-LongKou dataset.The overall accuracy

from our binary Agrinet model is 87.6% , and the

average accuracy is 97.53%. While the model might

exhibit lower performance in direct comparison to the

multiclass classiﬁer. In future we wish to show it’s

advantage over multiclass model when a new class is

added.

5.0.2 Analysis on WHU-Hi-HanChuan Dataset

The performance comparison of our Agrinet model

on the HanChan is shown in Table 2. OA and AA

of Agrinet exhibit an approximate 2% to 3% increase

compared to FPGA, which stands out as the supe-

rior model among the comparative approaches. The

Agrinet model performs well in this data, which is

in a more complex urban-rural setting. The disparity

becomes more pronounced in the case of the kappa

value, with Agrinet showcasing an approximately 2%

higher value.

5.0.3 Analysis on WHU-Hi-HongHu Dataset

As illustrated in Table 2, our model surpasses the

performance of other models by a substantial mar-

gin on this dataset, which presents complexity due

to its diverse crop labels within a single region and

many class labels. In particular, Agrinet’s OA and

AA scores are approximately 4% to 5% higher than

most of the models, which stands as the leading com-

parative model. The most pronounced distinction is

observed in the κ value, with Agrinet exhibiting an

approximate 3% increase.

5.1 Conclusion and Future Work

Agrinet performs at a level on par with the state-

of-the-art models and excels in all benchmark cate-

gories, including AA, OA, and κ. These results high-

light Agrinet’s effectiveness, especially with complex

datasets. However, it’s essential to acknowledge that

our model’s high accuracy results from fully labeled,

high-resolution training data, which may not reﬂect

real-world conditions with incomplete labeling and

lower spatial resolution. For future research, we plan

to conduct in-depth ablation studies on the various hy-

perparameters of the models to gain a better under-

standing of their impact on model performance.

ACKNOWLEDGEMENTS

The authors acknowledge computational and

funding support from the project numbered

CSE2122001FACEKALI and titled Design and

Development of Disaster Response Dashboard for

India for carrying out the work.

Agrinet: A Hyperspectral Image Based Precise Crop Classiﬁcation Model

565

REFERENCES

Ahmad, M., Khan, A. M., Mazzara, M., Distefano, S., Ali,

M., and Sarfraz, M. S. (2022). A fast and compact

3-d cnn for hyperspectral image classiﬁcation. IEEE

Geoscience and Remote Sensing Letters, 19:1–5.

Ahmed, A. and Reddy, G. (2021). A mobile-based system

for detecting plant leaf diseases using deep learning.

AgriEngineering, 3:478–493.

Bender, F. (2020). Statistical methods for food and agricul-

ture. CRC Press.

Calicioglu, O., Flammini, A., Bracco, S., Bell

u, L., and

Sims, R. (2019). The future challenges of food and

agriculture: An integrated analysis of trends and solu-

tions. Sustainability, 11(1):222.

Guo, B.and Gunn, S., Damper, B., and Nelson, J. (2005).

Adaptive band selection for hyperspectral image fu-

sion using mutual information. ” In 2005 7th Interna-

tional Conference on Information Fusion, IEEE, 1.

Harakannanavar, S., Rudagi, J., Puranikmath, V., Siddiqua,

A., and Pramodhini, R. (2022). Plant leaf disease de-

tection using computer vision and machine learning

algorithms. Global Transitions Proceedings, 3:305–

310. International Conference on Intelligent Engi-

neering Approach(ICIEA-2022).

Heble, S., Kumar, A., Prasad, K., Samirana, S.and Rajalak-

shmi, P., and Desai, U. (2018). A low power iot net-

work for smart agriculture. In 2018 IEEE 4th World

Forum on Internet of Things (WF-IoT), pages 609–

614.

Khan, M., Khan, H., Yousaf, A., Khurshid, K., and Ab-

bas, A. (2018). Modern trends in hyperspectral image

analysis: A review. Ieee Access, 6:14118–14129.

Lorena, A., De Carvalho, A., and Gama, J. (2008). A review

on the combination of binary classiﬁers in multiclass

problems. Artiﬁcial Intelligence Review, 30:19–37.

Luo, Y., Zou, J., Yao, C., Zhao, X., Li, T., and Bai, G.

(2018). Hsi-cnn: A novel convolution neural network

for hyperspectral image. In 2018 International Con-

ference on Audio, Language and Image Processing

(ICALIP), pages 464–469. IEEE.

Mc Carthy, U., Uysal, I., Badia-Melis, R., Mercier, S.,

O’Donnell, C., and Ktenioudaki, A. (2018). Global

food security–issues, challenges and technological so-

lutions. Trends in Food Science & Technology, 77:11–

20.

Mogili, U. and Deepak, B. (2018). Review on application

of drone systems in precision agriculture. Procedia

computer science, 113:502–509.

Mohanty, S.P.and Hughes, D. and Salath

e, M. (2016). Using

deep learning for image-based plant disease detection.

Frontiers in plant science.

Pereira, L. (2017). Water, agriculture and food: chal-

lenges and issues. Water Resources Management,

31(10):2985–2999.

Ramirez, W., Achanccaray, P.and Mendoza, L., and

Pacheco, M. (2020). Deep convolutional neural net-

works for weed detection in agricultural crops using

optical aerial images. IEEE Latin American GRSS &

ISPRS Remote Sensing Conference (LAGIRS), pages

133–137.

Ravikanth, L., Jayas, D., White, N., Fields, P., and Sun, D.

(2017). Extraction of spectral information from hyper-

spectral data and application of hyperspectral imaging

for food and agricultural products. Food and biopro-

cess technology, 10:1–33.

Rosegrant, M. and Cline, S. (2003). Global food security:

challenges and policies. Science, 302(5652):1917–

1919.

Singh, P., Singh, V., Pandey, M., and Karthikeyan, S.

(2020). One-class classiﬁer ensemble based enhanced

semisupervised classiﬁcation of hyperspectral remote

sensing images. 2020 International Conference on

Emerging Smart Computing and Informatics (ESCI),

pages 22–27.

Veroustraete, F. (2015). The rise of the drones in agricul-

ture. EC agriculture, 2(2):325–327.

Xu, B., Wang, W., Falzon, G., Kwan, P., Guo, L., Sun, Z.,

and Li, C. (2020). Livestock classiﬁcation and count-

ing in quadcopter aerial images using mask r-cnn. In-

ternational Journal of Remote Sensing, 41(21):8121–

8142.

Yamamoto, K., Togami, T., and Yamaguchi, N. (2017).

Super-resolution of plant disease images for the ac-

celeration of image-based phenotyping and vigor di-

agnosis in agriculture. Sensors, 17(11):2557.

Zhong, Y., Hu, X., Luo, C., Wang, X., Zhao, J., and Zhang,

L. (2020a). Whu-hi: Uav-borne hyperspectral with

high spatial resolution (h2) benchmark datasets and

classiﬁer for precise crop identiﬁcation based on deep

convolutional neural network with crf. Remote Sens-

ing of Environment.

Zhong, Y., Hu, X., Luo, C., Wang, X., Zhao, J., and Zhang,

L. (2020b). Whu-hi: Uav-borne hyperspectral with

high spatial resolution (h2) benchmark datasets and

classiﬁer for precise crop identiﬁcation based on deep

convolutional neural network with crf. Remote Sens-

ing of Environment, 250:112012.

VISAPP 2024 - 19th International Conference on Computer Vision Theory and Applications

566