Depth-Enhanced 3D Deep Learning for Strawberry Detection and Widest Region Identification in Polytunnels

Gabriel Lins Tenorio, Gabriel Lins Tenorio, Weria Khaksar, Wouter Caarls

2024

Abstract

This paper presents an investigation into the use of 3D Deep Learning models for enhanced strawberry detection in polytunnels. We focus on two main tasks: firstly, fruit detection, comparing the standard MaskRCNN and an adapted version that integrates depth information (MaskRCNN-D), both capable of classifying strawberries based on their maturity (ripe, unripe) and health status (affected by disease or fungus); secondly, for the identification of the widest region of strawberries, we compare a contour-based algorithm with an enhanced version of the VGG-16 model. Our findings demonstrate that integrating depth data into the MaskRCNN-D results in up to a 13.7% improvement in mean Average Precision (mAP) from 0.81 to 0.92 across various strawberry test sets, including simulated ones, emphasizing the model’s effectiveness in both real-world and simulated agricultural scenarios. Furthermore, our end-to-end pipeline approach, which combines the fruit detection (MaskRCNN-D) and widest region identification models (enhanced VGG-16), shows a remarkably low localization error, achieving down to 11.3 pixels of Root Mean Square Error (RMSE) in a 224 × 224 strawberry cropped image. This pipeline integration, combining the strengths of both models, provides the most effective result, enabling their application in autonomous fruit monitoring systems.

Paper Citation

in Harvard Style

Lins Tenorio G., Khaksar W. and Caarls W. (2024). Depth-Enhanced 3D Deep Learning for Strawberry Detection and Widest Region Identification in Polytunnels. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 471-481. DOI: 10.5220/0012425200003636

in Bibtex Style

@conference{icaart24,
author={Gabriel Lins Tenorio and Weria Khaksar and Wouter Caarls},
title={Depth-Enhanced 3D Deep Learning for Strawberry Detection and Widest Region Identification in Polytunnels},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2024},
pages={471-481},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012425200003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - Depth-Enhanced 3D Deep Learning for Strawberry Detection and Widest Region Identification in Polytunnels
SN - 978-989-758-680-4
AU - Lins Tenorio G.
AU - Khaksar W.
AU - Caarls W.
PY - 2024
SP - 471
EP - 481
DO - 10.5220/0012425200003636
PB - SciTePress