Effects of Model Drift on Ship Detection Models

Namita Agarwal, Anh Vo, Michela Bertolotto, Alan Barnett, Ahmed Khalid, Merry Globin

2024

Abstract

: The rapid and accurate detection of ships within the wide sea area is essential for maritime applications. Many machine learning (ML) based object detection models have been investigated to detect ships in remote sensing imagery in previous research. Despite the availability of large-scale training datasets, the performance of object detection models can decrease significantly when the statistical properties of input images vary according to, for example, weather conditions. This is known as model drift. The occurrence of ML model drift degrades the object detection accuracy and this reduction in accuracy can produce skewed outputs such as, incorrectly classified images or inaccurate semantic tagging, thus making the detection task vulnerable to malicious attacks. The majority of existing approaches that deal with model drift relate to time series. While there is some work on model drift for imagery data and in the context of object detection, the problem has not been extensively investigated for object detection tasks in remote sensing images, especially with large-scale image datasets. In this paper, the effects of model drift on the detection of ships from satellite imagery data are investigated. Firstly, a YOLOv5 ship detection model is trained and validated using a publicly available dataset. Subsequently, the performance of the model is validated against images subjected to artificial blurriness, which is used in this research as a form of synthetic concept drift. The reduction of the model’s performance according to increasing levels of blurriness demonstrates the effect of model drift. Specifically, the average precision of the model dropped by more than 74% when the images were blurred at the maximum level with a 11×11 Gaussian kernel size. More importantly, as the level of blurriness increased, the mean confidence score of the detections decreased up to 20.8% and the number of detections also reduced. Since the confidence scores and the number of detections are independent of ground truth data, such information has the potential to be utilised to detect model drift in future research.

Paper Citation

in Harvard Style

Agarwal N., Vo A., Bertolotto M., Barnett A., Khalid A. and Globin M. (2024). Effects of Model Drift on Ship Detection Models. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 750-754. DOI: 10.5220/0012443600003660

in Bibtex Style

@conference{visapp24,
author={Namita Agarwal and Anh Vo and Michela Bertolotto and Alan Barnett and Ahmed Khalid and Merry Globin},
title={Effects of Model Drift on Ship Detection Models},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2024},
pages={750-754},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012443600003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Effects of Model Drift on Ship Detection Models
SN - 978-989-758-679-8
AU - Agarwal N.
AU - Vo A.
AU - Bertolotto M.
AU - Barnett A.
AU - Khalid A.
AU - Globin M.
PY - 2024
SP - 750
EP - 754
DO - 10.5220/0012443600003660
PB - SciTePress