Authors:
Namita Agarwal
1
;
Anh Vo
1
;
Michela Bertolotto
1
;
Alan Barnett
2
;
Ahmed Khalid
2
and
Merry Globin
2
Affiliations:
1
School of Computer Science, University College Dublin, Dublin, Ireland
;
2
Dell Technologies, Cork, Ireland
Keyword(s):
Object Detection, Remote Sensing Images, LEVIR-Ship Dataset, YOLO Models, Model Drift.
Abstract:
: The rapid and accurate detection of ships within the wide sea area is essential for maritime applications.
Many machine learning (ML) based object detection models have been investigated to detect ships in remote
sensing imagery in previous research. Despite the availability of large-scale training datasets, the performance
of object detection models can decrease significantly when the statistical properties of input images vary
according to, for example, weather conditions. This is known as model drift. The occurrence of ML model
drift degrades the object detection accuracy and this reduction in accuracy can produce skewed outputs such
as, incorrectly classified images or inaccurate semantic tagging, thus making the detection task vulnerable
to malicious attacks. The majority of existing approaches that deal with model drift relate to time series.
While there is some work on model drift for imagery data and in the context of object detection, the problem
has not been exte
nsively investigated for object detection tasks in remote sensing images, especially with
large-scale image datasets. In this paper, the effects of model drift on the detection of ships from satellite
imagery data are investigated. Firstly, a YOLOv5 ship detection model is trained and validated using a publicly
available dataset. Subsequently, the performance of the model is validated against images subjected to artificial
blurriness, which is used in this research as a form of synthetic concept drift. The reduction of the model’s
performance according to increasing levels of blurriness demonstrates the effect of model drift. Specifically,
the average precision of the model dropped by more than 74% when the images were blurred at the maximum
level with a 11×11 Gaussian kernel size. More importantly, as the level of blurriness increased, the mean
confidence score of the detections decreased up to 20.8% and the number of detections also reduced. Since
the confidence scores and the number of detections are independent of ground truth data, such information has
the potential to be utilised to detect model drift in future research.
(More)