Specification Based Testing of Object Detection for Automated Driving Systems via BBSL

Kento Tanaka, Toshiaki Aoki, Tatsuji Kawai, Takashi Tomita, Daisuke Kawakami, Nobuo Chida

2023

Abstract

Automated driving systems(ADS) are major trend and the safety of such critical system has become one of the most important research topics. However, ADS are complex systems that involve various elements. Moreover, it is difficult to ensure safety using conventional testing methods due to the diversity of driving environments. Deep Neural Network(DNN) is effective for object detection processing that takes diverse driving environments as input. A method such as Intersection over Union (IoU) that defines a threshold value for the discrepancy between the bounding box of the inference result and the bounding box of the ground-truth-label can be used to test the DNN. However, there is a problem that these tests are difficult to sufficiently test to what extent they meet the specifications of ADS. Therefore, we propose a method for converting formal specifications of ADS written in Bounding Box Specification Language (BBSL) into tests for object detection. BBSL is a language that can mathematically describe the specification of OEDR (Object and Event Detection and Response), one of the tasks of ADS. Using these specifications, we define specification based testing of object detection for ADS. Then, we evaluate that this test is more safety-conscious for ADS than tests using IoU.

Paper Citation

in Harvard Style

Tanaka K., Aoki T., Kawai T., Tomita T., Kawakami D. and Chida N. (2023). Specification Based Testing of Object Detection for Automated Driving Systems via BBSL. In Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE, ISBN 978-989-758-647-7, SciTePress, pages 250-261. DOI: 10.5220/0011997400003464

in Bibtex Style

@conference{enase23,
author={Kento Tanaka and Toshiaki Aoki and Tatsuji Kawai and Takashi Tomita and Daisuke Kawakami and Nobuo Chida},
title={Specification Based Testing of Object Detection for Automated Driving Systems via BBSL},
booktitle={Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE,},
year={2023},
pages={250-261},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011997400003464},
isbn={978-989-758-647-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering - Volume 1: ENASE,
TI - Specification Based Testing of Object Detection for Automated Driving Systems via BBSL
SN - 978-989-758-647-7
AU - Tanaka K.
AU - Aoki T.
AU - Kawai T.
AU - Tomita T.
AU - Kawakami D.
AU - Chida N.
PY - 2023
SP - 250
EP - 261
DO - 10.5220/0011997400003464
PB - SciTePress