New Method for Evaluation of Video Segmentation Quality

Mahmud Abdulla Mohammad, Ioannis Kaloskampis, Yulia Hicks

2015

Abstract

Segmentation is an important stage in image/video analysis and understanding. There are many different approaches and algorithms for image/video segmentation, hence their evaluation is also important in order to assess the quality of segmentation results. Nonetheless, so far there was little research aimed specifically at evaluation of video segmentation quality. In this article, we propose the criteria of good quality of video segmentation suitable for assessment of video segmentations by including a requirement for temporal region consistency. We also propose a new method for evaluation of video segmentation quality on the basis of the proposed criteria. The new method can be used both for supervised and unsupervised evaluation. We designed a test video set specifically for evaluation of our method and evaluated the proposed method using both this set and segmentations of real life videos. We compared our method against a state of the art supervised evaluation method. The comparison showed that our method is better at evaluation of perceptual qualities of video segmentations as well as at highlighting certain defects of video segmentations.

References

  1. Borsotti, M., Campadelli, P., and Schettini, R. (1998). Quantitative evaluation of color image segmentation results. Pattern Recognition Letters, 19(8):741 - 747.
  2. Chabrier, S., Emile, B., Rosenberger, C., and Laurent, H. (2006). Unsupervised performance evaluation of image segmentation. EURASIP Journal on Applied Signal Processing, 2006:217-217.
  3. Charron, C. and Hicks, Y. (2010). An evolving mog for online image sequence segmentation. In ICIP, pages 2189-2192.
  4. Chen, A. Y. C. and Corso, J. J. (2010). Propagating multiclass pixel labels throughout video frames. In Proc. of Western New York Image Processing Workshop.
  5. Chen, H.-C. and Wang, S.-J. (2004). The use of visible color difference in the quantitative evaluation of color image segmentation. Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), 3:iii593-6 vol.3.
  6. Correia, P. and Pereira, F. (2003). Objective evaluation of video segmentation quality. IEEE Trans. on Image Processing, 12(2):186-200.
  7. Dey, V., Zhang, Y., and Zhong, M. (2010). A review on image segmentation techniques with remote sensing perspective. In Proc. of the Int. Society for Photogrammetry and Remote Sensing Symposium (ISPRS), volume 38, pages 5-7.
  8. Erdem, C¸ . E., Sankur, B., and Tekalp, A. M. (2004). Performance measures for video object segmentation and tracking. IEEE Trans. on Image Processing, 13(7):937-951.
  9. Galasso, F., Nagaraja, N. S., Cárdenas, T. J., Brox, T., and Schiele, B. (2013). A unified video segmentation benchmark: Annotation, metrics and analysis. In IEEE Int. Conf. on Computer Vision (ICCV).
  10. Gelasca, E. D. and Ebrahimi, T. (2006). On evaluating metrics for video segmentation algorithms. In Second Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM ).
  11. Greenspan, H., Dvir, G., and Rubner, Y. (2004). Contextdependent segmentation and matching in image databases. Computer Vision and Image Understanding, 93(1):86-109.
  12. Grundmann, M., Kwatra, V., Han, M., and Essa, I. (2010). Efficient hierarchical graph-based video segmentation. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 2141-2148.
  13. Haralick, R. M. and Shapiro, L. G. (1985). Image segmentation techniques. Computer vision, graphics, and image processing, 29(1):100-132.
  14. Kaloskampis, I. and Hicks, Y. (2014). Estimating adaptive coefficients of evolving gmms for online video segmentation. In 6th Int. Symp. on Communications, Control and Signal Processing (ISCCSP), pages 513- 516.
  15. Levine, M. D. and Nazif, A. M. (1985). Dynamic measurement of computer generated image segmentations. IEEE Trans. on Pattern Analysis and Machine Intelligence, PAMI-7(2):155-164.
  16. Liu, J. and Yang, Y.-H. (1994). Multiresolution color image segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 16(7):689-700.
  17. Martin, D., Fowlkes, C., Tal, D., and Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. Eighth IEEE Int. Conf. on Computer Vision (ICCV), volume 2, pages 416-423.
  18. Meila?, M. (2003). Comparing clusterings by the variation of information. In Learning theory and kernel machines, volume 2777, pages 173-187.
  19. Morris, O., Lee, M. d. J., and Constantinides, A. (1986). Graph theory for image analysis: an approach based on the shortest spanning tree. Communications, Radar and Signal Processing, IEE Proc. F, 133(2):146-152.
  20. Rosenberger, C. and Chehdi, K. (2000). Genetic fusion: application to multi-components image segmentation. In Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), volume 6, pages 2223- 2226.
  21. Unnikrishnan, R., Pantofaru, C., and Hebert, M. (2005). A measure for objective evaluation of image segmentation algorithms. In IEEE Computer Society Conf. on Computer Vision and Pattern Recognition-Workshops (CVPR) Workshops, pages 34-34.
  22. Unnikrishnan, R., Pantofaru, C., and Hebert, M. (2007). Toward objective evaluation of image segmentation algorithms. IEEE Trans. on Pattern Analysis and Machine Intelligence, 29(6):929-944.
  23. Xu, C. and Corso, J. J. (2012). Evaluation of super-voxel methods for early video processing. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 1202-1209.
  24. Zhang, H., Fritts, J. E., and Goldman, S. A. (2004). An entropy-based objective evaluation method for image segmentation. In Electronic Imaging, pages 38-49.
  25. Zhang, H., Fritts, J. E., and Goldman, S. A. (2008). Image segmentation evaluation: A survey of unsupervised methods. Computer Vision and Image Understanding, 110(2):260-280.
Download


Paper Citation


in Harvard Style

Mohammad M., Kaloskampis I. and Hicks Y. (2015). New Method for Evaluation of Video Segmentation Quality . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-089-5, pages 523-530. DOI: 10.5220/0005306205230530


in Bibtex Style

@conference{visapp15,
author={Mahmud Abdulla Mohammad and Ioannis Kaloskampis and Yulia Hicks},
title={New Method for Evaluation of Video Segmentation Quality},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={523-530},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005306205230530},
isbn={978-989-758-089-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2015)
TI - New Method for Evaluation of Video Segmentation Quality
SN - 978-989-758-089-5
AU - Mohammad M.
AU - Kaloskampis I.
AU - Hicks Y.
PY - 2015
SP - 523
EP - 530
DO - 10.5220/0005306205230530