REAL-TIME ENHANCEMENT OF IMAGE AND VIDEO SALIENCY USING SEMANTIC DEPTH OF FIELD

Zhaolin Su, Shigeo Takahashi

2010

Abstract

In this paper, we propose a method for automatically directing viewers' visual attention to important regions of images and videos in low-level vision. Inspired by the modern model of visual attention, the importance map of an input scene is automatically calculated by the combination of low-level features such as intensity and color, which are extracted using spatial filters in different spatial frequencies, together with a set of temporal features extracted using a temporal filter in case of dynamic scenes. A variable-kernel-convolution based on the importance map is then performed on the input scene, in order to make semantic depth of field effects in a way that important regions remain focused while others are blurred. The pipeline of our method is efficient enough to be executed in real time on modern low-end machines, and the associated experiment demonstrates that the proposed system can be complementary to the human visual system.

References

  1. Achanta, R., Hemami, S., Estrada, F., and Susstrunk, S. (2009). Frequency-tuned Salient Region Detection. In Proc. IEEE International Conf. Computer Vision and Pattern Recognition (CVPR2009), pages 1597-1604.
  2. Bruce, N. and Tsotsos, J. (2007). Attention based on information maximization. Journal of Vision, 7(9):950.
  3. Gao, D. and Vasconcelos, N. (2007). Bottom-up saliency is a discriminant process. In Proc. IEEE International Conf. Computer Vision (ICCV2007), pages 1-6.
  4. Harel, J., Koch, C., and Perona, P. (2006). Graph-based visual saliency. In Proc. Neural Information Processing Systems (NIPS2006), pages 570-577.
  5. Hou, X. and Zhang, L. (2007). Saliency detection: A spectral residual approach. In Proc. IEEE International Conf. Computer Vision and Pattern Recognition (CVPR2007), pages 1-8.
  6. Itti, L. and Baldi, P. F. (2009). Bayesian surprise attracts human attention. Vision Research, 49(10):1295-1306.
  7. Itti, L. and Koch, C. (2001). Computational modelling of visual attention. Nature Reviews Neuroscience, 2(3):194-203.
  8. Itti, L., Koch, C., and Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(11):1254-1259.
  9. Koch, C. and Ullman, S. (1985). Shifts in selective visual attention: towards the underlying neural circuitry. Human Neurobiology, 4(4):219-227.
  10. Kosara, R., Miksch, S., and Hauser, H. (2001). Semantic depth of field. In Proc. IEEE Symp. Information Visualization 2001 (INFOVIS2001), pages 97-104.
  11. Kosara, R., Miksch, S., and Hauser, H. (2002a). Focus+Context taken literally. IEEE Computer Graphics and Applications, 22(1):22-29.
  12. Kosara, R., Miksch, S., Hauser, H., Schrammel, J., Giller, V., and Tscheligi, M. (2002b). Useful properties of semantic depth of field for better F+C visualization. In Proc. Symp. Data Visualisation 2002 (VISSYM2002), pages 205-210.
  13. Ma, Y.-F. and Zhang, H.-J. (2003). Contrast-based image attention analysis by using fuzzy growing. In Proc. 11th ACM International Conf. Multimedia (MULTIMEDIA2003), pages 374-381.
  14. Simons, D. J. (2000). Current approaches to change blindness. Visual Cognition, 7:1-15.
  15. Tsotsos, J. (1990). Analyzing vision at the complexity level. Behavioral and Brain Sciences, 13(3):423-445.
  16. Zhang, L., Tong, M. H., and Cottrell, G. W. (2009). Sunday: Saliency using natural statistics for dynamic analysis of scenes. In Proc. 31st Annual Cognitive Science Society Conf. (CogSci2009), pages 2944-2949.
  17. Zhang, L., Tong, M. H., Marks, T. K., Shan, H., and Cottrell, G. W. (2008). SUN: A Bayesian framework for saliency using natural statistics. Journal of Vision, 8(7):1-20.
Download


Paper Citation


in Harvard Style

Su Z. and Takahashi S. (2010). REAL-TIME ENHANCEMENT OF IMAGE AND VIDEO SALIENCY USING SEMANTIC DEPTH OF FIELD . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010) ISBN 978-989-674-029-0, pages 370-375. DOI: 10.5220/0002825703700375


in Bibtex Style

@conference{visapp10,
author={Zhaolin Su and Shigeo Takahashi},
title={REAL-TIME ENHANCEMENT OF IMAGE AND VIDEO SALIENCY USING SEMANTIC DEPTH OF FIELD},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)},
year={2010},
pages={370-375},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002825703700375},
isbn={978-989-674-029-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2010)
TI - REAL-TIME ENHANCEMENT OF IMAGE AND VIDEO SALIENCY USING SEMANTIC DEPTH OF FIELD
SN - 978-989-674-029-0
AU - Su Z.
AU - Takahashi S.
PY - 2010
SP - 370
EP - 375
DO - 10.5220/0002825703700375