TOWARDS INTENT DEPENDENT IMAGE ENHANCEMENT - State-of-the-art and Recent Attempts

Marco Bressan, Gabriela Csurka, Sebastien Favre

2007

Abstract

Image enhancement is mostly driven by intent and its future largely relies on our ability to map the space of intentions with the space of possible enhancements. Taking into account the semantic content of an image is an important step in this direction where contextual and aesthetic dimensions are also likely to have an important role. In this article we detail the state-of-the-art and some recent efforts in for semantic or content-dependent enhancement. Through a concrete example we also show how image understanding and image enhancement tools can be brought together. We show how the mapping between semantic space and enhancements can be learnt from user evaluations when the purpose is subjective quality measured by user preference. This is done by introducing a discretization of both spaces and notions of coherence, agreement and relevance to the user response. Another example illustrates the feasibility of solving the situation where the binary option of whether or not to enhance is considered.

References

  1. Adams, J. E., Hamilton, J. F., Gindele, E. B., and Pillman, B. H. (2003). Method for automatic white balance of digital images. US Patent 6573932, Kodak.
  2. Allen, D. J., Carley, A. L., and Levantovsky, V. (2004). Method of adaptively enhancing a digital image. US Patent 6807313, Oak Technology, Inc. (Sunnyvale, CA).
  3. Avril, C. and Nguyen-Trong, T. (1992). Linear filtering for reducing blocking effects in orthogonal transform image coding,. J. Electronic Imaging, 1(2).
  4. Barnard, K., Duygulu, P., D. Forsyth, N. de Freitas, D. B., and Jordan, M. (2003). Matching words and pictures. J. of Machine Learning Research, 3.
  5. Barnard, K., Martin, L., Coath, A., and Funt, B. (2002). A comparison of computational color constancy algorithms. IEEE Trans. on Image Processing, 11(9).
  6. Battiato, S., Bosco, A., Castorina, A., and Messina, G. (2003). Automatic global image enhancement by skin dependent exposure correction. In IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing.
  7. Bosch, A., Zisserman, A., and Munoz, X. (2006). Scene classification via pLSA. In ECCV.
  8. Boutell, M. and Luo, J. (2007). Beyond pixels: Exploiting camera metadata for photo classification. pattern recognition. Special Issue on Image Understanding for Digital Photos. to appear.
  9. Bressan, M., Dance, C. R., Poirier, H., and Arregui, D. (2007). LCE: (automatic) local contrast enhancement. In SPIE, Electronic Imaging.
  10. Buswell, G. (1935). How People Look at Pictures. Chicago University Press, Chicago.
  11. Carbonetto, P., de Freitas, N., and Barnard, K. (2004). A statistical model for general contextual object recognition. In ECCV.
  12. Chambah, M., Semani, D., Renouf, A., Coutellemont, P., and Rizzi, A. (2004). Underwater color constancy: enhancement of automatic live fish recognition. In SPIE Electronic Imaging, Science and Technology, volume 5293.
  13. Chen, Y. and Wang, J. Z. (2004). Image categorization by learning and reasoning with regions. JMLR, 5.
  14. Chiu, K., Herf, K., Shirley, M., Swamy, P., Wang, S., and Zimmerman, K. (1993). Spatially nonuniform scaling functions for high contrast images. In Kaufmann, M., editor, Proc. Graphics Interface 7893.
  15. Crandall, D. and Huttenlocher, D. (2006). Weakly supervised learning of part-based spatial models for visual object recognition. In ECCV.
  16. Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. (2004). Visual categorization with bags of keypoints. In ECCV Workshop on Statistical Learning for Computer Vision.
  17. Csurka, G., Willamowski, J., Dance, C., and Perronnin, F. (2005). Incorporating geometry information with weak classifiers for improved generic visual categorization. In Int. Conf. on Image Analysis and Processing.
  18. Datta, R., Joshi, D., Li, J., and Wang, J. (2006). Studying aesthetics in photographic images using a computational approach. In Leonardis, A., Bischof, H., and Pinz, A., editors, ECCV.
  19. Devlin, K., Chalmers, A., Wilkie, A., and Purgathofer, W. (2002). Star: Tone reproduction and physically based spectral rendering. In State of the Art Reports, Eurographics.
  20. DiCarlo, J. and Wandell, B. (2001). Rendering high dynamic range images. In SPIE: Image Sensors, volume 3965.
  21. Durand, F. and Dorsey, J. (2002). Fast bilateral filtering for the display of high dynamic range images. ACM Trans. on Graphics 21, 3.
  22. Eschbach, R. and Fuss, W. (1999). Automatic enhancement of scanned photographs. In EI Color Imaging: Device Independent Color, Color Hardcopy and Graphic Arts IV (ei16).
  23. Eschbach, R., Waldron, B., and Fuss, W. (1995). Us patent 5340502: Image-dependent luminance enhancement. Xerox Corporation.
  24. Evans, R. M. (1951). Method for correcting photographic color print. US Patent 2571697, Kodak.
  25. Everingham, M., Gool, L. V., Williams, C., and Zisserman, A. (2005). The pascal visual object classes challenge results. http://www.pascalnetwork.org/challenges/VOC/voc2005/results.pdf.
  26. Everingham, M., Zisserman, A., Williams, C., and Gool, L. V. (2006). The pascal visual object classes challenge 2006. http://www.pascalnetwork.org/challenges/VOC/voc2006/results.pdf.
  27. Fan, Z. and de Queiroz, R. (2003). Identification of bitmap compression history: Jpeg detection and quantizer estimation. IEEE Trans. on Image Processing, 12(2).
  28. Farquhar, J., Szedmak, S., Meng, H., and Shawe-Taylor, J. (2005). Improving “bag-of-keypoints” image categorisation. Technical report, University of Southampton.
  29. Fattal, R., Lischinski, D., and Werman, M. (2002). Gradient domain high dynamic range compression. ACM Trans. on Graphics 21, 3.
  30. Fei-Fei, L. and Perona, P. (2005). A Bayesian hierarchical model for learning natural scene categories. In CVPR, volume 2.
  31. Fergus, R., Perona, P., and Zisserman, A. (2003). Object class recognition by unsupervised scale-invariant learning. In CVPR.
  32. Fergus, R., Singh, B., Hertzmann, A., Roweis, S., and Freeman, W. T. (2006). Removing camera shake from a single image. In SIGGRAPH.
  33. Fidler, S., Berginc, G., and Leonardis, A. (2006). Hierarchical statistical learning of generic parts of object structure. In CVPR.
  34. Fischer, M., Parades, J., and Arce, G. (2002). Weighted median image sharpeners for the world wide web. IEEE Trans. On Image Processing, 11(7).
  35. Fredembach, C., Schr öder, M., and Süsstrunk, S. (2003). Region-based image classification for automatic color correction. In IS&T Color Imaging Conference.
  36. Furuki, I. and Yamada, K. (2006). Image enhancement device and image enhancement method of thermal printer. US Patent Application 20050168561, Mitsubishi Denki Kabushiki Kaisha.
  37. Gallagher, A. and Bruehs, W. (2006). Method and system for improving an image characteristic based on image content. US Patent 20060228040, Eastman Kodak Company (Rochester, NY).
  38. Gasparini, F. and Schettini, R. (2004). Color balancing of digital photos using simple image statistics. Pattern Recognition, 37.
  39. Gasparini, F. and Schettini, R. (2005). Automatic redeye removal for smart enhancement of photos of unknown origin. In Int. Conf. on Visual information systems.
  40. Gaubatz, M. and Ulichney, R. (2002). Automatic red-eye detection and correction. In ICIP.
  41. Gonzalez, R. C. and Woods, R. (1992). Digital image processing. Addison-Wesley Pub. Comp, Inc., Reading, MA.
  42. Haeberli, P. and Voorhies, D. (1994). Image processing by linear interpolation and extrapolation. IRIS Universe Magazine, Silicon Graphics, 28.
  43. Henderson, J. and Hollingworth, A. (1999). scene perception. Annu. Rev. Psychol., 50.
  44. Hillebrand, G., Miyamoto, K., and et al (2003). Skin imaging and analysis systems and methods. US Patent 6571003,The Procter & Gamble Company.
  45. Hoffenberg, S. (2006). Changing cameraphone user behaviour. Half-day seminar at Photokina.
  46. Ichikawa, T. and Miyasaka, T. (2005). Web print system with image enhancement. US Patent 6914694, Seiko Epson Corporation (Tokyo, JP).
  47. Jalobeanu, A., Blanc-Fraud, L., and Zerubia, J. (2002). Estimation of blur and noise parameters in remote sensing. In Int. Conf. on Acoustics, Speech and Signal Processing.
  48. Kim, N., Jang, I. H., Kim, D., and Hong, W. H. (1998). Reduction of blocking artifact in block-coded images using using wavelet transform. IEEE Trans. Circuits and Systems, 8(3).
  49. Ledda, P., Chalmers, A., Troscianko, T., and Seetzen, H. (2005). Evaluation of tone mapping operators using a high dynamic range display. In Proc. ACM SIGGRAPH 7805.
  50. Lefebvre, G., Laurent, C., Ros, J., and Garcia, C. (2006). Supervised image classification by som activity map comparison. In ICPR.
  51. Leibe, B., Leonardis, A., and Schiele, B. (2004). Combined object categorization and segmentation with an implicit shape model. In ECCV Workshop on Statistical Learning for Computer Vision.
  52. Li, J. and Wang, J. Z. (2003). Automatic linguistic indexing of pictures by a statistical modeling approach. PAMI, 25:9.
  53. Li, Y., Bilmes, J. A., and Shapiro, L. G. (2004). Object class recognition using images of abstract regions. In ICPR.
  54. Lin, Q., Atkins, C., and Tretter, D. (2002). Image enhancement using face detection. US Patent Application 20020172419, Hewlett Packard Company.
  55. Lin, Q. and Tretter, D. (2005). Camera meta-data for content categorization. US Patent 6977679, Hewlett Packard Company.
  56. Luo, J. (2003). Determining orientation of images containing blue sky. US Patent 6512846, Eastman Kodak Company (Rochester, NY).
  57. Luo, J. and Etz, S. (2002). A physical model based approach to detecting sky in photographic images. IEEE Trans. on Image Processing, 11(3).
  58. Marée, R., Geurts, P., Piater, J., and Wehenkel, L. (2005). Random subwindows for robust image classification. In CVPR, volume 1.
  59. Martin, A., Doddington, G., Kamm, T., Ordowski, M., and Przybocki, M. (1997). The DET curve in assessment of detection task performance. In EUROSPEECH.
  60. Meier, T., Ngan, K. N., and Crebbin, G. (1999). Reduction of blocking artifacts in image and video coding. IEEE Trans. on Circuits and Systems for Video Technology, 9(3).
  61. Minami, S. and Zakhor, A. (1995). An optimization approach for removing blocking effects in transform coding. IEEE Trans. Circuits and Systems for Video Technology, 5(4).
  62. Motwani, M., Gadiya, M., Motwani, R., and Harris, F. C. (2004). A survey of image denoising techniques. In Global Signal Processing Expo and Conference.
  63. Mutza, D. (2006). New fujifilm image intelligence: The next generation of automatic image quality optimization. In International Congress of Imaging Science. Fuji Photo Film (USA).
  64. Neelamani, R., Choi, H., and Baraniuk, R. (2004). Forward: Fourier-wavelet regularized deconvolution for ill-conditioned systems. IEEE Trans. on Signal Processing, 52.
  65. Novak, E., Jurie, F., and Triggs, B. (2006). Sampling strategies for bag-of-features image classification. In ECCV.
  66. Oberhardt, K., Taresch, G., and et al (2003). Method for the automatic detection of red-eye defects in photographic image data. US Patent Applications 20030044178,Milde & Hoffberg, L.L.P.
  67. O'Hare, N., Gurrin, C., Lee, H., Murphy, N., Smeaton, A. F., and Jones, G. J. (2005). My digital photos: where and when? In Annual ACM international conference on Multimedia.
  68. Perona, P. and Malik, J. (1990). Scale-space and edge detection using anisotropic diffusion. PAMI, 12(7).
  69. Perronnin, F., Dance, C., Csurka, G., and Bressan, M. (2006). Adapted vocabularies for generic visual categorization. In European Conf. on Computer Vision.
  70. Polesel, A., Ramponi, G., and Mathews, V. J. (2000). Image enhancement via adaptive unsharp masking. IEEE Trans. On Image Processing, 9(3).
  71. Portilla, J., Strela, V., Wainwright, M. J., and Simoncelli, E. P. (2003). Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12(11).
  72. Quelard, S. (2004). Image quality improvement for a cmos mobile phone digital camera. Technical report, KTH Stockholm Royal Innstitute of Technology.
  73. Quelhas, P., Monay, F., Odobez, J.-M., Gatica-Perez, D., Tuytelaars, T., and Gool, L. V. (2005). Modeling scenes with local descriptors and latent aspects. In ICCV.
  74. Ramamurthi, B. and Gersho, A. (1986). Nonlinear spacevariant postprocessing of block coded images. IEEE Trans. Acoust., Speech, Signal Processing, ASSP-34.
  75. Reeve, H. C. and Lim, J. S. (1984). Reduction of blocking effects in image coding. Optical Engineering, 23(1).
  76. Rosenfeld, A. and Kak, A. (1982). Digital picture processing. Academic Press, Inc., New York.
  77. Sadovsky, V., Yuan, P., Ivory, A. S., and Turner, R. (2004). Automatic analysis and adjustment of digital images upon acquisition. US Patent Application 20040258308, Microsoft.
  78. Saito, T., Harada, H., Satsumabayashi, J., and Komatsu, T. (2003). Color image sharpening based on nonlinear reaction-diffusion. In ICIP.
  79. Simon, R. and Matraszek, W. (2006). Method and system for enhancing portrait image that are processed in a batch mode. US Patent Application 7050636, Eastman Kodak Company (Rochester, NY).
  80. Sivic, J., Russell, B., Efros, A., Zisserman, A., and Freeman, W. (2005). Discovering objects and their locations in images. In ICCV.
  81. Stern, A., Kruchakov, I., Yoavi, E., and Kopeika, N. (2002). Recognition of motion-blurred images by use of the method of moments. Applied Optics, 41.
  82. Sudderth, E., Torralba, A., Freeman, W., and Willsky, A. (2006). Learning hierarchical models of scenes, objects, and parts. In ICCV.
  83. Tan, K. T. and Ghanbari, M. (2000). Blockiness detection for mpeg-2-coded video. IEEE Signal Processing Letters, 7.
  84. Tomasi, C. and Manduchi, R. (1998). Bilateral filtering for gray and color images. In ICCV.
  85. Tumblin, J. and Rushmeier, H. (1993). Tone reproduction for realistic images. IEEE Computer Graphics and Applications, 13(6).
  86. Tumblin, J. and Turk, G. (1999). Lcis: A boundary hierarchy for detail-preserving contrast reduction. In SIGGRAPH.
  87. Tzou, K. H. (1988). Post-filtering of transform-coded images. In SPIE: Applications of Digital Image Processing XI, volume 974.
  88. Utsugi, R. (2003). Method of correcting face image, makeup simulation method, makeup method makeup supporting device and foundation transfer film. US Patent 6502583, DRDC Limited (Tokyo, JP); Scalar Corporation (Tokyo, JP).
  89. Xerox (2006). Xerox's Automatic Image Enhancement System. http://www.xerox.com/downloads/usa/en/f/ FILE PROD AIE Brochure.pdf.
  90. Xiong, Z., Orchard, M. T., and Zhang, Y. Q. (1997). A deblocking algorithm for jpeg compressed images using overcomplete wavelet representations. IEEE Trans. Circuits and Systems for Video Technology, 7(4).
  91. Yang, M.-H., Kriegman, D., and Ahuja, N. (2002). Detecting faces in images: A survey. PAMI, 24(1).
  92. Zafarifar, B. and de With, P. H. N. (2006). Blue sky detection for picture quality enhancement. In Advanced Concepts for Intelligent Vision Systems.
  93. Zhang, Y., Wen, C., and Zhang, Y. (2000). Estimation of motion, parameters from blurred images. Pattern Recognition Letters, 21.
  94. Zhao, W., Chellappa, R., Phillips, P., and Rosenfeld, A. (2003). Face recognition: A literature survey. ACM Comput. Surv., 35.
  95. Zuiderveld, K. (1994). Contrast limited adaptive histogram equalization. In Press, A., editor, Graphic Gems IV.
Download


Paper Citation


in Harvard Style

Bressan M., Csurka G. and Favre S. (2007). TOWARDS INTENT DEPENDENT IMAGE ENHANCEMENT - State-of-the-art and Recent Attempts . In Proceedings of the Second International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, ISBN 978-972-8865-74-0, pages 208-218. DOI: 10.5220/0002068202080218


in Bibtex Style

@conference{visapp07,
author={Marco Bressan and Gabriela Csurka and Sebastien Favre},
title={TOWARDS INTENT DEPENDENT IMAGE ENHANCEMENT - State-of-the-art and Recent Attempts},
booktitle={Proceedings of the Second International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP,},
year={2007},
pages={208-218},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002068202080218},
isbn={978-972-8865-74-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Second International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP,
TI - TOWARDS INTENT DEPENDENT IMAGE ENHANCEMENT - State-of-the-art and Recent Attempts
SN - 978-972-8865-74-0
AU - Bressan M.
AU - Csurka G.
AU - Favre S.
PY - 2007
SP - 208
EP - 218
DO - 10.5220/0002068202080218