MULTIMODAL SEARCH FOR GRAPHIC DESIGNERS

Sandra Skaff, David Rouquet, Emmanuel Dellandrea, Achille Falaise, Valérie Bellynck, Hervé Blanchon, Christian Boitet, Didier Schwab, Liming Chen, Alexandre Saidi, Gabriela Csurka, Luca Marchesotti

2011

Abstract

This paper describes OMNIA, a system and interface for searching in multimodal image collections. OMNIA includes a set of tools which allow the user to retrieve assets using different features. The tools are based on extracting different types of asset features, which are content, aesthetic, and emotion. Visual-based features are used to retrieve assets using each of these tools. In addition, text-based features can be used to retrieve image assets based on content. Different datasets are used in OMNIA and retrieved assets are displayed in such a way which facilitates user navigation. It is shown how OMNIA can be used for simple, efficient, and intuitive asset search in the context of graphic design applications.

References

  1. Ah-Pine, J., Clinchant, S., Csurka, G., and Liu, Y. (2009). XRCE's participation to ImageCLEF 2009. In Proc. of the Working Notes of the 2009 CLEF Workshop, Crete, Greece.
  2. Ah-Pine, J., Clinchant, S., Csurka, G., Perronnin, F., and Renders, J.-M. (2010). Leveraging image, text and cross-media similarities for diversity-focused multimedia retrieval. In The Information Retrieval Series. Springer.
  3. Barnard, K., Duygulu, P., de Freitas, N., Forsyth, D., Blei, D., and Jordan, M. (2004). Matching words and pictures. Journal of Machine Learning Research, 3:1107-1135.
  4. Boitet, C., Boguslavskij, I., and Cardeosa, J. (2009). An evaluation of UNL usability for high quality multilingualization and projections for a future UNL++ language. In Proc. of the International Conference on Computational Linguistics and Intelligent Text Processing, pages 361-373.
  5. Chen, Y. and Wang, J. (2002). A region-based fuzzy feature matching approach to content based image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24:1252-1267.
  6. Chum, O., Perdoch, M., and Matas, J. (2009). Geometric min-hashing: Finding a thick needle in a haystack. In Proc. of the IEEE Conference on Comptuer Vision and Pattern Recognition.
  7. Colmerauer, A. (1970). Les systmes-q ou un formalisme pour analyser et synthtiser des phrases sur ordinateur. dpartement d'informatique de l'Universit de Montral, publication interne, 43.
  8. Columbo, C., Bimbo, A. D., and Pala, P. (1999). Semantics in visual information retrieval. IEEE Multimedia, 6(3):38-53.
  9. Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. (2004). Visual categorization with bags of keypoints. In Proc. of the ECCV Workshop on Statistical Learning for Computer Vision.
  10. Csurka, G., Skaff, S., Marchesotti, L., and Saunders, C. (2010). Learning moods and emotions from color combinations. In Proc. of the Indian Conference on Computer Vision, Graphics, and Image Processing.
  11. Daoud, D. (2006). Il faut et on peut construire des systmes de commerce lectronique interface en langue naturelle restreints (et multilingues) en utilisant des mthodes orientes vers les sous-langages et le contenu. PhD thesis, Université Joseph Fourier.
  12. Datta, R., Joshi, D., Li, J., and Wang, J. (2008a). Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 40(2):1-60.
  13. Datta, R., Joshi, D., Li, J., and Wang, J. Z. (2006). Studying aesthetics in photographic images using a computational approach. In Proc. of the European Conference on Computer Vision, volume 3, pages 288-301.
  14. Datta, R., Li, J., and Wang, J. Z. (2008b). Algorithmic inferencing of aesthetics and emotion in natural images. In Proc. of the IEEE International Conference on Image Processing, San Diego, CA.
  15. Davis, B. and Lazebnik, S. (2008). Analysis of human attractiveness using manifold kernel regression. In Proc. of the IEEE International Conference on Image Processing.
  16. Dellandréa, E., Liu, N., and Chen, L. (2010). Classification of affective semantics in images based on discrete and dimensional models of emotions. Proc. of the International Workshop on Content-Based Multimedia Indexing, pages 1-6.
  17. Desolneux, A., Moisan, L., and Morel, J.-M. (2004). Seeing, Thinking and Knowing, chapter Gestalt Theory and Computer Vision, pages 71-101. A. Carsetti ed., Kluwer Academic Publishers.
  18. Dunker, P., Nowak, S., Begau, A., and Lanz, C. (2009). Content-based mood classification for photos and music. Proc. of the ACM International Conference on Multimedia Information Retrieval, pages 97-104.
  19. Falaise, A., Rouquet, D., Schwab, D., Blanchon, H., and Boitet, C. (2010). Ontology driven content extraction using interlingual annotation of texts in the OMNIA project. In Proc. of the International Workshop On Cross Lingual Information Access, Peking, China.
  20. Fedorovskaya, E., Neustaedter, C., and Wei, H. (2008). Image harmony for consumer images. In Proc. of the IEEE International Conference on Image Processing.
  21. Flickner, M., Sawhney, H., Niblack, W., Ashley, J., Huang, Q., Dom, B., Gorkani, M., Hafner, J., Lee, D., Petkovic, D., Steele, D., and Yanker, P. (1995). Query by image and video content: the QBIC system. IEEE Computer, 28:23-32.
  22. Guillaumin, M., Mensink, T., Verbeek, J., and Schmid, C. (2009). Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In Proc. of the IEEE International Conference on Computer Vision.
  23. Itten, J. (1961). The art of color. Otto Maier Verlab, Ravensburg, Germany.
  24. Jacobsen, T., Schubotz, R. I., Hfel, L., and v. Cramon, D. Y. (2006). Brain correlates of aesthetic judgment of beauty. NeuroImage, 29:276-285.
  25. Jegou, H., Douze, M., and Schmid, C. (2008). Hamming embedding and weak geometric consistency for large scale image search. In Proc. of the European Conference on Computer Vision.
  26. Jeon, J., Lavrenko, V., and Manmatha, R. (2003). Automatic image annotation and retrieval using crossmedia relevance models. In Proc. of the Annual ACM SIGIR conference on Research and development in informaion retrieval.
  27. Cardeosa et al. (2009). The sortium (accessed on september http://www.unl.fi.upm.es/consorcio/index.php.
  28. Kasutani, E. (2007). Image retrieval apparatus and image retrieving method, US Patent application.
  29. Laaksonen, J., Koskela, M., and Oja, E. (2002). Picsom self-organizing image retrieval with mpeg-7 content descriptions. IEEE Transactions on Neural Networks, 13:841-853.
  30. Li, X., Chen, L., Zhang, L., Lin, F., and Ma, W. (2006). Image annotation by large-scale content based image retrieval. In Proc. of the ACM International Conference on Multimedia.
  31. Loui, A., Wood, M. D., Scalise, A., and Birkelund, J. (2008). Multidimensional image value assessment and rating for automated albuming and retrieval. In Proc. of the IEEE International Conference on Image Processing.
  32. Monay, F. and Gatica-Perez, D. (2003). On image autoannotation with latent space models. In Proc. of the International Conference On Multimedia.
  33. Müller, H., Clough, P., Deselaers, T., and Caputo, B. (2010). Imageclef- experimental evaluation in visual information retrieval. In The Information Retrieval Series. Springer.
  34. Nowak, S. and Lukashevich, H. (2010). Multilabel classification evaluation using ontology information. In Proc. of the International Conference on Multimedia Information Retrieval, pages 35-44.
  35. Perronnin, F. and Dance, C. (2007). Fisher kernels on visual vocabularies for image categorization. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition.
  36. Perronnin, F., Liu, Y., Sanchez, J., and Poirier, H. (2010). Large-scale image retrieval with compressed fisher vectors. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition.
  37. Rouquet, D. and Nguyen, H. (2009). Multilingusation d'une ontologie par des corespondances avec un lexique pivot. In TOTh09, Annecy, France.
  38. Rouquet, D., Trojahn, C., Scwab, D., and Srasset, G. (2010). Building correspondences between ontologies and lexical resources. In to be published.
  39. Sahbi, H., Audibert, J., and Keriven, R. (2007). Graph-cut transducers for relevance feedback in content based image retrieval. In Proc. of the IEEE International Conference on Computer Vision.
  40. Smets, P. (1990). The combination of evidence in the transferable belief model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(5):447-458.
  41. Squire, D., W.M üller, H.Mü ller, and Raki, J. (1999). Content-based query of image databases, inspirations from text retrieval: Inverted files, frequency-based weights and relevance feedback. In Proc. of the Scandinavian Conference on Image Analysis.
  42. Swain, M. and Ballard, D. (1991). Color indexing. International Journal of Computer Vision, 7:11-32.
  43. Tahir, M., Kittler, J., Mikolajczyk, K., Yan, F., van de Sande, K., and Gevers, T. (2009). Visual category recognition using spectral regression and kernel discriminant analysis. In Proc. of the IEEE International Conference on Computer Vision Workshop on Subspace Methods.
  44. Tamura, H., Mori, S., and Yamawaki, T. (1978). Textural features corresponding to visual perception. IEEE Transaction on Systems, Man, and Cybernetics, 8(6):460.
  45. Tsujimura, K. and Bannai, Y. (1996). Image searching method and apparatus thereof using color information of an input image, US Patent application.
  46. Uchida Hiroshi et al. (2009). The UNDL foundation (accessed on september 2009). http://www.undl.org/.
  47. van de Sande, K. E. A., Gevers, T., , and Smeulders, A. W. M. (2009). The university of amsterdam's concept detection system at imageclef 2009. In Proc. of the Working Notes for the CLEF 2009 Workshop, Corfu, Greece.
  48. Wang, S. and Wang, X. (2005). Emotion semantics image retrieval: a brief overview. Proc. of the International Conference on Affective Computing and Intelligent Interaction, pages 490-497.
  49. Wang, W. and He, Q. (2008). A survey on emotional semantic image retrieval. Proc. of the IEEE International Conference on Image Processing, pages 117-120.
  50. Yang, J., Fan, J., Hubball, D., and Gao, Y. (2006). Semantic image browser: Bridging information visualization with automated intelligent image analysis. In Proc. of the IEEE Symposium on Visual Analytics Science And Technology.
  51. Zhang, J., Marszalek, M., Lazebnik, S., and Schmid, C. (2007). Local features and kernels for classification of texture and object categories: a comprehensive study. International Journal of Computer Vision, 73(2).
  52. Zhang, X. W. L., Jing, F., and Ma, W. (2006). Annosearch: Image auto-annotation by search. In Proc. of the IEEE
Download


Paper Citation


in Harvard Style

Skaff S., Rouquet D., Dellandrea E., Falaise A., Bellynck V., Blanchon H., Boitet C., Schwab D., Chen L., Saidi A., Csurka G. and Marchesotti L. (2011). MULTIMODAL SEARCH FOR GRAPHIC DESIGNERS . In Proceedings of the International Conference on Imaging Theory and Applications and International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2011) ISBN 978-989-8425-46-1, pages 164-176. DOI: 10.5220/0003322401640176


in Bibtex Style

@conference{ivapp11,
author={Sandra Skaff and David Rouquet and Emmanuel Dellandrea and Achille Falaise and Valérie Bellynck and Hervé Blanchon and Christian Boitet and Didier Schwab and Liming Chen and Alexandre Saidi and Gabriela Csurka and Luca Marchesotti},
title={MULTIMODAL SEARCH FOR GRAPHIC DESIGNERS},
booktitle={Proceedings of the International Conference on Imaging Theory and Applications and International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2011)},
year={2011},
pages={164-176},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003322401640176},
isbn={978-989-8425-46-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Imaging Theory and Applications and International Conference on Information Visualization Theory and Applications - Volume 1: IVAPP, (VISIGRAPP 2011)
TI - MULTIMODAL SEARCH FOR GRAPHIC DESIGNERS
SN - 978-989-8425-46-1
AU - Skaff S.
AU - Rouquet D.
AU - Dellandrea E.
AU - Falaise A.
AU - Bellynck V.
AU - Blanchon H.
AU - Boitet C.
AU - Schwab D.
AU - Chen L.
AU - Saidi A.
AU - Csurka G.
AU - Marchesotti L.
PY - 2011
SP - 164
EP - 176
DO - 10.5220/0003322401640176