WHAT DO OBJECTS FEEL LIKE? - Active Perception for a Humanoid Robot

Jens Kleesiek, Stephanie Badde, Stefan Wermter, Andreas K. Engel


We present a recurrent neural architecture with parametric bias for actively perceiving objects. A humanoid robot learns to extract sensorimotor laws and based on those to classify eight objects by exploring their multi-modal sensory characteristics. The network is either trained with prototype sequences for all objects or just two objects. In both cases the network is able to self-organize the parametric bias space into clusters representing individual objects and due to that, discriminates all eight categories with a very low error rate. We show that the network is able to retrieve stored sensory sequences with a high accuracy. Furthermore, trained with only two objects it is still able to generate fairly accurate sensory predictions for unseen objects. In addition, the approach proves to be very robust against noise.


  1. Aloimonos, J., Weiss, I., and Bandyopadhyay, A. (1988). Active vision. International Journal of Computer Vision, 1:333-356.
  2. Bajcsy, R. (1988). Active perception. Proceedings of the IEEE, 76(8):966 -1005.
  3. Ballard, D. H. (1991). Animate vision. Artificial Intelligence, 48(1):57-86.
  4. Bradski, G. (2000). The OpenCV Library. Dr. Dobb's Journal of Software Tools.
  5. Bridgeman, B. and Tseng, P. (2011). Embodied cognition and the perception-action link. Phys Life Rev, 8(1):73-85.
  6. Burt, P. (1988). Smart sensing within a pyramid vision machine. Proceedings of the IEEE, 76(8):1006 -1015.
  7. Chang, C.-C. and Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27.
  8. Cuijpers, R. H., Stuijt, F., and Sprinkhuizen-Kuyper, I. G. (2009). Generalisation of action sequences in RNNPB networks with mirror properties. In Proceedings of the 17th European symposium on Artifical Neural Networks (ESANN), pages 251-256.
  9. Dewey, J. (1896). The reflex arc concept in psychology. Psychological Review, 3:357-370.
  10. Fitzpatrick, P. and Metta, G. (2003). Grounding vision through experimental manipulation. Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 361(1811):2165-2185.
  11. Gibson, J. J. (1977). The theory of affordances. In Shaw, R. and Bransford, J., editors, Perceiving, acting, and knowing: Toward an ecological psychology, pages 67-82. Hillsdale, NJ: Erlbaum.
  12. Held, R., Ostrovsky, Y., Degelder, B., Gandhi, T., Ganesh, S., Mathur, U., and Sinha, P. (2011). The newly sighted fail to match seen with felt. Nat Neurosci, 14(5):551-3.
  13. Hu, M.-K. (1962). Visual pattern recognition by moment invariants. Information Theory, IRE Transactions on, 8(2):179 -187.
  14. Kolen, J. F. and Kremer, S. C. (2001). A field guide to dynamical recurrent networks. IEEE Press, New York.
  15. LeCun, Y., Bottou, L., Orr, G., and Mü ller, K. (1998). Efficient backprop. Lecture Notes in Computer Science, 1524:5-50.
  16. Martín H., J. A., Santos, M., and de Lope, J. (2010). Orthogonal variant moments features in image analysis. Inf. Sci., 180:846-860.
  17. Merleau-Ponty, M. (1963). The structure of behavior. Beacon Press, Boston.
  18. Ogata, T., Ohba, H., Tani, J., Komatani, K., and Okuno, H. G. (2005). Extracting multi-modal dynamics of objects using RNNPB. Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Edmonton, pages 160- 165.
  19. Olsson, L. A., Nehaniv, C. L., and Polani, D. (2006). From unknown sensors and actuators to actions grounded in sensorimotor perceptions. Connection Science, 18(2):121-144.
  20. O'Regan, J. K. and Noë, A. (2001). A sensorimotor account of vision and visual consciousness. Behav Brain Sci, 24(5):939-73; discussion 973-1031.
  21. Patel, K., Macklem, W., Thrun, S., and Montemerlo, M. (2005). Active sensing for high-speed offroad driving. In Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International Conference on, pages 3162 - 3168.
  22. Pfeifer, R., Lungarella, M., and Iida, F. (2007). Selforganization, embodiment, and biologically inspired robotics. Science, 318(5853):1088-93.
  23. Rasolzadeh, B., Björkman, M., Huebner, K., and Kragic, D. (2009). An active vision system for detecting, fixating and manipulating objects in real world. The International Journal of Robotics Research.
  24. Riedmiller, M. and Braun, H. (1993). A direct adaptive method for faster backpropagation learning: the RPROP algorithm. In Neural Networks, 1993., IEEE International Conference on, pages 586 -591 vol.1.
  25. Suzuki, S. and Be, K. (1985). Topological structural analysis of digitized binary images by border following. Computer Vision, Graphics, and Image Processing, 30(1):32-46.
  26. Tani, J. and Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 33(4):481 - 488.
  27. Tani, J., Ito, M., and Sugita, Y. (2004). Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using rnnpb. Neural Netw, 17(8-9):1273-89.
  28. Varela, F. J., Thompson, E., and Rosch, E. (1991). The embodied mind: cognitive science and human experience. MIT Press, Cambridge, Mass.

Paper Citation

in Harvard Style

Kleesiek J., Badde S., Wermter S. and K. Engel A. (2012). WHAT DO OBJECTS FEEL LIKE? - Active Perception for a Humanoid Robot . In Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-8425-95-9, pages 64-73. DOI: 10.5220/0003729900640073

in Bibtex Style

author={Jens Kleesiek and Stephanie Badde and Stefan Wermter and Andreas K. Engel},
title={WHAT DO OBJECTS FEEL LIKE? - Active Perception for a Humanoid Robot},
booktitle={Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},

in EndNote Style

JO - Proceedings of the 4th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - WHAT DO OBJECTS FEEL LIKE? - Active Perception for a Humanoid Robot
SN - 978-989-8425-95-9
AU - Kleesiek J.
AU - Badde S.
AU - Wermter S.
AU - K. Engel A.
PY - 2012
SP - 64
EP - 73
DO - 10.5220/0003729900640073