Learning Visual Odometry with a Convolutional Network

Kishore Konda, Roland Memisevic

2015

Abstract

We present an approach to predicting velocity and direction changes from visual information (”visual odometry”) using an end-to-end, deep learning-based architecture. The architecture uses a single type of computational module and learning rule to extract visual motion, depth, and finally odometry information from the raw data. Representations of depth and motion are extracted by detecting synchrony across time and stereo channels using network layers with multiplicative interactions. The extracted representations are turned into information about changes in velocity and direction using a convolutional neural network. Preliminary results show that the architecture is capable of learning the resulting mapping from video to egomotion.

References

  1. Badino, H., Yamamoto, A., and Kanade, T. (2013). Visual odometry by multi-frame feature integration. In Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on, pages 222-229. IEEE.
  2. Bergstra, J., Breuleux, O., Bastien, F., Lamblin, P., Pascanu, R., Desjardins, G., Turian, J., Warde-Farley, D., and Bengio, Y. (2010). Theano: a CPU and GPU math expression compiler. In SciPy.
  3. Chandraker, M., Reddy, D., Wang, Y., and Ramamoorthi, R. (2013). What object motion reveals about shape with unknown brdf and lighting. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 2523-2530. IEEE.
  4. Eigen, D., Puhrsch, C., and Fergus, R. (2014). Depth map prediction from a single image using a multi-scale deep network. In Advances in neural information processing systems.
  5. Geiger, A., Lenz, P., and Urtasun, R. (2012). Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Computer Vision and Pattern Recognition (CVPR).
  6. Kitt, B., Geiger, A., and Lategahn, H. (2010). Visual odometry based on stereo image sequences with ransacbased outlier rejection scheme. In Intelligent Vehicles Symposium (IV), 2010 IEEE, pages 486-492. IEEE.
  7. Konda, K. R. and Memisevic, R. (2013). Unsupervised learning of depth and motion. CoRR, abs/1312.3429.
  8. Konda, K. R., Memisevic, R., and Michalski, V. (2014). Learning to encode motion using spatio-temporal synchrony. In Proceedings of ICLR.
  9. Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105.
  10. Memisevic, R. (2011). Gradient-based learning of higherorder image features. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1591- 1598. IEEE.
  11. Memisevic, R., Konda, K. R., and Krueger, D. (2014). Zerobias autoencoders and the benefits of co-adapting features. CoRR, abs/1402.3337.
  12. Nistér, D., Naroditsky, O., and Bergen, J. (2004). Visual odometry. In Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on, volume 1, pages I652. IEEE.
Download


Paper Citation


in Harvard Style

Konda K. and Memisevic R. (2015). Learning Visual Odometry with a Convolutional Network . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-089-5, pages 486-490. DOI: 10.5220/0005299304860490


in Bibtex Style

@conference{visapp15,
author={Kishore Konda and Roland Memisevic},
title={Learning Visual Odometry with a Convolutional Network},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={486-490},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005299304860490},
isbn={978-989-758-089-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2015)
TI - Learning Visual Odometry with a Convolutional Network
SN - 978-989-758-089-5
AU - Konda K.
AU - Memisevic R.
PY - 2015
SP - 486
EP - 490
DO - 10.5220/0005299304860490