Pose Interpolation for Rolling Shutter Cameras using Non Uniformly Time-Sampled B-splines

Bertrand Vandeportaele, Philippe-Antoine Gohard, Michel Devy, Benjamin Coudrin

Abstract

Rolling Shutter (RS) cameras are predominant in the tablet and smartphone market due to their low cost and small size. However, these cameras require specific geometric models when either the camera or the scene is in motion to account for the sequential exposure of the different lines of the image. This paper proposes to improve a state-of-the-art model for RS cameras through the use of Non Uniformly Time-Sampled B-splines. This allows to interpolate the pose of the camera taking into account the varying dynamic of the motion by adding more control points where needed while keeping a low number of control points where the motion is smooth. Two methods are proposed to determine adequate distributions for the control points, using either an IMU sensor or an iterative reprojection error minimization. Results on simple synthetic data sets are shown to prove the concept and future works are introduced that should lead to the integration of our model in a SLAM algorithm.

References

  1. Davison, A. J. (2003). Real-time simultaneous localisation and mapping with a single camera. In 9th IEEE International Conference on Computer Vision (ICCV 2003), 14-17 October 2003, Nice, France, pages 1403-1410.
  2. Engel, J., Sch öps, T., and Cremers, D. (2014). LSD-SLAM: Large-scale direct monocular SLAM. In ECCV.
  3. Furgale, P., Barfoot, T. D., and Sibley, G. (2012). Continuous-time batch estimation using temporal basis functions. In Robotics and Automation (ICRA), 2012 IEEE International Conference on, pages 2088- 2095.
  4. Gonzalez, A. (2013). Localisation par vision multispectrale. Application aux systèmes embarqués. Theses, INSA de Toulouse.
  5. Hartley, R. I. and Zisserman, A. (2004). Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition.
  6. Hedborg, J., Forssén, P. E., Felsberg, M., and Ringaby, E. (2012). Rolling shutter bundle adjustment. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 1434-1441.
  7. Hedborg, J., Ringaby, E., Forssén, P. E., and Felsberg, M. (2011). Structure and motion estimation from rolling shutter video. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pages 17-23.
  8. Klein, G. and Murray, D. (2007). Parallel tracking and mapping for small AR workspaces. In Proc. Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'07), Nara, Japan.
  9. Klein, G. and Murray, D. (2009). Parallel tracking and mapping on a camera phone. In Proc. Eigth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'09), Orlando.
  10. Li, M., Kim, B., and Mourikis, A. I. (2013). Real-time motion estimation on a cellphone using inertial sensing and a rolling-shutter camera. In Proceedings of the IEEE International Conference on Robotics and Automation, pages 4697-4704, Karlsruhe, Germany.
  11. Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., and Sayd, P. (2006). Real time localization and 3d reconstruction. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), volume 1, pages 363-370.
  12. Mur-Artal, R., Montiel, J. M. M., and Tardós, J. D. (2015). Orb-slam: A versatile and accurate monocular slam system. IEEE Transactions on Robotics, 31(5):1147- 1163.
  13. Patron-Perez, A., Lovegrove, S., and Sibley, G. (2015). A spline-based trajectory representation for sensor fusion and rolling shutter cameras. Int. J. Comput. Vision, 113(3):208-219.
  14. Roussillon, C., Gonzalez, A., Solà, J., Codol, J., Mansard, N., Lacroix, S., and Devy, M. (2012). RT-SLAM: A generic and real-time visual SLAM implementation. CoRR, abs/1201.5450.
  15. Steven Lovegrove, Alonso Patron-Perez, G. S. (2013). Spline fusion: A continuous-time representation for visual-inertial fusion with application to rolling shutter cameras. In Proceedings of the British Machine Vision Conference. BMVA Press.
  16. Strasdat, H., Montiel, J., and Davison, A. J. (2010). Realtime monocular slam: Why filter? In Robotics and Automation (ICRA), 2010 IEEE International Conference on, pages 2657-2664. IEEE.
Download


Paper Citation


in Harvard Style

Vandeportaele B., Gohard P., Devy M. and Coudrin B. (2017). Pose Interpolation for Rolling Shutter Cameras using Non Uniformly Time-Sampled B-splines . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-227-1, pages 286-293. DOI: 10.5220/0006171802860293


in Bibtex Style

@conference{visapp17,
author={Bertrand Vandeportaele and Philippe-Antoine Gohard and Michel Devy and Benjamin Coudrin},
title={Pose Interpolation for Rolling Shutter Cameras using Non Uniformly Time-Sampled B-splines},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={286-293},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006171802860293},
isbn={978-989-758-227-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)
TI - Pose Interpolation for Rolling Shutter Cameras using Non Uniformly Time-Sampled B-splines
SN - 978-989-758-227-1
AU - Vandeportaele B.
AU - Gohard P.
AU - Devy M.
AU - Coudrin B.
PY - 2017
SP - 286
EP - 293
DO - 10.5220/0006171802860293