A Saliency-based Framework for 2D-3D Registration

Mark Brown, Jean-Yves Guillemaut, David Windridge

2014

Abstract

Here we propose a saliency-based filtering approach to the problem of registering an untextured 3D object to a single monocular image. The principle of saliency can be applied to a range of modalities and domains to find intrinsically descriptive entities from amongst detected entities, making it a rigorous approach to multi-modal registration. We build on the Kadir-Brady saliency framework due to its principled information-theoretic approach which enables us to naturally extend it to the 3D domain. The salient points from each domain are initially aligned using the SoftPosit algorithm. This is subsequently refined by aligning the silhouette with contours extracted from the image. Whereas other point based registration algorithms focus on corners or straight lines, our saliency-based approach is more general as it is more widely applicable e.g. to curved surfaces where a corner detector would fail. We compare our salient point detector to the Harris corner and SIFT keypoint detectors and show it generally achieves superior registration accuracy.

References

  1. Arvo, J. (1992). Fast random rotation matrices. pages 117- 120.
  2. Chung, A. J., Deligianni, F., Hu, X.-P., and Yang, G.-Z. (2004). Visual feature extraction via eye tracking for saliency driven 2D/3D registration. In Proc. of the 2004 symposium on Eye tracking research & applications, ETRA 7804, pages 49-54.
  3. David, P., DeMenthon, D., Duraiswami, R., and Samet, H. (2002). Softposit: Simultaneous pose and correspondence determination. In Proc. of the 7th European Conference on Computer Vision-Part III, ECCV 7802, pages 698-714.
  4. Egnal, G. (2000). Mutual information as a stereo correspondence measure. Technical report, Univ. of Pennsylvania.
  5. Enqvist, O., Josephson, K., and Kahl, F. (2009). Optimal correspondences from pairwise constraints. In International Conference on Computer Vision.
  6. Fitzgibbon, A. W. (2001). Robust registration of 2D and 3D point sets. In British Machine Vision Conference, pages 662-670.
  7. Gendrin, C., Furtado, H., Weber, C., Bloch, C., Figl, M., Pawiro, S. A., Bergmann, H., Stock, M., Fichtinger, G., Georg, D., and Birkfellner, W. (2011). Monitoring tumor motion by real time 2d/3d registration during radiotherapy. Radiother Oncol.
  8. Granger, S. and Pennec, X. (2002). Multi-scale em-icp: A fast and robust approach for surface registration. In European Conference on Computer Vision (ECCV 2002), volume 2353 of LNCS, pages 418-432.
  9. Guillemaut, J.-Y. and Hilton, A. (2011). Joint multi-layer segmentation and reconstruction for free-viewpoint video applications. Int. J. Comput. Vision, 93(1):73- 100.
  10. Huynh, D. Q. (2009). Metrics for 3d rotations: Comparison and analysis. J. Math. Imaging Vis., 35(2):155-164.
  11. Itti, L. (2000). A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Research, 40(10-12):1489-1506.
  12. Judd, T., Durand, F., and Torralba, A. (2012). A benchmark of computational models of saliency to predict human fixations. Technical report, MIT.
  13. Kadir, T. and Brady, M. (2001). Saliency, scale and image description. Int. J. Comput. Vision, 45(2):83-105.
  14. Kotsas, P. and Dodd, T. (2011). A review of methods for 2d/3d registration. World Academy of Science, Engineering and Technology.
  15. Lee, C. H., Varshney, A., and Jacobs, D. W. (2005). Mesh saliency. ACM Trans. Graph., 24(3):659-666.
  16. Mastin, A., Kepner, J., and Fisher III, J. W. (2009). Automatic registration of lidar and optical images of urban scenes. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on.
  17. Moreno-Noguer, F., Lepetit, V., and Fua, P. (2008). Pose priors for simultaneously solving alignment and correspondence. In Proc. of the 10th European Conference on Computer Vision: Part II, ECCV 7808, pages 405-418.
  18. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1, CVPR 7806, pages 519-528.
  19. Shao, L., Kadir, T., and Brady, M. (2007). Geometric and photometric invariant distinctive regions detection. Inf. Sci., 177(4):1088-1122.
  20. Taubin, G. (1995). Estimating the tensor of curvature of a surface from a polyhedral approximation. In Proceedings of the Fifth International Conference on Computer Vision, ICCV 7895.
  21. van de Kraats, E. B., Penney, G. P., Tomazevic, D., van Walsum, T., and Niessen, W. J. (2004). Standardized evaluation of 2d-3d registration. In MICCAI (1), pages 574-581.
  22. Wu, C., Clipp, B., Li, X., Frahm, J.-M., and Pollefeys, M. (2008). 3d model matching with viewpoint-invariant patches (VIP). In 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2008).
  23. Yang, W., Yi, D., Lei, Z., Sang, J., and Li, S. Z. (2008). 2D-3D Face Matching using CCA. In 8th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2008), pages 1-6.
Download


Paper Citation


in Harvard Style

Brown M., Guillemaut J. and Windridge D. (2014). A Saliency-based Framework for 2D-3D Registration . In Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2014) ISBN 978-989-758-003-1, pages 265-273. DOI: 10.5220/0004675402650273


in Bibtex Style

@conference{visapp14,
author={Mark Brown and Jean-Yves Guillemaut and David Windridge},
title={A Saliency-based Framework for 2D-3D Registration},
booktitle={Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2014)},
year={2014},
pages={265-273},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004675402650273},
isbn={978-989-758-003-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2014)
TI - A Saliency-based Framework for 2D-3D Registration
SN - 978-989-758-003-1
AU - Brown M.
AU - Guillemaut J.
AU - Windridge D.
PY - 2014
SP - 265
EP - 273
DO - 10.5220/0004675402650273