Global Patch Search Boosts Video Denoising

Thibaud Ehret, Pablo Arias, Jean-Michel Morel

2017

Abstract

With the increasing popularity of mobile imaging devices and the emergence of HdR video surveillance, the need for fast and accurate denoising algorithms has also increased. Patch-based methods, which are currently state-of-the-art in image and video denoising, search for similar patches in the signal. This search is generally performed locally around each target patch for obvious complexity reasons. We propose here a new and efficient approximate patch search algorithm. It permits for the first time to evaluate the impact of a global search on the video denoising performance. A global search is particularly justified in video denoising, where a strong temporal redundancy is often available. We first verify that the patches found by our new approximate search are far more concentrated than those obtained by exact local search, and are obtained in comparable time. To demonstrate the potential of the global search in video denoising, we take two patch-based image denoising algorithms and apply them to video. While with a classical local search their performance is poor, with the proposed global search they even improve the latest state-of-the-art video denoising methods.

References

  1. Andoni, A. and Indyk, P. (2006). Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In Foundations of Computer Science, 2006. FOCS'06. 47th Annual IEEE Symposium on, pages 459-468. IEEE.
  2. Barnes, C., Shechtman, E., Goldman, D. B., and Finkelstein, A. (2010). The generalized patchmatch correspondence algorithm. In Computer Vision-ECCV 2010, pages 29-43. Springer.
  3. Barnes, C., Zhang, F.-L., Lou, L., Wu, X., and Hu, S.-M. (2015). Patchtable: Efficient patch queries for large datasets and applications. In ACM Transactions on Graphics (Proc. SIGGRAPH).
  4. Bentley, J. L. (1975). Multidimensional binary search trees used for associative searching. Communications of the ACM, 18(9):509-517.
  5. Buades, A., Lisani, J. L., and Miladinovi, M. (2016). Patchbased video denoising with optical flow estimation. IEEE Transactions on Image Processing, 25(6):2573- 2586.
  6. Dabov, K., Foi, A., and Egiazarian, K. (2007a). Video denoising by sparse 3D transform-domain collaborative filtering. InEUSIPCO, pages 145-149.
  7. Dabov, K., Foi, A., and Egiazarian, K. (2007b). Video denoising by sparse 3D transform-domain collaborative filtering. In Proc. 15th European Signal Processing Conference, volume 1, page 7.
  8. Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K. (2007c). Image denoising by sparse 3-D transformdomain collaborative filtering. Image Processing, IEEE Transactions on, 16(8):2080-2095.
  9. Dabov, K., Foi, A., Katkovnik, V., and Egiazarian, K. (2007d). Image denoising by sparse 3d transformdomain collaborative filtering. IEEE Trans. on IP, 16(8):2080-2095.
  10. He, K. and Sun, J. (2012). Computing Nearest-Neighbor Fields via Propagation-Assisted KD-trees. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 111-118. IEEE.
  11. Korman, S. and Avidan, S. (2011). Coherency sensitive hashing. In Computer Vision (ICCV), 2011 IEEE International Conference on, pages 1607-1614. IEEE.
  12. Kumar, N., Zhang, L., and Nayar, S. (2008). What is a good nearest neighbors algorithm for finding similar patches in images? In Computer Vision-ECCV 2008 , pages 364-378. Springer.
  13. Lebrun, M. (2012). An Analysis and Implementation of the BM3D Image Denoising Method. Image Processing On Line, 2:175-213.
  14. Lebrun, M., Buades, A., and Morel, J.-M. (2013a). A Nonlocal Bayesian Image Denoising Algorithm. SIAM Journal on Imaging Sciences, 6(3):1665-1688.
  15. Lebrun, M., Buades, A., and Morel, J.-M. (2013b). Implementation of the “Non-Local Bayes” (NL-Bayes) Image Denoising Algorithm. Image Processing On Line, 3:1-42.
  16. Li, W., Zhang, J., and Dai, Q.-H. (2011). Video denoising using shape-adaptive sparse representation over similar spatio-temporal patches. Signal Processing: Image Communication, 26:250-265.
  17. Liu, C. and Freeman, W. T. (2010a). A high-quality video denoising algorithm based on reliable motion estimation. In ECCV, pages 706-719.
  18. Liu, C. and Freeman, W. T. (2010b). A high-quality video denoising algorithm based on reliable motion estimation. In Computer Vision-ECCV 2010 , pages 706- 719. Springer.
  19. Maggioni, M., Boracchi, G., Foi, A., and Egiazarian, K. (2012a). Video denoising, deblocking, and enhancement through separable 4-D nonlocal spatiotemporal transforms. IEEE Transactions on Image Processing, 21(9):3952-3966.
  20. Maggioni, M., Boracchi, G., Foi, A., and Egiazarian, K. (2012b). Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotemporal transforms. Image Processing, IEEE Transactions on, 21(9):3952-3966.
  21. Mairal, J., Bach, F., Ponce, J., Sapiro, G., and Zisserman, A. (2009). Non-local sparse models for image restoration. In Computer Vision, 2009 IEEE 12th International Conference on, pages 2272-2279.
  22. O'Hara, S., Draper, B., et al. (2013). Are you using the right approximate nearest neighbor algorithm? In Applications of Computer Vision (WACV), 2013 IEEE Workshop on, pages 9-14. IEEE.
  23. Olonetsky, I. and Avidan, S. (2012). TreeCANN - kdtree Coherence Approximate Nearest Neighbor algorithm. In Computer Vision-ECCV 2012 , pages 602- 615. Springer.
  24. Protter, M. and Elad, M. (2009). Image sequence denoising via sparse and redundant representations. IEEE Transactions on Image Processing, 18(1):27-35.
  25. Yianilos, P. N. (1993). Data structures and algorithms for nearest neighbor search in general metric spaces. In SODA, volume 93, pages 311-321.
Download


Paper Citation


in Harvard Style

Ehret T., Arias P. and Morel J. (2017). Global Patch Search Boosts Video Denoising . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-225-7, pages 124-134. DOI: 10.5220/0006175601240134


in Bibtex Style

@conference{visapp17,
author={Thibaud Ehret and Pablo Arias and Jean-Michel Morel},
title={Global Patch Search Boosts Video Denoising},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={124-134},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006175601240134},
isbn={978-989-758-225-7},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, (VISIGRAPP 2017)
TI - Global Patch Search Boosts Video Denoising
SN - 978-989-758-225-7
AU - Ehret T.
AU - Arias P.
AU - Morel J.
PY - 2017
SP - 124
EP - 134
DO - 10.5220/0006175601240134