A SURVEY OF IMAGE-BASED RELIGHTING TECHNIQUES

Biswarup Choudhury, Sharat Chandran

2006

Abstract

Image-based Relighting (IBRL) has recently attracted a lot of research interest for its ability to relight real objects or scenes, from novel illuminations captured in natural/synthetic environments. Complex lighting effects such as subsurface scattering, interreflection, shadowing, mesostructural self-occlusion, refraction and other relevant phenomena can be generated using IBRL. The main advantage of Image-based graphics is that the rendering time is independent of scene complexity as the rendering is actually a process of manipulating image pixels, instead of simulating light transport. The goal of this paper is to provide a complete and systematic overview of the research in Image-based Relighting. We observe that essentially all IBRL techniques can be broadly classified into three categories, based on how the scene/illumination information is captured: Reflectance function based, Basis function based, and Plenoptic function based. We discuss the characteristics of each of these categories and their representative methods. We also discuss about sampling density and types of light source, relevant issues of IBRL.

References

  1. Adelson, E. H. and Bergen, J. R. (1991). The plenoptic function and the elements of early vision. Computational Models of Visual Processing, pages 3-20.
  2. Debevec, P., Hawkins, T., Tchou, C., Duiker, H.-P., Sarokin, W., and Sagar, M. (2000). Acquiring the reflectance field of a human face. In SIGGRAPH 7800: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 145-156, New York, NY, USA. ACM Press/Addison-Wesley Publishing Co.
  3. Dobashi, Y., Kaneda, K., Nakatani, H., and Yamashita, H. (1995). A quick rendering method using basis functions for interactive lighting design. Computer Graphics Forum, 14(3):229-240.
  4. Dumont, O., Masselus, V., Zaenen, P., Wagemans, J., and Dutre, P. (2005). A perceptual quality scale for imagebased relighting of glossy surfaces. In Tech. Rep. CW417, Katholieke Universiteit Leuven, June 2005.
  5. Fuchs, M., Blanz, V., and Seidel, H.-P. (2005). Bayesian relighting. In Rendering Techniques, pages 157-164.
  6. Furukawa, R., Kawasaki, H., Ikeuchi, K., and Sakauchi, M. (2002). Appearance based object modeling using texture database: acquisition, compression and rendering. In EGRW 7802: Proceedings of the 13th Eurographics workshop on Rendering, pages 257-266, Aire-la-Ville, Switzerland, Switzerland. Eurographics Association.
  7. Georghiades, A. S. (2003). Recovering 3-d shape and reflectance from a small number of photographs. In EGRW 7803: Proceedings of the 14th Eurographics workshop on Rendering, pages 230-240, Aire-laVille, Switzerland, Switzerland. Eurographics Association.
  8. Georghiades, A. S., Belhumeur, P. N., and Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell., 23(6):643- 660.
  9. Hawkins, T., Wenger, A., Tchou, C., Gardner, A., Göransson, F., and Debevec, P. E. (2004). Animatable facial reflectance fields. In Rendering Techniques, pages 309-321.
  10. Jensen, H. W., Marschner, S. R., Levoy, M., and Hanrahan, P. (2001). A practical model for subsurface light transport. In SIGGRAPH 7801: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 511-518, New York, NY, USA.
  11. Kajiya, J. T. (1985). Anisotropic reflection models. In SIGGRAPH 7885: Proceedings of the 12th annual conference on Computer graphics and interactive techniques, pages 15-21, New York, NY, USA. ACM Press.
  12. Kautz, J., Sloan, P.-P., and Snyder, J. (2002). Fast, arbitrary brdf shading for low-frequency lighting using spherical harmonics. In EGRW 7802: Proceedings of the 13th Eurographics workshop on Rendering, pages 291-296, Aire-la-Ville, Switzerland, Switzerland. Eurographics Association.
  13. Koudelka, M. L., Belhumeur, P. N., Magda, S., and Kriegman, D. J. (2001). Image-based modeling and rendering of surfaces with arbitrary brdfs. In CVPR (1), pages 568-575.
  14. Lensch, H. P. A., Kautz, J., Goesele, M., Heidrich, W., and Seidel, H.-P. (2003). Image-based reconstruction of spatial appearance and geometric detail. ACM Trans. Graph., 22(2):234-257.
  15. Lin, Z., Wong, T.-T., and Shum, H.-Y. (2002). Relighting with the reflected irradiance field: Representation, sampling and reconstruction. Int. J. Comput. Vision, 49(2-3):229-246.
  16. Malzbender, T., Gelb, D., and Wolters, H. (2001). Polynomial texture maps. In SIGGRAPH 7801: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 519-528, New York, NY, USA. ACM Press.
  17. Marschner, S. R. (1998). Inverse rendering for computer graphics. PhD thesis, Cornell University. AdviserDonald P. Greenberg.
  18. Masselus, V., Dutre;, P., and Anrys, F. (2002). The freeform light stage. In EGRW 7802: Proceedings of the 13th Eurographics workshop on Rendering, pages 247-256, Aire-la-Ville, Switzerland, Switzerland. Eurographics Association.
  19. Masselus, V., Peers, P., Dutre;, P., and Willems, Y. D. (2003). Relighting with 4d incident light fields. ACM Trans. Graph., 22(3):613-620.
  20. Masselus, V., Peers, P., Dutre, P., and Willems, Y. D. (2004). Smooth reconstruction and compact representation of reflectance functions for image-based relighting. In Rendering Techniques, pages 287-298.
  21. Matusik, W., Loper, M., and Pfister, H. (2004). Progressively-refined reflectance functions from natural illumination. In Rendering Techniques, pages 299-308.
  22. Matusik, W., Pfister, H., Ziegler, R., Ngan, A., and McMillan, L. (2002). Acquisition and rendering of transparent and refractive objects. In EGRW 7802: Proceedings of the 13th Eurographics workshop on Rendering, pages 267-278, Aire-la-Ville, Switzerland, Switzerland. Eurographics Association.
  23. Ng, R., Ramamoorthi, R., and Hanrahan, P. (2003). Allfrequency shadows using non-linear wavelet lighting approximation. ACM Trans. Graph., 22(3):376-381.
  24. Ng, R., Ramamoorthi, R., and Hanrahan, P. (2004). Triple product wavelet integrals for all-frequency relighting. ACM Trans. Graph., 23(3):477-487.
  25. Nicodemus, F., Richmond, J., Hsia, J. J., Ginsberg, I. W., and Limperis, T. (1977). Geometric considerations and nomenclature for reflectance. In NBS Monograph 160, National Bureau of Standards (US).
  26. Nimeroff, J. S., Simoncelli, E., and Dorsey, J. (1994). Efficient Re-rendering of Naturally Illuminated Environments. In Fifth Eurographics Workshop on Rendering, pages 359-373, Darmstadt, Germany. SpringerVerlag.
  27. Nishino, K. and Nayar, S. K. (2004). Eyes for relighting. ACM Trans. Graph., 23(3):704-711.
  28. Osadchy, M. and Keren, D. (2001). Image detection under varying illumination and pose. In ICCV, pages 668- 673.
  29. Peers, P. and Dutre, P. (2003). Wavelet environment matting. In EGRW 7803: Proceedings of the 14th Eurographics workshop on Rendering, pages 157-166, Aire-la-Ville, Switzerland, Switzerland. Eurographics Association.
  30. Ramamoorthi, R. and Hanrahan, P. (2001). An efficient representation for irradiance environment maps. In SIGGRAPH 7801: Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 497-500, New York, NY, USA. ACM Press.
  31. Shum, H.-Y. and Kang, S. B. (2000). A review of image-based rendering techniques. In IEEE/SPIE Visual Communications and Image Processing (VCIP), pages 2-13.
  32. Sloan, P.-P., Hall, J., Hart, J., and Snyder, J. (2003). Clustered principal components for precomputed radiance transfer. ACM Trans. Graph., 22(3):382-391.
  33. Sloan, P.-P., Kautz, J., and Snyder, J. (2002). Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments. In SIGGRAPH 7802: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 527-536, New York, NY, USA. ACM Press.
  34. Suykens, F., Vom, K., Lagae, A., and Dutre, P. (2003). Interactive rendering with bidirectional texture functions. Computer Graphics Forum, 22(3).
  35. Tchou, C., Stumpfel, J., Einarsson, P., Fajardo, M., and Debevec, P. (2004). Unlighting the parthenon. SIGGRAPH 2004 Sketch.
  36. Tong, X., Zhang, J., Liu, L., Wang, X., Guo, B., and Shum, H.-Y. (2002). Synthesis of bidirectional texture functions on arbitrary surfaces. In SIGGRAPH 7802: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 665-672, New York, NY, USA. ACM Press.
  37. Vasilescu, M. A. O. and Terzopoulos, D. (2004). Tensortextures: multilinear image-based rendering. ACM Trans. Graph., 23(3):336-342.
  38. Wang, R., Tran, J., and Luebke, D. (2005). All-frequency interactive relighting of translucent objects with single and multiple scattering. ACM Trans. Graph., 24(3):1202-1207.
  39. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. (2005). Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Trans. Graph., 24(3):756-764.
  40. Wong, T.-T. and Heng, P.-A. (2004). Image-based relighting: representation and compression. Integrated image and graphics technologies, pages 161-180.
  41. Wong, T.-T., Heng, P.-A., and Fu, C.-W. (2001). Interactive relighting of panoramas. IEEE Comput. Graph. Appl., 21(2):32-41.
  42. Wong, T.-T., Heng, P.-A., Or, S.-H., and Ng, W.-Y. (1997). Image-based rendering with controllable illumination. In Proceedings of the Eurographics Workshop on Rendering Techniques 7897, pages 13-22, London, UK. Springer-Verlag.
  43. Zhang, C. and Chen, T. (2001). Generalized plenoptic sampling. In Tech. Rep. AMP01-06, Carnegie Mellon Technical Report, 2001.
  44. Zhang, C. and Chen, T. (2003). Spectral analysis for sampling image-based rendering data. In IEEE Transaction on Circuit, System on Video Technology 13, 11 (2003), 1038- 1050. 1.
  45. Zhang, C. and Chen, T. (2004). A survey on image-based rendering: Representation, sampling and compression. SPIC 2004, 19(1):1-28.
  46. Zongker, D. E., Werner, D. M., Curless, B., and Salesin, D. H. (1999). Environment matting and compositing. In SIGGRAPH 7899: Proceedings of the 26th annual conference on Computer graphics and interactive techniques, pages 205-214, New York, NY, USA. ACM Press/Addison-Wesley Publishing Co.
Download


Paper Citation


in Harvard Style

Choudhury B. and Chandran S. (2006). A SURVEY OF IMAGE-BASED RELIGHTING TECHNIQUES . In Proceedings of the First International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP, ISBN 972-8865-39-2, pages 176-183. DOI: 10.5220/0001359201760183


in Bibtex Style

@conference{grapp06,
author={Biswarup Choudhury and Sharat Chandran},
title={A SURVEY OF IMAGE-BASED RELIGHTING TECHNIQUES},
booktitle={Proceedings of the First International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP,},
year={2006},
pages={176-183},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001359201760183},
isbn={972-8865-39-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the First International Conference on Computer Graphics Theory and Applications - Volume 1: GRAPP,
TI - A SURVEY OF IMAGE-BASED RELIGHTING TECHNIQUES
SN - 972-8865-39-2
AU - Choudhury B.
AU - Chandran S.
PY - 2006
SP - 176
EP - 183
DO - 10.5220/0001359201760183