Illumination Estimation and Relighting using an RGB-D Camera

Yohei Ogura, Takuya Ikeda, Francois de Sorbier, Hideo Saito

2015

Abstract

In this paper, we propose a relighting system combined with an illumination estimation method using RGBD camera. Relighting techniques can achieve the photometric registration of composite images. They often need illumination environments of the scene which include a target object and the background scene. Some relighting methods obtain the illumination environments beforehand. In this case, they cannot be used under the unknown dynamic illumination environment. Some on-line illumination estimation methods need light probes which can be invade the scene geometry. In our method, the illumination environment is estimated from pixel intensity, normal map and surface reflectance based on inverse rendering in on-line processing. The normal map of the arbitrary object which is used in the illumination estimation part and the relighting part is calculated from the denoised depth image on each frame. Relighting is achieved by calculating the ratio for the estimated Illumination environment of the each scene. Thus our implementation can be used for dynamic illumination or a dynamic object.

References

  1. Aldrian, O. and Smith, W. (2011). Inverse rendering with a morphable model: A multilinear approach. In Proceedings of the British Machine Vision Conference. BMVA Press.
  2. Debevec, P. (2008). Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In ACM SIGGRAPH. ACM.
  3. Debevec, P., Hawkins, T., Tchou, C., Duiker, H. P., Sarokin, W., and Sagar, M. (2000). Acquiring the reflectance field of a human face. In ACM SIGGRAPH. ACM.
  4. Gruber, L., Richter-Trummer, T., and Schmalstieg, D. (2011). Real-time photometric registration from arbitrary geometry. In IEEE Int'l Symposium on Mixed and Augmented Reality. IEEE.
  5. Holzer, S., Rusu, R. B., Dixon, M., Gedikli, S., and Navab, N. (2012). Adaptive neighborhood selection for realtime surface normal estimation from organized point cloud data using integral images. In IEEE/RSJ Int'l Conference on Intelligent Robots and Systems. IEEE.
  6. Matsumoto, K., Song, C., de Sorbier, F., and Saito, H. (2014). Joint upsampling and noise reduction for real-time depth map enhancement. In Proceedings of IS&T/SPIE Electronic Imaging. SPIE.
  7. Microsoft Corporation (2012). Kinect for Windows. Microsoft.
  8. Nowrouzezahrai, D., Geiger, S., Mitchell, K., Sumner, R., Jarosz, W., and Gross, M. (2011). Light factorization for mixed-frequency shadows in augmented reality. In IEEE Int'l Symposium on Mixed and Augmented Reality. IEEE.
  9. Nowrouzezahrai, D., Simari, P., and Fiume, E. (2012). Sparse zonal harmonic factorization for efficient sh rotation. ACM Transactionson Graphic.
  10. Panagopoulos, A., Wang, C., Samaras, D., and Paragios, N. (2011). Illumination estimation and cast shadow detection through a higher-order graphical model. In IEEE Conference on Computer Vision and Pattern Recognition. IEEE.
  11. Ramamoorthi, R. and Hanrahan, P. (2001a). An efficient representation for irradiance environment maps. In ACM SIGGRAPH. ACM.
  12. Ramamoorthi, R. and Hanrahan, P. (2001b). A signalprocessing framework for inverse rendering. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM.
  13. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. (2005). Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Transactions on Graphics.
  14. Zhen, W., Liu, Z., and Huang, T. S. (2003). Face relighting with radiance environment maps. In IEEE Conference on Computer Vision and Pattern Recognition. IEEE.
Download


Paper Citation


in Harvard Style

Ogura Y., Ikeda T., de Sorbier F. and Saito H. (2015). Illumination Estimation and Relighting using an RGB-D Camera . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015) ISBN 978-989-758-090-1, pages 305-312. DOI: 10.5220/0005295403050312


in Bibtex Style

@conference{visapp15,
author={Yohei Ogura and Takuya Ikeda and Francois de Sorbier and Hideo Saito},
title={Illumination Estimation and Relighting using an RGB-D Camera},
booktitle={Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)},
year={2015},
pages={305-312},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005295403050312},
isbn={978-989-758-090-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP, (VISIGRAPP 2015)
TI - Illumination Estimation and Relighting using an RGB-D Camera
SN - 978-989-758-090-1
AU - Ogura Y.
AU - Ikeda T.
AU - de Sorbier F.
AU - Saito H.
PY - 2015
SP - 305
EP - 312
DO - 10.5220/0005295403050312