(a) frame: 001 (b) frame: 350 (c) frame: 700 (d) Ground Truth
Figure 7: Comparison with Ground Truth.
6 CONCLUSION
In this paper, we proposed a relighting method which
combined with illumination estimation using RGB-D
camera. Before illumination estimation, we denoise
depth images by bilateral filter and temporal filter to
get smooth normal map. Based on inverse rendering,
Illumination environment is estimated from a color
image, normal map from a denoised depth image and
surface reflectance. After that, Relighting process is
done with estimated illumination data. Our method
estimates the illumination on each frame, and also ob-
tain normal map of the target object on each frame.
Therefore, our method can be applied to dynamic illu-
mination or dynamic target. In experiment, we tested
our method on two types of situation. We will im-
proveour method more robust, and also apply to spec-
ular objects.
ACKNOWLEDGEMENTS
This work was partially supported by MEXT/JSPS
Grant-in-Aid for Scientific Research(S) 24220004,
and JST CREST ”Intelligent Information Process-
ing Systems Creating Co-Experience Knowledge and
Wisdom with Human-Machine Harmonious Collabo-
ration”.
REFERENCES
Aldrian, O. and Smith, W. (2011). Inverse rendering with a
morphable model: A multilinear approach. In Pro-
ceedings of the British Machine Vision Conference.
BMVA Press.
Debevec, P. (2008). Rendering synthetic objects into real
scenes: Bridging traditional and image-based graph-
ics with global illumination and high dynamic range
photography. In ACM SIGGRAPH. ACM.
Debevec, P., Hawkins, T., Tchou, C., Duiker, H. P., Sarokin,
W., and Sagar, M. (2000). Acquiring the reflectance
field of a human face. In ACM SIGGRAPH. ACM.
Gruber, L., Richter-Trummer, T., and Schmalstieg, D.
(2011). Real-time photometric registration from ar-
bitrary geometry. In IEEE Int’l Symposium on Mixed
and Augmented Reality. IEEE.
Holzer, S., Rusu, R. B., Dixon, M., Gedikli, S., and Navab,
N. (2012). Adaptive neighborhood selection for real-
time surface normal estimation from organized point
cloud data using integral images. In IEEE/RSJ Int’l
Conference on Intelligent Robots and Systems. IEEE.
Matsumoto, K., Song, C., de Sorbier, F., and Saito, H.
(2014). Joint upsampling and noise reduction for
real-time depth map enhancement. In Proceedings of
IS&T/SPIE Electronic Imaging. SPIE.
Microsoft Corporation (2012). Kinect for Windows. Mi-
crosoft.
Nowrouzezahrai, D., Geiger, S., Mitchell, K., Sumner, R.,
Jarosz, W., and Gross, M. (2011). Light factorization
for mixed-frequency shadows in augmented reality. In
IEEE Int’l Symposium on Mixed and Augmented Re-
ality. IEEE.
Nowrouzezahrai, D., Simari, P., and Fiume, E. (2012).
Sparse zonal harmonic factorization for efficient sh ro-
tation. ACM Transactionson Graphic.
Panagopoulos, A., Wang, C., Samaras, D., and Paragios,
N. (2011). Illumination estimation and cast shadow
detection through a higher-order graphical model. In
IEEE Conference on Computer Vision and Pattern
Recognition. IEEE.
Ramamoorthi, R. and Hanrahan, P. (2001a). An efficient
representation for irradiance environment maps. In
ACM SIGGRAPH. ACM.
Ramamoorthi, R. and Hanrahan, P. (2001b). A signal-
processing framework for inverse rendering. In Pro-
ceedings of the 28th annual conference on Computer
graphics and interactive techniques. ACM.
Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins,
T., and Debevec, P. (2005). Performance relighting
and reflectance transformation with time-multiplexed
illumination. ACM Transactions on Graphics.
Zhen, W., Liu, Z., and Huang, T. S. (2003). Face relighting
with radiance environment maps. In IEEE Conference
on Computer Vision and Pattern Recognition. IEEE.
VISAPP2015-InternationalConferenceonComputerVisionTheoryandApplications
312