Authors:
Nicholas Beato
;
Remo Pillat
and
Charles E. Hughes
Affiliation:
University of Central Florida, United States
Keyword(s):
Mixed Reality, Image Matting, Natural Image Matting, Real-time, Depth Sensor, Chroma-keying, Trimap Generation.
Related
Ontology
Subjects/Areas/Topics:
Augmented, Mixed and Virtual Environments
;
Computer Vision, Visualization and Computer Graphics
;
Hardware Technologies for Augmented, Mixed and Virtual Environments
;
Interactive Environments
;
Real-Time Graphics
;
Real-Time Rendering
;
Rendering
Abstract:
In Mixed Reality scenarios, background replacement is a common way to immerse a user in a synthetic environment. Properly identifying the background pixels in an image or video is a difficult problem known as matting. Proper alpha mattes usually come from human guidance, special hardware setups, or color dependent algorithms. This is a consequence of the under-constrained nature of the per pixel alpha blending equation. While the field of natural image matting has made progress finding a least squares solution for an alpha matte, the generation of trimaps, indicating regions of known foreground and background pixels, normally requires human interaction or offline computation. We overcome these limitations by combining a low fidelity depth image that segments the original video signal with a real-time parallel natural image matting technique that favors objects with similar colors in the foreground and background. This allows us to provide real-time alpha mattes for Mixed Reality scen
arios that take place in relatively controlled environments. As a consequence, while monochromatic backdrops (such as green screens or retro-reflective material) aid the algorithm’s accuracy, they are not an explicit requirement.
(More)