System and methods for depth regularization and semiautomatic interactive matting using RGB-D images
First Claim
1. An array camera, comprising:
- a plurality of cameras that capture images of a scene from different viewpoints;
memory containing an image processing pipeline application;
wherein the image processing pipeline application direct the processor to;
capture a set of images using a group of cameras from the plurality of cameras;
receive (i) an image comprising a plurality of pixel color values for pixels in the image and (ii) an initial depth map corresponding to the depths of the pixels within the image, wherein the initial depth map is generated using the set of images; and
regularize the initial depth map into a dense depth map using pixels for which depth is known to estimate depths of pixels for which depth is unknown by using affine combinations of the depths of nearby known pixels to compute depths for the unknown pixel depths, wherein regularizing the initial depth map into the dense depth map further comprises performing Laplacian matting to compute a Laplacian L, wherein the Laplacian matting is optimized by solving a reduced linear system for depth values only in unknown regions;
wherein regularizing the initial depth map into the dense depth map further comprises;
finding an approximate dense depth map using a diffusion process where LD is a diffusion Laplacian constructed such that each pixel is connected to a plurality of its surrounding neighbors using spatial proximity;
pruning the Laplacian L based upon the approximate dense depth map; and
detecting and correcting depth bleeding across edges by computing a Laplacian residual R based upon the pruned Laplacian and removing incorrect depth values based on the Laplacian residual R; and
using the dense depth map to perform image-based rendering.
14 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods in accordance with embodiments of this invention perform depth regularization and semiautomatic interactive matting using images. In an embodiment of the invention, the image processing pipeline application directs a processor to receive (i) an image (ii) an initial depth map corresponding to the depths of pixels within the image, regularize the initial depth map into a dense depth map using depth values of known pixels to compute depth values of unknown pixels, determine an object of interest to be extracted from the image, generate an initial trimap using the dense depth map and the object of interest to be extracted from the image, and apply color image matting to unknown regions of the initial trimap to generate a matte for image matting.
1080 Citations
17 Claims
-
1. An array camera, comprising:
-
a plurality of cameras that capture images of a scene from different viewpoints; memory containing an image processing pipeline application; wherein the image processing pipeline application direct the processor to; capture a set of images using a group of cameras from the plurality of cameras; receive (i) an image comprising a plurality of pixel color values for pixels in the image and (ii) an initial depth map corresponding to the depths of the pixels within the image, wherein the initial depth map is generated using the set of images; and regularize the initial depth map into a dense depth map using pixels for which depth is known to estimate depths of pixels for which depth is unknown by using affine combinations of the depths of nearby known pixels to compute depths for the unknown pixel depths, wherein regularizing the initial depth map into the dense depth map further comprises performing Laplacian matting to compute a Laplacian L, wherein the Laplacian matting is optimized by solving a reduced linear system for depth values only in unknown regions; wherein regularizing the initial depth map into the dense depth map further comprises;
finding an approximate dense depth map using a diffusion process where LD is a diffusion Laplacian constructed such that each pixel is connected to a plurality of its surrounding neighbors using spatial proximity;pruning the Laplacian L based upon the approximate dense depth map; and detecting and correcting depth bleeding across edges by computing a Laplacian residual R based upon the pruned Laplacian and removing incorrect depth values based on the Laplacian residual R; and using the dense depth map to perform image-based rendering. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
Specification