Fast adaptive edge-aware matting

• US 8,879,835 B2
• Filed: 05/29/2012
• Issued: 11/04/2014
• Est. Priority Date: 08/26/2011
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Krishnaswamy, Aravind, Tao, Michael W.
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US13/482,767
• Publication Number:
  US 20130051663A1
• Time in Patent Office: 889 Days
• Field of Search: 382/164, 382/264, 382/115, 382/124, 382/125, 382/155, 382/162, 382/190, 382/154, 382/275, 382/274, 345/420, 345/589, 345/88
• US Class Current: 382/164
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/20152 Watershed segmentation
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/155 involving morphological operators
• CPC Class Codes: H04N 1/4092 Edge or detail enhancement
• CPC Class Codes: H04N 1/6047 wherein the test pattern is part of an arbitrary user image

Abstract

Methods, apparatus, and computer-readable storage media for fast adaptive edge-aware matting in which a matting technique adaptively feathers selections, provides smooth color correspondence matting, and performs well in textured regions. The matting technique may require fewer strokes and less parameter tuning than conventional matting techniques. The matting technique may have two components implemented in a matting pipeline. A color similarity component implements a color similarity constraint technique based on a radial basis function (RBF) technique to generate a color-constrained mask, and a locality constraint component implements a locality constraint technique based on a fast flood fill technique to generate a locality-constrained mask. The final mask (or matte) output may be an element multiply of the masks generated by the two components.

Claims

1. A method, comprising:

performing, by one or more computing devices;

obtaining an image and a specified region of the image that indicates a color selection to be used in generating a mask for the image;

generating a color constraint mask for the image by;

generating a set of samples according to an importance sampling technique that weights selection of samples towards areas in the image that include a dominant color of the specified region;

generating a radial basis function according to the generated set of samples; and

computing a value at each pixel location of the color constraint mask according to the radial basis function;

performing a texture analysis on the image to generate a locality constraint mask for the image that produces smoothed selections in regions of the locality constraint mask corresponding to low frequency regions of the image while maintaining edges in regions of the locality constraint mask corresponding to high frequency regions of the image; and

blending the color constraint mask and the locality constraint mask to generate an output mask for the image.

2. The method as recited in claim 1, further comprising, prior to said generating a set of samples according to an importance sampling technique:

applying a color clustering technique to the image to generate color clustering information for the image; and

determining the dominant color of the specified region according to the color clustering information.

3. The method as recited in claim 2, wherein the color clustering technique is a K-means technique.

4. The method as recited in claim 1, wherein said performing a texture analysis on the image to generate a locality constraint mask for the image comprises:

generating a flood fill mask for the image according to a flood fill technique, wherein the flood fill mask indicates edges in the image and a selection region of the image;

determining edges in the selection region corresponding to low frequency regions of the image according to a frequency analysis technique;

blurring the edges in the selection region according to a blur technique to generate a blur mask indicating one or more blur regions in the image in which edges are to be smoothed; and

blending the flood fill mask and the blur mask to generate the locality constraint mask for the image.

5. The method as recited in claim 4, further comprising applying an edge-preserving filter to the image to generate a filtered image, wherein the flood fill technique is applied to a hue channel of the filtered image to generate the flood fill mask.

6. The method as recited in claim 4, wherein said determining edges in the selection region corresponding to low frequency regions of the image according to a frequency analysis technique comprises:

generating a gradient domain mask for the flood fill mask;

generating a gradient domain mask for the image; and

blending the gradient domain mask for the flood fill mask and the gradient domain mask for the image to generate a mask indicating the edges in the selection region corresponding to the low frequency regions of the image.

7. The method as recited in claim 6, further comprising applying an edge-preserving filter to the image to generate a filtered image, wherein the gradient domain mask for the image is generated from a hue channel of the filtered image.

8. The method as recited in claim 4, wherein said blending the flood fill mask and the blur mask to generate the locality constraint mask for the image comprises, for each pixel in the locality constraint mask, assigning a value to the pixel according to a maximum of values at corresponding pixel locations in the flood fill mask and the blur mask.

9. The method as recited in claim 1, wherein said blending the color constraint mask and the locality constraint mask to generate an output mask for the image comprises performing an element multiply of values at corresponding pixels in the color constraint mask and the locality constraint mask to generate values for respective pixels in the output mask.

10. A system, comprising:

at least one processor; and

a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to;

obtain an image and a specified region of the image that indicates a color selection to be used in generating a mask for the image;

generate a color constraint mask for the image by;

generating a set of samples according to an importance sampling technique that weights selection of samples towards areas in the image that include a dominant color of the specified region;

generating a radial basis function according to the generated set of samples; and

compute a value at each pixel location of the color constraint mask according to the radial basis function;

perform a texture analysis on the image to generate a locality constraint mask for the image that produces smoothed selections in regions of the locality constraint mask corresponding to low frequency regions of the image while maintaining edges in regions of the locality constraint mask corresponding to high frequency regions of the image; and

blend the color constraint mask and the locality constraint mask to generate an output mask for the image.

11. The system as recited in claim 10, wherein the program instructions are further executable by the at least one processor to, prior to said generating a set of samples according to an importance sampling technique:

apply a color clustering technique to the image to generate color clustering information for the image; and

determine the dominant color of the specified region according to the color clustering information.

12. The system as recited in claim 10, wherein, to perform a texture analysis on the image to generate a locality constraint mask for the image, the program instructions are executable by the at least one processor to:

apply an edge-preserving filter to the image to generate a filtered image;

generate a flood fill mask for the image according to a flood fill technique applied to a hue channel of the filtered image, wherein the flood fill mask indicates edges in the image and a selection region of the image;

determine edges in the selection region corresponding to low frequency regions of the image according to a frequency analysis technique;

blur the edges in the selection region according to a blur technique to generate a blur mask indicating one or more blur regions in the image in which edges are to be smoothed; and

blend the flood fill mask and the blur mask to generate the locality constraint mask for the image.

13. The system as recited in claim 12, wherein, to determine edges in the selection region corresponding to low frequency regions of the image according to a frequency analysis technique, the program instructions are executable by the at least one processor to:

generate a gradient domain mask for the flood fill mask;

generate a gradient domain mask for the hue channel of the filtered image; and

blend the gradient domain mask for the flood fill mask and the gradient domain mask for the hue channel of the filtered image to generate a mask indicating the edges in the selection region corresponding to the low frequency regions of the image.

20. The system as recited in claim 11, wherein the color clustering technique is a K-means technique.

14. A non-transitory computer-readable storage medium storing program instructions that are computer-executable to implement a masking module configured to:

obtain an image and a specified region of the image that indicates a color selection to be used in generating a mask for the image;

generate a color constraint mask for the image by;

generating a set of samples according to an importance sampling technique that weights selection of samples towards areas in the image that include a dominant color of the selected region;

generating a radial basis function according to the generated set of samples; and

computing a value at each pixel location of the color constraint mask according to the radial basis function;

perform a texture analysis on the image to generate a locality constraint mask for the image that produces smoothed selections in regions of the locality constraint mask corresponding to low frequency regions of the image while maintaining edges in regions of the locality constraint mask corresponding to high frequency regions of the image; and

blend the color constraint mask and the locality constraint mask to generate an output mask for the image.

15. The non-transitory computer-readable storage medium as recited in claim 14, wherein the masking module is further operable to, prior to said generating a set of samples according to an importance sampling technique:

apply a color clustering technique to the image to generate color clustering information for the image; and

determine the dominant color of the specified region according to the color clustering information.

16. The non-transitory computer-readable storage medium as recited in claim 14, wherein, to perform a texture analysis on the image to generate a locality constraint mask for the image, the masking module is configured to:

apply an edge-preserving filter to the image to generate a filtered image;

generate a flood fill mask for the image according to a flood fill technique applied to a hue channel of the filtered image, wherein the flood fill mask indicates edges in the image and a selection region of the image;

determine edges in the selection region corresponding to low frequency regions of the image according to a frequency analysis technique;

blur the edges in the selection region according to a blur technique to generate a blur mask indicating one or more blur regions in the image in which edges are to be smoothed; and

blend the flood fill mask and the blur mask to generate the locality constraint mask for the image.

17. The non-transitory computer-readable storage medium as recited in claim 16, wherein, to determine edges in the selection region corresponding to low frequency regions of the image according to a frequency analysis technique, the masking module is configured to:

generate a gradient domain mask for the flood fill mask;

generate a gradient domain mask for the hue channel of the filtered image; and

blend the gradient domain mask for the flood fill mask and the gradient domain mask for the hue channel of the filtered image to generate a mask indicating the edges in the selection region corresponding to the low frequency regions of the image.

18. The non-transitory computer-readable storage medium of claim 15, wherein the color clustering technique is a K-means technique.

19. The non-transitory computer-readable storage medium as recited in claim 16, wherein generating the locality constraint mask for the image comprises assigning a value to each pixel in the locality constraint mask according to a maximum of values at corresponding pixel locations in the flood fill mask and the blur mask.