INTERACTIVE IMAGE MATTING

• US 20120023456A1
• Filed: 07/21/2010
• Published: 01/26/2012
• Est. Priority Date: 07/21/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Jian, He, Kaiming, Liu, Jiangyu
• Granted Patent: US 8,386,964 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation

Abstract

A user interface enables interactive image matting to be performed on an image The user interface may provide results including an alpha matte as feedback in real time. The user interface may provide interactive tools for selecting a portion of the image, and an unknown region for alpha matte processing may be automatically generated adjacent to the selected region. The user may interactively refine the alpha matte as desired to obtain a satisfactory result.

Claims

1. A method comprising:

receiving an indication of a brush stroke made on an image displayed in a user interface;

identifying an area of the image as a foreground region based on a location of the brush stroke on the image;

generating a boundary area on a border of the foreground region, the boundary area containing an unknown region; and

solving, by a processor, the unknown region to provide an alpha matte that distinguishes a foreground portion from a background portion in the unknown region; and

displaying the alpha matte for the unknown region in real time in response to receiving the indication of the brush stroke.

2. The method according to claim 1, wherein the brush stroke is made with a first brush type to indicate selection of the foreground region, and further comprising:

receiving an indication of a second brush stroke made with a second brush type to indicate an expanded unknown region;

solving the expanded unknown region to provide a modified alpha matte; and

displaying the modified alpha matte for the expanded unknown region.

3. The method according to claim 1, wherein the border area is initially generated having a predetermined width specified through the user interface.

4. The method according to claim 1, wherein

the user interface provides at least two brush types;

a first brush type is for identifying the foreground region of the image; and

a second brush type is for identifying the unknown region of the image.

5. The method according to claim 4, further comprising an automated brush selection mechanism causing the first brush type to be automatically selected when a cursor of an input device is located over the foreground region and the second brush type to be automatically selected when the cursor is located over the unknown region.

6. The method according to claim 5, further comprising a third brush type for identifying a background region, the automated brush selection mechanism causing the third brush type to be automatically selected when the cursor is located over the background region.

7. A method comprising:

receiving a user input identifying a portion of an image;

in response to the user input, identifying, by a processor, an unknown region adjacent to a foreground region; and

displaying an alpha matte that distinguishes foreground elements from background element in the unknown region.

8. The method according to claim 7, wherein

the user input is a brush stroke that identifies a portion of the foreground region;

additional portions of the foreground region are automatically identified based on statistically determined color similarities with the identified portion of the foreground region; and

the unknown region is a boundary of a specified width generated between the foreground region and a background region.

9. The method according to claim 7, wherein the user input is a first brush stroke made using a first brush type, and further comprising:

receiving as second user input as a second brush stroke made using a second brush type, the second brush stroke identifying a portion of a background region to be included in the unknown region;

providing a modified alpha matte including the updated portion of the unknown region in real time in response to the second brush stroke.

10. The method according to claim 9, further comprising automatically switching between the first brush type and the second brush type, wherein

when a cursor of an input device is located over the foreground region, the first brush type is automatically selected;

and

when the cursor is located over the unknown region, the second brush type is automatically selected.

11. The method according to claim 7, further comprising solving the alpha matte by

employing a plurality of rectangles covering the unknown region, the plurality of rectangles having a size based on a number of unknown pixels in a corresponding plurality of segments of the unknown region;

and

solving for the alpha matte over the plurality of rectangles using an integral images technique.

12. The method according to claim 7, further comprising solving the alpha matte by sub-sampling the image into a coarse resolution used to solve the unknown area, and subsequently perform bilinear interpolation to generate the alpha matte at a finer resolution based on a resolution of the image.

13. The method according to claim 7, further comprising calculating foreground colors with calculation of the alpha matte in parallel for placing the foreground on a new background as a composite image.

14. Computer-readable storage media containing processor-executable instructions to be executed by a processor for carrying out the method according to claim 7.

15. A system comprising:

a processor in communication with computer-readable storage media;

a user interface component, maintained in the computer-readable storage media and executed on the processor, to receive a user input indicating a foreground region of an image, the user interface component generating an unknown region adjacent to the foreground region indicated; and

a matting component to provide an alpha matte that distinguishes a foreground portion from a background portion in the unknown region.

16. The system according to claim 15, wherein the matting component solves the unknown region of the image using a kernel size selected relative to a size of the unknown region.

17. The system according to claim 15, the user interface component further comprising a brush component that provides:

a first brush type for selecting the foreground region to provide a fixed width boundary area adjacent to the foreground region, the fixed width boundary area containing the unknown region; and

a second brush type for identifying portions of the image for inclusion in the unknown region.

18. The system according to claim 15, the matting component further comprising a localized integral images component to employ a plurality of local windows covering the unknown region to solve for the alpha matte using an integral images technique.

19. The system according to claim 15, the matting component further comprising a coarse-to-fine approximation component to sub-sample the image into a coarse resolution used to solve the unknown area, and subsequently perform bilinear interpolation to generate the alpha matte at a finer resolution based on a resolution of the image.

20. The system according to claim 15, the matting component further comprising a foreground estimation component to estimate foreground colors and the alpha matte in parallel.