IMAGE FOREGROUND EXTRACTION USING A PRESENTATION APPLICATION

• US 20090060334A1
• Filed: 08/06/2008
• Published: 03/05/2009
• Est. Priority Date: 08/06/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Rayner, Tyler Christian
• Granted Patent: US 8,559,732 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/10 Indexing; Addressing; Timing or synchronising; Measuring tape travel
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

A presentation application is provided that allows a user to extract a portion of the foreground or background from an image. The user can define a polygon outlining a portion of the image, and can adjust the shape and width of the polygon. The presentation application may use the user-defined polygon to identify a background region associated with pixels lying entirely outside the polygon, a foreground region associated with pixels enclosed by the polygon, and a set of pixels that are initially unknown within the width of the polygon. The presentation application then determines whether each unknown pixel is part of the background, foreground, or part of both. Based on the classification, the presentation application can extract the foreground of the image and mask the remaining portions of the image.

Claims

1. A method of extracting a portion of an image, wherein the image is formed by a plurality of pixels, the method comprising:

generating a polygon in response to receiving one or more user inputs;

displaying the polygon as an overlay on the image;

categorizing pixels of the image as part of a foreground or background of the image, wherein the categorizing comprises;

classifying pixels that are enclosed by an inner boundary of the polygon as part of the foreground;

classifying pixels that are not enclosed by an outer boundary of the polygon as part of the background; and

determining whether each pixel between the inner and outer boundary of the polygon is part of the background, foreground, or both the background and foreground; and

extracting a foreground of the image based on the pixels categorized as part of the foreground.

2. The method of claim 1, wherein generating a polygon comprises receiving a plurality of selections to form vertices of the polygon.

3. The method of claim 1, wherein one of the one or more user inputs is a user request to close the polygon.

4. The method of claim 1, further comprising:

receiving a user indication to change a width of the polygon;

displaying the polygon with the width based on the user indication; and

categorizing the pixels of the image according to the changed width of the polygon.

5. The method of claim 1, wherein categorizing the pixels occurs in response to receiving a user indication to extract the portion of the image, the method further comprising displaying the image, wherein the pixels categorized as part of the background are displayed in an altered form.

6. A method of extracting a portion of an image, wherein the image is formed by a plurality of pixels, the method comprising:

generating a polygon in response to receiving one or more user inputs;

displaying the polygon as an overlay on the image;

categorizing pixels of the image as part of a foreground or background of the image, wherein the categorizing comprises;

classifying pixels that are enclosed by an inner boundary of the polygon as part of the foreground;

classifying pixels that are not enclosed by an outer boundary of the polygon as part of the background; and

determining whether each pixel between the inner and outer boundary of the polygon is part of the background, foreground, or both the background and foreground; and

extracting a background of the image based on the pixels categorized as part of the background.

7. A method of classifying a pixel as part of a foreground or background of an image, wherein the pixel is one of a plurality of contiguous pixels not classified as part of the foreground or background of the image, the method comprising:

identifying a first pixel path leading away from the pixel and directed towards the foreground of the image;

identifying a second pixel path leading away from the pixel and directed towards the background of the image; and

classifying the pixel based on properties of the first pixel path and the second pixel path.

8. The method of claim 7, further comprising determining a color resistance between each pair of neighboring pixels of the contiguous pixels.

9. The method of claim 8, wherein determining a color resistance comprises computing, for each pixel of the contiguous pixels, a color resistance to neighboring pixels in four directions.

10. The method of claim 9, wherein the four directions comprise directions up, up-right, right, and down-right.

11. The method of claim 8, wherein the properties of the first pixel path and the second pixel path each comprises a total color resistance accumulated when traversing each path.

12. The method of claim 7, wherein the properties of the first pixel path comprises a first type of a pixel ending the first pixel path, and wherein the properties of the second pixel path comprises a second type of pixel ending the second pixel path.

13. The method of claim 7, further comprising identifying, for each pixel of the contiguous pixels, a first guidance vector pointing towards a closest foreground pixel.

14. The method of claim 13, wherein identifying the first pixel path comprises directing the first pixel path towards the closest foreground pixel based on the vector.

15. The method of claim 7, further comprising identifying, for each pixel of the contiguous pixels, a second guidance vector pointing towards a closest background pixel.

16. The method of claim 15, wherein identifying the second pixel path comprises directing the second pixel path towards the closest background pixel based on the second guidance vector.

17. An electronic device for performing an extraction of a portion of an image formed by a plurality of pixels, the electronic device comprising a display, an input device, and a processor, the processor operative to:

generate a polygon in response to receiving one or more user inputs from the input device;

display the polygon on the display as an overlay on the image;

categorize pixels of the image as part of a foreground or background of the image by;

classifying pixels that are enclosed by an inner boundary of the polygon as part of the foreground;

classifying pixels that are not enclosed by an outer boundary of the polygon as part of the background; and

determining whether each pixel between the inner and outer boundary of the polygon is part of the background, foreground, or both the background and foreground; and

extract a foreground of the image based on the pixels categorized as part of the foreground.

18. The electronic device of claim 17, wherein the processor is further operative to receive a plurality of selections to form vertices of the polygon.

19. The electronic device of claim 17, wherein one of the one or more user inputs is a user request to close the polygon.

20. The electronic device of claim 17, wherein the processor is further operative to:

receive a user indication to change a width of the polygon;

display, on the display, the polygon with the width based on the user indication; and

categorize the pixels of the image according to the changed width of the polygon.

21. The electronic device of claim 17, wherein the processor categorizes the pixels in response to receiving a user indication to extract the portion of the image, and wherein the processor is further operative to display the image, wherein the pixels categorized as part of the background are displayed in an altered form.

22. An electronic device for performing an extraction of a portion of an image formed by a plurality of pixels, the electronic device comprising a display, an input device, and a processor, the processor operative to:

generate a polygon in response to receiving one or more user inputs from the input device;

display the polygon on the display as an overlay on the image;

categorize pixels of the image as part of a foreground or background of the image by;

classifying pixels that are enclosed by an inner boundary of the polygon as part of the foreground;

classifying pixels that are not enclosed by an outer boundary of the polygon as part of the background; and

determining whether each pixel between the inner and outer boundary of the polygon is part of the background, foreground, or both the background and foreground; and

extract a background of the image based on the pixels categorized as part of the background.

23. An electronic device for classifying a pixel as part of a foreground or background of an image, wherein the pixel is one of a plurality of contiguous pixels not classified as part of the foreground or background of the image, the electronic device comprising a processor configured to:

identify a first pixel path leading away from the pixel and directed towards the foreground of the image;

identify a second pixel path leading away from the pixel and directed towards the background of the image; and

classify the pixel based on properties of the first pixel path and the second pixel path.

24. The electronic device of claim 23, wherein the processor is further operative to determine a color resistance between each pair of neighboring pixels of the contiguous pixels.

25. The electronic device of claim 24, wherein the processor is configured to compute, for each pixel of the contiguous pixels, a color resistance to neighboring pixels in four directions.

26. The electronic device of claim 25, wherein the four directions comprise directions up, up-right, right, and down-right.

27. The electronic device of claim 24, wherein the properties of the first pixel path and the second pixel path each comprises a total color resistance accumulated when traversing each path.

28. The electronic device of claim 23, wherein the properties of the first pixel path comprises a first type of a pixel ending the first pixel path, and wherein the properties of the second pixel path comprises a second type of pixel ending the second pixel path.

29. The electronic device of claim 23, wherein the processor is further operative to identify, for each pixel of the contiguous pixels, a first guidance vector pointing towards a closest foreground pixel.

30. The electronic device of claim 29, wherein the processor is further operative to direct the first pixel path towards the closest foreground pixel based on the vector.

31. The electronic device of claim 23, wherein the processor is further operative to identify, for each pixel of the contiguous pixels, a second guidance vector pointing towards a closest background pixel.

32. The electronic device of claim 31, wherein the processor is further configured to direct the second pixel path towards the closest background pixel based on the second guidance vector.