Foreground / background separation in digital images

• US 9,117,282 B2
• Filed: 01/28/2013
• Issued: 08/25/2015
• Est. Priority Date: 05/03/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Corcoran, Peter, Prilutsky, Yury, Bigioi, Petronel, Pososin, Alexei, Ciuc, Mihai, Zamfir, Adrian, Capata, Adrian, Steinberg, Eran
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/751,973
• Publication Number:
  US 20130208979A1
• Time in Patent Office: 939 Days
• Field of Search: 382/118, 382/173
• US Class Current: 1/1
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A method for providing improved foreground/background separation in a digital image of a scene is disclosed. The method comprises providing a first map comprising one or more regions provisionally defined as one of foreground or background within the digital image; and providing a subject profile corresponding to a region of interest of the digital image. The provisionally defined regions are compared with the subject profile to determine if any of the regions intersect with the profile region. The definition of one or more of the regions in the map is changed based on the comparison.

Claims

1. A method comprising:

using a processor-based digital image acquisition device that includes a processor programmed by processor-readable code embedded that is stored within one or more digital storage media to perform the steps of;

providing a first map comprising one or more regions that are within a main digital image of a scene, wherein each of the one or more regions has one or more pixels with a common characteristic;

providing an object profile corresponding to a region of interest of said main digital image, wherein the region of interest includes an object candidate;

wherein said object profile comprises an object portion and at least a second portion that is adjacent to said object portion and has a known association and orientation relative to the object portion;

determining that the object candidate within the region of interest matches said object portion of the object profile;

determining an orientation of the region of interest within said main digital image;

comparing said region of interest with said object profile to determine whether said region of interest matches said object profile; and

designating said region of interest as a foreground or background region based on said comparing.

2. The method of claim 1 wherein providing said first map comprises defining each of said one or more regions of said main digital image as a foreground or background region and wherein said one or more regions comprise at least one region defined as a foreground region.

3. The method of claim 1 wherein the providing of said first map is based on a comparison of two or more images approximately of said scene.

4. A-method as claimed in The method of claim 1 wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene;

calculating from said images a depth of focus map indicating pixels of said main digital image as either a foreground region or a background region;

blurring said focus map;

thresholding said blurred map to an intermediate focus map comprising indicating regions within said intermediate focus map as either a foreground region or a background region; and

filling the regions within said intermediate focus map to provide said first map.

5. The method of claim 1, wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene; and

comparing high frequency coefficients of corresponding regions in said images to determine whether said one or more regions comprise a foreground or background region.

6. The method of claim 1 wherein the providing of said first map comprises:

providing two or more images at different exposure levels approximately of said scene; and

comparing luminance levels of corresponding regions in said two or more images to determine whether said one or more regions are foreground or background regions.

7. The method of claim 1 wherein said providing a the first map comprises:

analyzing at least one region of the main digital image in a color space to determine a color distribution within said at least one region, said color distribution having a plurality of distinct color peaks; and

segmenting said at least one region based on proximity of a pixel'"'"'s color to said plurality of distinct color peaks.

8. The method of claim 1 wherein said main digital image is one of a stream of images and the method comprises:

determining at least one region of said main image that includes an object;

detecting the object in at least one image of said stream acquired prior to said main digital image; and

tracking said object through said stream of images to determine said object in said main digital image.

9. An apparatus for providing improved separation between foreground and background regions in a digital image of a scene, comprising a processor and one or more processor-readable media that are configured to program the processor to perform a method comprising:

acquiring a main digital image;

providing a first map comprising one or more regions that are within the main digital image of a scene, wherein each of the one or more regions has one or more pixels with a common characteristic;

providing an object profile corresponding to a region of interest of said main digital image, wherein the region of interest includes an object candidate;

wherein said object profile comprises an object portion and at least a second portion that is adjacent to said object portion and has a known association and orientation relative to the object portion;

determining that the object candidate within the region of interest matches said object portion of the object profile;

determining an orientation of the region of interest within said main digital image;

comparing said region of interest with said object profile to determine whether said region of interest matches said object profile; and

designating said region of interest as a foreground or background region based on said comparing.

10. The apparatus of claim 9 wherein providing said first map comprises defining each of said one or more regions of said main digital image as a foreground or background region and wherein said one or more regions comprise at least one region defined as a foreground region.

11. The apparatus of claim 9 wherein the providing of said first map is based on a comparison of two or more images approximately of said scene.

12. The apparatus of claim 9 wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene;

calculating from said images a depth of focus map indicating pixels of said main digital image as either a foreground region or a background region;

blurring said focus map;

thresholding said blurred map to an intermediate focus map comprising indicating regions within said intermediate focus map as either a foreground region or a background region; and

filling the regions within said intermediate focus map to provide said first map.

13. The apparatus of claim 9, wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene; and

comparing high frequency coefficients of corresponding regions in said images to determine whether said one or more regions comprise a foreground or background region.

14. The apparatus of claim 9 wherein the providing of said first map comprises:

providing two or more images at different exposure levels approximately of said scene; and

comparing luminance levels of corresponding regions in said two or more images to determine whether said one or more regions are foreground or background regions.

15. An apparatus as claimed in The apparatus of claim 9 wherein said providing a the first map comprises:

analyzing at least one region of the main digital image in a color space to determine a color distribution within said at least one region, said color distribution having a plurality of distinct color peaks; and

segmenting said at least one region based on proximity of a pixel'"'"'s color to said plurality of distinct color peaks.

16. The apparatus of claim 9 wherein said main digital image is one of a stream of images and the method comprises:

determining at least one region of said main image that includes an object;

detecting the object in at least one image of said stream acquired prior to said main digital image; and

tracking said object through said stream of images to determine said object in said main digital image.

17. One or more non-transitory processor-readable media having code stored therein, wherein the code is configured to program one or more processors to perform a method comprising:

providing a first map comprising one or more regions within a main digital image of a scene, wherein each of the one or more regions has one or more pixels with a common characteristic;

providing an object profile corresponding to a region of interest of said main digital image, wherein the region of interest includes an object candidate;

wherein said object profile comprises an object portion and at least a second portion that is adjacent to said object portion and has a known association and orientation relative to the object portion;

determining that the object candidate within the region of interest matches said object portion of the object profile;

determining an orientation of the region of interest within said main digital image;

comparing said region of interest with said object profile to determine whether said region of interest matches said object profile; and

designating said region of interest as a foreground or background region based on said comparing.

18. The non-transitory processor-readable media of claim 17 wherein providing said first map comprises defining each of said one or more regions of said main digital image as a foreground or background region and wherein said one or more regions comprise at least one region defined as a foreground region.

19. The non-transitory processor-readable media of claim 17 wherein the providing of said first map is based on a comparison of two or more images approximately of said scene.

20. The non-transitory processor-readable media of claim 17 wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene;

calculating from said images a depth of focus map indicating pixels of said main digital image as either a foreground region or a background region;

blurring said focus map;

thresholding said blurred map to an intermediate focus map comprising indicating regions within said intermediate focus map as either a foreground region or a background region; and

filling the regions within said intermediate focus map to provide said first map.

21. The non-transitory processor-readable media of claim 17, wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene; and

comparing high frequency coefficients of corresponding regions in said images to determine whether said one or more regions comprise a foreground or background region.

22. The non-transitory processor-readable media of claim 17 wherein the providing of said first map comprises:

providing two or more images at different exposure levels approximately of said scene; and

comparing luminance levels of corresponding regions in said two or more images to determine whether said one or more regions are foreground or background regions.

23. The non-transitory processor-readable media of claim 17 wherein said providing the first map comprises:

analyzing at least one region of the main digital image in a color space to determine a color distribution within said at least one region, said color distribution having a plurality of distinct color peaks; and

segmenting said at least one region based on proximity of a pixel'"'"'s color to said plurality of distinct color peaks.

24. The non-transitory processor-readable media of claim 17 wherein said main image is one of a stream of images and the method comprises:

determining at least one region of said main image that includes an object;

detecting the object in at least one image of said stream acquired prior to said main digital image; and

tracking said object through said stream of images to determine said object in said main digital image.