Foreground / Background Separation in Digital Images

• US 20070269108A1
• Filed: 05/03/2007
• Published: 11/22/2007
• Est. Priority Date: 05/03/2006
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Zamfir, Adrian, Capata, Adrian, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei, Ciuc, Mihai
• Granted Patent: US 8,363,908 B2
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

A method for providing improved foreground/background separation in a digital image of a scene is disclosed. The method comprises providing a first map comprising one or more regions provisionally defined as one of foreground or background within the digital image; and providing a subject profile corresponding to a region of interest of the digital image. The provisionally defined regions are compared with the subject profile to determine if any of the regions intersect with the profile region. The definition of one or more of the regions in the map is changed based on the comparison.

Claims

1. A method for providing foreground/background separation in a digital image of a scene, comprising:

providing a first map comprising one or more regions within a main digital image, each having one or more pixels with a common characteristic;

providing a subject profile corresponding to a region of interest of said main digital image;

comparing one or more of said regions with said subject profile to determine if any of said one or more regions intersect with said profile region; and

designating one or more of said regions as a foreground region based on said comparison.

2. A method as claimed in claim 1 wherein providing said first map comprises provisionally defining each region of an image as foreground or background and wherein said one or more regions comprise at least one region provisionally defined as foreground.

3. A method as claimed in claim 2 wherein said designating comprises:

comparing a foreground region with said subject profile; and

responsive to said foreground region not substantially intersecting said subject profile, changing a designation of said foreground region to a background region.

4. A method as claimed in claim 1 wherein the providing of said first map is based on a comparison of two or more images nominally of said scene.

5. A method as claimed in claim 4 wherein one or more of said two or more images that are compared comprises a lower resolution version of said main digital image.

6. A method as claimed in claim 4 wherein one of said two or more images that are compared comprises said main digital image.

7. A method as claimed in claim 4 further comprising aligning said two or more images that are compared.

8. A method as claimed in claim 4 further comprising matching resolutions of said two or more images that are compared.

9. A method as claimed in claim 4 wherein at least one of said two or more images that are compared is captured just before or after said main digital image is captured.

10. A method as claimed in claim 1 wherein the providing of said first map comprises:

providing two of more images each of different focus and nominally of said scene;

calculating from said images a depth of focus map indicating pixels of said main digital image as either foreground or background;

blurring said focus map;

thresholding said blurred map to an intermediate focus map indicating regions as either foreground or background; and

filling regions within said intermediate focus map to provide said first map.

11. A method as claimed in claim 1, wherein the providing of said first map comprises:

providing two or more images each of different focus and nominally of said scene; and

comparing high frequency coefficients of corresponding regions in said images to determine whether said regions are foreground or background to provide said first map.

12. A method as claimed in claim 1 wherein the providing of said first map comprises:

providing two or more images at different exposure levels nominally of said scene; and

comparing luminance levels of corresponding regions in said images to determine whether said regions are foreground or background to provide said first map.

13. A method as claimed in claim 1 operable in a digital image acquisition device, said device being arranged to select said subject profile according to content of said main digital image.

14. A method as claimed in claim 1 operable in a digital image acquisition device, said device being arranged to operate in a portrait mode and wherein said subject profile comprises an outline of a person.

15. A method as claimed in claim 14 wherein said outline comprises one of a number of user selectable outlines.

16. A method as claimed in claim 14 wherein said outline is automatically selected from a plurality of outlines based on the content of said main digital image.

17. A method as claimed in claim 1 operable in a general purpose computer arranged to receive said first map in association with said main digital image.

18. A method as claimed in claim 1 operable in a general purpose computer arranged to receive one or more additional images nominally of said scene in association with said main digital image and being arranged to calculate said first map from any combination of said one or more additional images and said main digital image.

19. A method as claimed in claim 1 wherein said providing a subject profile comprises;

determining at least one region of said image including a face;

determining an orientation of said at least one face within said main image; and

defining said subject profile as comprising said at least one face region and a respective region below said face region in said main image in accordance with said orientation.

20. A method as claimed in claim 19 wherein said providing a first map comprises:

analyzing at least one region of the main digital image in a color space to determine a color distribution within said regions, said color distribution having a plurality of distinct color peaks; and

segmenting said regions based on proximity of a pixel'"'"'s color to said color peaks.

21. A method as claimed in claim 20 wherein said comparing comprises:

for each region intersecting said subject profile, calculating a reference reflectance characteristic;

for each region not intersecting said subject profile, calculating a reflectance characteristic; and

comparing the non-intersecting region reflectance characteristic with the reference reflectance characteristic for a region corresponding in color to said non-intersecting region; and

wherein said designating comprises designating a non-intersecting region as foreground when said non-intersecting region reflectance characteristic is determined to be within a threshold of said reference reflectance characteristic.

22. A method as claimed in claim 21 comprising providing a second image nominally of said scene;

and calculating said reflectance characteristics as a function of a difference luminance levels between corresponding regions in said main and second images.

23. A method as claimed in claim 19 wherein said main image is one of a stream of images and wherein determining at least one region of said main image including a face comprises:

detecting a face in at least one image of said stream acquired prior to said main image; and

tracking said face through said stream of images to determine said face region in said main image.

24. A method for providing foreground/background separation in a digital image of a scene comprising:

providing a first map comprising one or more regions provisionally defined as one of foreground or background within a main digital image;

analyzing one or more of said regions to determine a distribution of luminance within pixels of the region;

responsive to said luminance distribution for a region having more than one distinct luminance peak, dividing said region into more than one sub-region based on proximity of a pixel luminance to said luminance peaks; and

changing in said map a designation of said sub-regions based on said division.

25. A method as claimed in claim 24 further comprising:

providing a subject profile corresponding to a region of interest of said main digital image;

comparing at least some of said one or more provisionally defined regions with said subject profile to determine if any of said regions or sub-regions of said first map intersect with said profile region;

changing in said map a designation of one or more of said regions or sub-regions based on said comparison.

26. A method as claimed in claim 24 wherein providing said first map comprises of:

analyzing at least some regions of the digital image in a color space to determine a color distribution within said regions, said color distribution having a plurality of distinct color peaks; and

segmenting said regions based on proximity of pixel color to said color peaks.

27. A method as claimed in claim 26 wherein said analyzed regions are regions provisionally defined as foreground within said first map.

28. A method as claimed in claim 26 wherein said digital image is provided in LAB space and wherein said color space comprises [a,b] values for pixels and wherein said luminance comprises L values for pixels.

29. A method for providing improved foreground/background separation in a digital image of a scene comprising of:

acquiring a main digital image;

determining at least one region of said main image including a face;

determining an orientation of said at least one face within said main image; and

defining a foreground region of said image comprising said at least one face region and a respective region below said face region in said image in accordance with said orientation.

30. An apparatus for providing improved foreground/background separation in a digital image of a scene, comprising a processor and one or more processor-readable media for programming the processor to control the apparatus to perform a method comprising:

acquiring a main digital image;

providing a first map comprising one or more regions provisionally defined as one of foreground or background within said main digital image, each having one or more pixels with a common characteristic;

providing a subject profile corresponding to a region of interest of said main digital image;

comparing at least some of said one or more provisionally defined regions with said subject profile to determine if any of said regions intersect with said profile region; and

changing in said map a designation of one or more of said regions as a foreground region based on said comparison.

31. An apparatus for providing foreground/background separation in a digital image of a scene, comprising a processor and one or more processor-readable media for programming the processor to control the apparatus to perform a method comprising:

acquiring a main digital image;

providing a first map comprising one or more regions provisionally defined as one of foreground or background within said main digital image;

analyzing one or more of said regions to determine a distribution of luminance within pixels of the region;

responsive to said luminance distribution for a region having more than one distinct luminance peak, dividing said region into more than one sub-region based on proximity of a pixel luminance to said luminance peaks; and

changing in said map a designation of at least one of said sub-regions based on said dividing.

32. An apparatus for providing foreground/background separation in a digital image of a scene, comprising a processor and one or more processor-readable media for programming the processor to control the apparatus to perform a method comprising:

acquiring a main digital image;

determining at least one region of said main image including a face;

determining an orientation of said face; and

defining a foreground region of said image comprising said face and a respective region below said face in said main image in accordance with said orientation.