Foreground/background segmentation in digital images

• US 8,170,350 B2
• Filed: 05/02/2011
• Issued: 05/01/2012
• Est. Priority Date: 08/16/2004
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Corcoran, Peter, Bigioi, Petronel
• Primary Examiner(s): COUSO, JOSE L
• Application Number:
  US13/099,335
• Publication Number:
  US 20110268359A1
• Time in Patent Office: 365 Days
• Field of Search: 382/173, 382/176, 382/180, 382/181, 382/190, 382/199, 382/218, 382/220, 382/221, 382/224, 382254-275, 382/284, 382/312, 348/333.05, 348/333.11, 348/370, 348/371, 396/374, 345/426, 345/428, 345/581, 345/582
• US Class Current: 382/224
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/35 Categorising the entire scene, e.g. birthday party or wedding scene
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

An analysis and classification tool compares at least a portion of a captured image and a reference image of nominally the same scene. One of the captured and reference images is taken with flash and the other is taken without flash. The tool provides a measure of the difference in illumination between the captured image and the reference image. The tool compares the measure with a threshold and segments a foreground region from a background region based on the measure.

Claims

1. A digital image acquisition system having no photographic film, comprising:

(a) an apparatus for capturing digital images, including a lens, an image sensor and a processor;

(b) a flash unit for providing illumination during image capture;

(c) an analysis tool for using two or more images of a sequence of images of approximately the same scene to create a motion map, said analysis tool providing a measure of relative differences in motion between regions within said scene; and

(d) a classification tool for segmenting a foreground region from a background region within said scene based on said measure.

2. A system according to claim 1 further comprising a segmentation tool for determining one or more regions that are indicative of the foreground region, or of the background region, or of the background region and the foreground region, within at least one portion of a captured image, and wherein said analysis tool is arranged to analyse said foreground region or said background region, or both.

3. A system according to claim 1, wherein said classification tool is responsive to said measure exceeding a high threshold to classify a region as a background region, and responsive to said measure not exceeding a low threshold to classify a region as a foreground region.

4. A system according to claim 3, wherein said high and low threshold are coincident.

5. A system according to claim 1, wherein the analysis tool simulates an ambient exposure of one of the images on another of the images including digitally simulating one or a combination of aperture, acquisition speed, color transformations or gain of the captured image on the reference image.

6. A system according to claim 5, wherein the simulating of the ambient exposure of the captured image comprises individual, non-uniform manipulating of individual regions or color channels or combinations thereof.

7. A system according to claim 1, wherein at least in respect of at least one portion, the segmentation tool determines corresponding pixels in the at least two images whose values differ by less than a predetermined threshold, and designates segments of the scene bounded by said determined pixels as foreground or background by comparing motion values in different segments within the scene.

8. A system according to claim 6, further comprising a face detection module.

9. A method of foreground/background segmentation in a captured digital image, comprising:

using a processor to perform said method of foreground/background segmentation;

analyzing two or more images of a sequence of images of approximately the same scene to create a motion map,

determining a measure of relative differences in motion between regions within said scene based on said motion map; and

classifying a foreground region segmented from a background region within said scene based on said measure.

10. A method as in claim 9, further comprising determining one or more regions that are indicative of the foreground region, or of the background region, or of the background region and the foreground region, within at least one portion of a captured image, and wherein said analysis tool is arranged to analyse said foreground region or said background region, or both.

11. A method as in claim 9, wherein said classifying is performed when said measure exceeds a high threshold to classify a region as a background region, and when said measure does not exceed a low threshold to classify a region as a foreground region.

12. A method as in claim 11, wherein said high and low thresholds are coincident.

13. A method as in claim 9, further comprising simulating of an ambient exposure of one of the images on another of the images including digitally simulating one or a combination of aperture, acquisition speed, color transformations or gain of the captured image on the reference image.

14. A method as in claim 13, wherein the simulating of the ambient exposure of the captured image comprises individual, non-uniform manipulating of individual regions or color channels or combinations thereof.

15. A method as in claim 9, wherein at least in respect of at least one portion, the classifying includes determining corresponding pixels in the at least two images whose values differ by less than a predetermined threshold, and designating segments of the scene bounded by said determined pixels as foreground or background by comparing motion values in different segments within the scene.

16. A method as in claim 9, further comprising detecting a face within the scene, and classifying the face as foreground.

17. One or more non-transitory, processor-readable media having code embedded therein for programming one or more processors to perform a method of foreground/background segmentation in a captured digital image, wherein the method comprises:

analyzing two or more images of a sequence of images of approximately the same scene to create a motion map,

determining a measure of relative differences in motion between regions within said scene based on said motion map; and

classifying a foreground region segmented from a background region within said scene based on said measure.

18. One or more non-transitory, processor-readable media as in claim 17, wherein the method further comprises determining one or more regions that are indicative of the foreground region, or of the background region, or of the background region and the foreground region, within at least one portion of a captured image, and wherein said analysis tool is arranged to analyse said foreground region or said background region, or both.

19. One or more non-transitory, processor-readable media as in claim 17, wherein said classifying is performed when said measure exceeds a high threshold to classify a region as a background region, and when said measure does not exceed a low threshold to classify a region as a foreground region.

20. One or more non-transitory, processor-readable media as in claim 19, wherein said high and low thresholds are coincident.

21. One or more non-transitory, processor-readable media as in claim 17, wherein the method further comprises simulating of an ambient exposure of one of the images on another of the images including digitally simulating one or a combination of aperture, acquisition speed, color transformations or gain of the captured image on the reference image.

22. One or more non-transitory, processor-readable media as in claim 21, wherein the simulating of the ambient exposure of the captured image comprises individual, non-uniform manipulating of individual regions or color channels or combinations thereof.

23. One or more non-transitory, processor-readable media as in claim 17, wherein at least in respect of at least one portion, the classifying includes determining corresponding pixels in the at least two images whose values differ by less than a predetermined threshold, and designating segments of the scene bounded by said determined pixels as foreground or background by comparing motion values in different segments within the scene.

24. One or more non-transitory, processor-readable media as in claim 17, wherein the method further comprises detecting a face within the scene, and classifying the face as foreground.