Face tracker and partial face tracker for red-eye filter method and apparatus

• US 8,520,093 B2
• Filed: 08/31/2009
• Issued: 08/27/2013
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Nanu, Florin, Steinberg, Eran, Corcoran, Peter, Pososin, Alexei, Bigioi, Petronel, Drimbarean, Alexandru, Petrescu, Stefan, Gangea, Mihnea, Capata, Adrian, Ciuc, Mihai, Zamfir, Adrian
• Primary Examiner(s): Ho, Tuan
• Assistant Examiner(s): BHUIYAN, FAYEZ A
• Application Number:
  US12/551,335
• Publication Number:
  US 20100053368A1
• Time in Patent Office: 1,457 Days
• Field of Search: 348/224.1, 382/103, 382/167
• US Class Current: 348/224.1
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An image acquisition device includes a flash and optical system for capturing digital images. A face tracker identifies face regions within a series of one or more relatively low resolution reference images, and predicts face regions within a main digital image. A face analyzer determines one or more partial face regions within the one or more face regions each including at least one eye. A red-eye filter modifies an area within the main digital image indicative of a red-eye phenomenon based on an analysis of one or more partial face regions within the one or more face regions identified and predicted by the face tracker.

Claims

1. A portable digital image capturing device having no photographic film, comprising:

a flash for providing illumination during image acquisition;

an optical system including a lens and an image sensor for capturing a main digital image;

a half-face tracker for identifying only one or more half-face regions within a series of one or more relatively low resolution reference images, and predicting an approximate location of said one or more half-face regions within the main digital image;

a half-face analyzer for determining one or more half-face regions in the vicinity of the approximate location within the main digital image predicted by the half-face tracker, each half-face region including only one eye and one or both of only a half nose or only a half mouth region; and

a red-eye filter for modifying an area within the main digital image indicative of a red-eye phenomenon based on an analysis of the one or more half-face regions within the main digital image, each including at least one half-face region predicted by the half-face tracker.

2. The device of claim 1, wherein said red-eye filter is adapted based on a type of at least one of said one or more half-face regions identified within the digital image.

3. The device of claim 1, wherein the analysis is performed at least in part for determining said area.

4. The device of claim 1, wherein the analysis is performed at least in part for determining said modifying.

5. The device of claim 1, wherein at least one half-face region within the digital image is not among said one or more half face regions identified within the digital image that are analyzed.

6. The device of claim 1, wherein said analysis is performed in part on a full resolution half-face region and in part on a subsample resolution of at least one different half-face region.

7. The device of claim 1, wherein said modifying the area is performed on a full resolution of at least one-half-face region within the digital image.

8. The device of claim 1, wherein said red-eye filter comprises a plurality of subfilters.

9. The device of claim 1, further comprising:

a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

10. The device of claim 9, further comprising a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

11. The device of claim 9, further comprising an exposure analyzer for determining if the digital image was acquired in a condition indicative of the red-eye phenomenon.

12. A method of capturing and processing within a portable device a digital image including one or more half-face regions not exhibiting any red eye defect, comprising:

acquiring a series of one or more relatively low resolution reference images;

identifying only one or more half-face regions within the one or more relatively low resolution reference images each including only one eye and one or both of only a half nose or only a half mouth region;

predicting an approximate location of each of the one or more half-face regions within a main digital image based on the identifying;

capturing the main digital image with a portable device that includes a lens and an image sensor;

providing flash illumination during the capturing of the main digital image with the portable device;

determining and analyzing one or more half-face regions each in a vicinity of an approximate location within the main digital image based on the predicting, each including said only one eye and said one or both of only a half nose or only a half mouth region, within the main digital image, while foregoing within the main digital image analysis of at least one other half face region not including an eye; and

modifying an area within the at least one half face region that is determined to be indicative of a red-eye phenomenon based on said analyzing.

13. The method of claim 12, further comprising adapting the modifying based on a type of said at least one partial face region.

14. The method of claim 12, further comprising determining said area based at least in part on the analyzing.

15. The method of claim 12, further comprising determining said modifying based at least in part on the analyzing.

16. The method of claim 12, wherein said analyzing is performed in part on a full resolution half-face region and in part on a subsample resolution of at least one different half-face region.

17. The method of claim 12, wherein said analyzing is performed on a subsample representation of at least one half-face region.

18. The method of claim 17, wherein said subsample representation comprises a subsample resolution representation.

19. The method of claim 12, wherein said modifying the area is performed on a full resolution of at least one half-face region within the image.

20. The method of claim 12, wherein the analyzing and modifying comprise:

locating pixels having a color indicative of the red-eye phenomenon;

determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

modifying the color of the pixels within the grouping.

21. One or more processor-readable storage devices having code embedded therein for programming a processor to perform a method of capturing and processing within a portable device a digital image including one or more half-face regions not exhibiting any red eye defect, the method comprising:

analyzing a series of one or more relatively low resolution reference images;

identifying only one or more half-face regions within the one or more relatively low resolution reference images each including only one eye and one or both of only a half nose or only a half mouth region;

predicting an approximate location of each of the one or more half-face regions within a main digital image based on the identifying;

capturing the main digital image with a portable device that includes a lens and an image sensor;

providing flash illumination during the capturing of the main digital image with the portable device;

determining and analyzing one or more half-face regions each in a vicinity of an approximate location within the main digital image based on the predicting, each including said only one eye and said one or both of only a half nose or only a half mouth region, within the main digital image, while foregoing within the main digital image analysis of at least one other half face region not including an eye; and

modifying an area within the at least one half face region that is determined to be indicative of a red-eye phenomenon based on said analyzing.

22. The one or more processor-readable storage devices of claim 21, wherein the method further comprises adapting the modifying based on a type of said at least one half-face region.

23. The one or more processor-readable storage devices of claim 21, wherein the method further comprises determining said area based at least in part on the analyzing.

24. The one or more processor-readable storage devices of claim 21, wherein the method further comprises determining said modifying based at least in part on the analyzing.

25. The one or more processor-readable storage devices of claim 21, wherein said analyzing is performed in part on a full resolution half-face region and in part on a subsample resolution of at least one different half-face region.

26. The one or more processor-readable storage devices of claim 21, wherein said analyzing is performed on a subsample representation of at least one half-face region.

27. The one or more processor-readable storage devices of claim 26, wherein said subsample representation comprises a subsample resolution representation.

28. The one or more processor-readable storage devices of claim 21, wherein said modifying the area is performed on a full resolution of at least one half-face region within the image.

29. The one or more processor-readable storage devices of claim 21, wherein the analyzing and modifying comprise:

locating pixels having a color indicative of the red-eye phenomenon;

determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

modifying the color of the pixels within the grouping.