Partial Face Detector Red-Eye Filter Method and Apparatus

• US 20110102643A1
• Filed: 11/08/2010
• Published: 05/05/2011
• Est. Priority Date: 02/04/2004
• Status: Abandoned Application

• Current Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Original Assignee: Tessera Technologies Ireland Limited (Adeia Inc.)
• Inventors: Gangea, Mihnea, Steinberg, Eran, Petrescu, Stefan, Ciuc, Mihai, Zamfir, Adrian, Capata, Adrian, Nanu, Florin, Drimbarean, Alexandru, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei
• Application Number:
  US12/941,995
• Publication Number:
  US 20110102643A1
• Time in Patent Office: Days
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/446 using Haar-like filters, e.g. using integral image techniques
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

A digital camera has an integral flash and stores and displays a digital image. Under certain conditions, a flash photograph taken with the camera may result in a red-eye phenomenon due to a reflection within an eye of a subject of the photograph. A digital apparatus has a red-eye filter which analyzes at least one partial face region identified within the digital image for the red-eye phenomenon and modifies the image to eliminate the red-eye phenomenon by changing the red area to black. The modification of the image is enabled when a photograph is taken under conditions indicative of the red-eye phenomenon. The modification is subject to anti-falsing analysis which further examines the area around the red-eye area for indicia of the eye of the subject. The detection and correction can be optimized for performance and quality by operating on subsample versions of the image when appropriate.

Claims

1. A portable digital image capturing device having no photographic film, comprising:

a flash for providing illumination during image acquisition;

an optical system including a lens and an image sensor for capturing a digital image;

one or more dedicated hardware units, including;

one or both of a face or a partial face detector subsystem for providing hardware-based identification of one or more face or partial face regions, or a combination thereof, within the digital image.

2. The device of claim 1, wherein at least one face or partial face region within the digital image is not among said one or more face or partial face regions identified within the digital image, and wherein the one or more dedicated hardware units further comprise a face analyzer for providing hardware-based analysis of the one or more identified face or partial face regions.

3. The device of claim 2, wherein said analysis is performed in part on a full resolution face or partial face region and in part on a subsample resolution of at least one different face or partial face region.

4. The device of claim 1, wherein the one or more dedicated hardware units further comprise a red-eye filter for providing hardware-based modification of an area within the image indicative of a red-eye phenomenon based on an analysis of a subsample representation comprising one or more face or partial face regions identified within the image, or both.

5. The device of claim 4, wherein said red-eye filter is adapted based on a type of at least one of said one or more face or partial face regions identified within the digital image.

6. The device of claim 4, wherein the analysis is performed at least in part for determining said area.

7. The device of claim 4, wherein the analysis is performed at least in part for determining said modifying.

8. The device of claim 4, wherein the one or more dedicated hardware units further comprise a module for changing the degree of said subsampling.

9. The device of claim 4, wherein said subsample representation is determined using spline interpolation.

10. The device of claim 4, wherein said subsample representation is determined using bi-cubic interpolation.

11. The device of claim 4, wherein said modifying the area is performed on a full resolution of at least one face or partial face region within the digital image.

12. The device of claim 4, wherein said red-eye filter comprises a plurality of sub-filters.

13. The device of claim 12, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between suspected red eye regions, or results of previous analysis of one or more other sub-filters, or combinations thereof.

14. The device of claim 4, further comprising:

a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

15. The device of claim 14, further comprising a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

16. The device of claim 14, further comprising an exposure analyzer for determining if the digital image was acquired in a condition indicative of the red-eye phenomenon.

17. The device of claim 1, wherein the one or more dedicated hardware units further comprise a subsampling analysis filter for providing hardware-based determination of changing a degree of subsampling based on image metadata information.

18. The device of claim 1, wherein the one or more dedicated hardware units further comprise a global luminance, color and/or grayscale histogram calculator for providing hardware-based calculations on raw and/or final image data.

19. A portable digital image capturing device having no photographic film, comprising:

a flash for providing illumination during image acquisition;

an optical system including a lens and an image sensor for capturing a main digital image; and

one or more dedicated hardware units, including;

one or both of a face or a partial face tracker for providing hardware-based identification of one or more face or partial face regions within a series of one or more relatively low resolution reference images, and predicting said one or more face or partial face regions within the main digital image.

20. The device of claim 19, wherein at least one face or partial face region within the digital image is not among said one or more face or partial face regions identified within the digital image, and wherein the one or more dedicated hardware units further comprise a face analyzer for providing hardware-based analysis of the one or more identified face or partial face regions.

21. The device of claim 20, wherein said analysis is performed in part on a full resolution partial face region and in part on a subsample resolution of at least one different partial face region.

22. The device of claim 19, wherein the one or more dedicated hardware units further comprise:

a red-eye filter for providing hardware-based modification of an area within the main digital image indicative of a red-eye phenomenon based on an analysis of said one or more face or partial face regions identified and predicted by the hardware-based face or partial face tracker.

23. The device of claim 22, wherein said red-eye filter is adapted based on a type of at least one of said one or more face or partial face regions identified within the digital image.

24. The device of claim 22, wherein the analysis is performed at least in part for determining said area.

25. The device of claim 22, wherein the analysis is performed at least in part for determining said modifying.

26. The device of claim 19, wherein said modifying the area is performed on a full resolution of at least one face or partial face region within the digital image.

27. The device of claim 19, wherein said red-eye filter comprises a plurality of sub-filters.

28. The device of claim 19, further comprising:

a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

29. The device of claim 28, further comprising a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

30. The device of claim 27, further comprising an exposure analyzer for determining if the digital image was acquired in a condition indicative of the red-eye phenomenon.

31. The device of claim 19, wherein the one or more dedicated hardware units further comprise a subsampling analysis filter for providing hardware-based determination of changing a degree of subsampling based on image metadata information.

32. The device of claim 19, wherein the one or more dedicated hardware units further comprise a global luminance, color and/or grayscale histogram calculator for providing hardware-based calculations on raw and/or final image data.

33. One or more dedicated hardware units for installation within a portable digital image capturing device having an optical system including a lens and an image sensor for capturing a digital image, and no photographic film, the one or more dedicated hardware units, comprising one or both of a face or a partial face detector subsystem for providing hardware-based identification of one or more face or partial face regions, or a combination thereof, within the digital image.

34. The one or more dedicated hardware units of claim 33, wherein at least one face or partial face region within the digital image is not among said one or more face or partial face regions identified within the digital image, and wherein the one or more dedicated hardware units further comprise a face analyzer for providing hardware-based analysis of the one or more identified face or partial face regions.

35. The one or more dedicated hardware units of claim 34, wherein said analysis is performed in part on a full resolution face or partial face region and in part on a subsample resolution of at least one different face or partial face region.

36. The one or more dedicated hardware units of claim 33, wherein the one or more dedicated hardware units further comprise a red-eye filter for providing hardware-based modification of an area within the image indicative of a red-eye phenomenon based on an analysis of a subsample representation comprising one or more face or partial face regions identified within the image, or both.

37. The one or more dedicated hardware units of claim 36, wherein said red-eye filter is adapted based on a type of at least one of said one or more face or partial face regions identified within the digital image.

38. The one or more dedicated hardware units of claim 36, wherein the analysis is performed at least in part for determining said area.

39. The one or more dedicated hardware units of claim 36, wherein the analysis is performed at least in part for determining said modifying.

40. The one or more dedicated hardware units of claim 36, further comprising a module for changing the degree of said subsampling.

41. The one or more dedicated hardware units of claim 36, wherein said subsample representation is determined using spline interpolation.

42. The one or more dedicated hardware units of claim 36, wherein said subsample representation is determined using bi-cubic interpolation.

43. The one or more dedicated hardware units of claim 36, wherein said modifying the area is performed on a full resolution of at least one face or partial face region within the digital image.

44. The one or more dedicated hardware units of claim 36, wherein said red-eye filter comprises a plurality of sub-filters.

45. The one or more dedicated hardware units of claim 44, wherein said subsampling for said sub-filters operating on selected regions of said image is determined by image size, a suspected red eye region size, filter computation complexity, empirical success rate of said sub-filter, empirical false detection rate of said sub-filter, falsing probability of said sub-filter, relations between suspected red eye regions, or results of previous analysis of one or more other sub-filters, or combinations thereof.

46. The one or more dedicated hardware units of claim 36, wherein the red eye filter further comprises:

a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

47. The one or more dedicated hardware units of claim 46, wherein the red eye filter further comprises a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

48. The one or more dedicated hardware units of claim 46, wherein the red eye filter further comprises an exposure analyzer for determining if the digital image was acquired in a condition indicative of the red-eye phenomenon.

49. The one or more dedicated hardware units of claim 33, further comprising a subsampling analysis filter for providing hardware-based determination of changing a degree of subsampling based on image metadata information.

50. The one or more dedicated hardware units of claim 33, further comprising a global luminance, color and/or grayscale histogram calculator for providing hardware-based calculations on raw and/or final image data.

51. One or more dedicated hardware units for installation within a portable digital image capturing device having an optical system including a lens and an image sensor for capturing a digital image, and no photographic film, the one or more dedicated hardware units, comprising one or both of a face or a partial face tracker for providing hardware-based identification of one or more face or partial face regions within a series of one or more relatively low resolution reference images, and predicting said one or more face or partial face regions within the main digital image.

52. The one or more dedicated hardware units of claim 51, wherein at least one face or partial face region within the digital image is not among said one or more face or partial face regions identified within the digital image, and wherein the one or more dedicated hardware units further comprise a face analyzer for providing hardware-based analysis of the one or more identified face or partial face regions.

53. The one or more dedicated hardware units of claim 52, wherein said analysis is performed in part on a full resolution partial face region and in part on a subsample resolution of at least one different partial face region.

54. The one or more dedicated hardware units of claim 51, wherein the one or more dedicated hardware units further comprise a red-eye filter for providing hardware-based modification of an area within the main digital image indicative of a red-eye phenomenon based on an analysis of said one or more face or partial face regions identified and predicted by the hardware-based face or partial face tracker.

55. The one or more dedicated hardware units of claim 54, wherein said red-eye filter is adapted based on a type of at least one of said one or more face or partial face regions identified within the digital image.

56. The one or more dedicated hardware units of claim 54, wherein the analysis is performed at least in part for determining said area.

57. The one or more dedicated hardware units of claim 54, wherein the analysis is performed at least in part for determining said modifying.

58. The one or more dedicated hardware units of claim 54, wherein said modification of the area is performed on a full resolution of at least one face or partial face region within the digital image.

59. The one or more dedicated hardware units of claim 54, wherein said red-eye filter comprises a plurality of sub-filters.

60. The one or more dedicated hardware units of claim 54, wherein the red eye filter further comprises:

a pixel locator for locating pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by the pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

61. The one or more dedicated hardware units of claim 60, wherein the red eye filter further comprises a falsing analyzer for further processing the digital image in a vicinity of the grouping for details indicative of an eye, and for enabling the pixel modifier in response thereto.

62. The one or more dedicated hardware units of claim 54, wherein the red eye filter further comprises an exposure analyzer for determining if the digital image was acquired in a condition indicative of the red-eye phenomenon.

63. The one or more dedicated hardware units of claim 51, wherein the one or more dedicated hardware units further comprise a subsampling analysis filter for providing hardware-based determination of changing a degree of subsampling based on image metadata information.

64. The one or more dedicated hardware units of claim 51, wherein the one or more dedicated hardware units further comprise a global luminance, color and/or grayscale histogram calculator for providing hardware-based calculations on raw and/or final image data.