Detecting red eye filter and apparatus using meta-data

• US 20040223063A1
• Filed: 08/05/2003
• Published: 11/11/2004
• Est. Priority Date: 10/09/1997
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Vision Limited (Adeia Inc.)
• Inventors: Steinberg, Eran, Prilutsky, Yury, Bigioi, Petronel, Corcoran, Peter, Pososin, Alexei, DeLuca, Michael J.
• Granted Patent: US 7,630,006 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

A method of filtering a red-eye phenomenon from an acquired digital image including a multiplicity of pixels indicative of color, the pixels forming various shapes of the image, includes analyzing meta-data information, determining one or more regions within the digital image suspected as including red eye artifact, and determining, based at least in part on the meta-data analysis, whether the regions are actual red eye artifact. The meta-data information may include information describing conditions under which the image was acquired, captured and/or digitized, acquisition device-specific information, and/film information.

Claims

1. A method of filtering a red-eye phenomenon from an acquired digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

(a) analyzing meta-data information including image acquisition device-specific information; and

(b) determining, based at least in part on said meta-data analysis, whether one or more regions within said digital image are suspected as including red eye artifact.

2. The method of claim 1, further comprising analyzing pixel information within one or more regions suspected as including red eye artifact based on said meta-data analysis, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said meta-data analysis.

3. The method of claim 1, further comprising analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed before said meta-data analysis.

4. The method of claim 1, further comprising analyzing pixel information within said digital image, and determining whether one or more same or different regions are suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed independent of said meta-data analysis.

5. The method of claim 1, the meta-data further comprising anthropometrical data.

6. The method of claim 1, said filtering being executed at least partially within a portable image acquisition device, having no photographic film.

7. The method of claim 1, said filtering being executed at least in part as a post-processing operation on an external computation device.

8. The method of claim 7, some or all of said meta-data analysis being performed on said image acquisition device.

9. The method of claim 8, further comprising marking certain regions determined to be suspected as including red eye artifact based on said analysis for correction at said post-processing step on said external computation device, said marking being performed on said image acquisition device.

10. The method of claim 7, some of said meta-data analysis or said suspected region determining or a combination thereof being performed on said image acquisition device, and some of said meta-data analysis or said suspected region determining or a combination thereof being performed at said post-processing step on said external computation device.

11. The method of claim 1, a lens being used to capture the image, said meta-data information comprising focal length of the lens at the time of acquisition.

12. The method of claim 11, said meta-data information further comprising focusing distance of the lens at time of acquisition.

13. The method of claim 11, said meta-data information comprising effective sensor size.

14. The method of claim 13, said actual red eye artifact being determined based on calculated expected size of said red eye artifact based on said meta-data information including said acquisition device-specific information.

15. The method of claim 14, said calculated expected size of said red eye artifact being defined as a range with a density probability function, the range being calculated based on depth of field.

16. The method of claim 13, said calculated expected size of said red eye artifact being defined as a range with a density probability function, the range being estimated.

17. The method of claim 13, said calculated expected size of said red eye artifact being defined as a range with a density probability function, said meta-data comprising anthropometrical data, and said range being determined by a statistical distribution of said anthropometrical data.

18. The method of claim 1, said determining operation including a probability determination process based upon a plurality of criteria.

19. The method of claim 1, further comprising:

(i) adjusting a pixel color within any of said regions wherein red eye artifact is determined; and

(ii) outputting image data to a printer.

20. The method of claim 19, the pixel color being adjusted within the printer.

21. The method of claim 1, said meta-data information comprising information describing conditions under which the image was acquired.

22. The method of claim 21, said meta-data information comprising an indication of whether a flash was used when the image was acquired.

23. The method of claim 21, said meta-data information comprising aperture at the time of the acquisition.

24. The method of claim 21, further comprising analyzing pixel information within one or more regions suspected as including red eye artifact based on said meta-data analysis, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said meta-data analysis.

25. The method of claim 21, the meta-data further comprising anthropometrical data.

26. The method of claim 21, a lens being used to capture the image, said meta-data information comprising focal length of the lens at the time of acquisition.

27. The method of claim 26, said meta-data information further comprising focusing distance of the lens at time of acquisition.

28. The method of claim 26, said meta-data information comprising effective sensor size.

29. The method of claim 21, further comprising:

(i) adjusting a pixel color within any of said regions wherein red eye artifact is determined; and

(ii) outputting image data to a printer.

30. The method of claim 29, the image being adjusted within the printer.

31. A method of filtering a red-eye phenomenon from an acquired digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

(a) analyzing meta-data information including a spectral response curve of a sensor of an acquisition device with which the image was acquired; and

(b) determining, based at least in part on said meta-data analysis, whether one or more regions within said digital image are suspected as including red eye artifact.

32. The method of claim 31, said meta-data information comprising a color transformation from raw sensor pixel values to saved image pixel values.

33. The method of claim 31, said determining operation including a probability determination process based upon a plurality of criteria.

34. The method of claim 31, said meta-data information comprising acquisition device-specific information.

35. The method of claim 34, said meta-data information comprising one or more conditions under which the image was acquired.

36. The method of claim 31, said meta-data comprising a color transformation from raw sensor pixel values to saved image pixel values.

37. The method of claim 36, color values of said pixels indicative of red eye color being calculated based on a spectral response of said red eye phenomenon.

38. The method-of claim 37, the spectral response of said red eye phenomenon being according to illumination by a spectral distribution of a camera flash unit.

39. The method of claim 38, the spectral distribution of said camera flash unit being as recorded by said sensor of said acquisition device with which said image was acquired.

40. The method of claim 39, said determining operation including comparing pixels indicative of red eye color and a multiplicity of pixels forming various shapes.

41. The method of claim 40, said pixels indicative of red eye color being calculated based on an inverse transformation of said color transformation from raw sensor pixel values to saved image pixel values.

42. A digital apparatus having no photographic film, comprising:

(a) a source of light for providing illumination during image capturing;

(b) a digital image capturing apparatus;

(c) at least one of an image display and an image output; and

(d) a red-eye filter for modifying pixels indicative of a red-eye phenomenon within said at least one of said image display and said image output.

43. The apparatus of claim 42, wherein said red-eye filter comprises:

a pixel locator for locating the pixels having a color indicative of the red-eye phenomenon;

a pixel analyzer for determining if a grouping of at least a portion of the pixels located by said pixel locator comprise one or more parameters indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

44. The apparatus of claim 43, further comprising a meta-data analyzer for determining whether to preclude a red-eye phenomenon determination based upon analysis of meta-data.

45. The apparatus of claim 44, said meta-data analysis being performed before said pixel analysis.

46. The apparatus of claim 44, said meta-data analysis being performed after said pixel analysis.

47. The apparatus of claim 44, said meta-data analyzer comprising an exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon.

48. The apparatus of claim 44, said meta-data comprising digital apparatus-specific information.

49. A portable digital apparatus having no photographic film, comprising:

(a) an integral flash for providing illumination during image recording;

(b) a digital image capturing apparatus for recording an image; and

(c) a red-eye filter for modifying an area within the image indicative of a red-eye phenomenon, and

(d) wherein whether said area within said image is indicative of said red-eye phenomenon is determined based at least in part on meta-data information,

50. The apparatus of claim 49, further comprising memory for recording said image after applying said filter for modifying pixels as a modified image.

51. The apparatus of claim 49, wherein said modified pixels are stored directly in said image by replacing said pixels within the image indicative of red-eye phenomenon to create said modified image.

52. The apparatus of claim 50, wherein said modified pixels are stored as an overlay of said image thus preserving the original said image.

53. The apparatus of claim 52, wherein said modified pixels are processed by an external device.

54. The apparatus of claim 53, wherein said external device is a personal computer.

55. The apparatus of claim 53, wherein said external device is a printer.

56. The apparatus of claim 49, further comprising an image output for downloading an integral image display for printing said image modified by said red-eye filter.

57. The apparatus of claim 49, the red-eye correction module generating an overlay for said pixels indicative of the red-eye phenomenon of the captured image for said at least one of image display and image output.

58. The apparatus of claim 49, wherein the pixels indicative of the red-eye phenomenon have a color and shape indicative of the red-eye phenomenon and the image is modified to change the color to a black color and further wherein:

said source of light selectively provides illumination during image capturing; and

said red-eye filter is enabled to modify the image in response to said source of light providing illumination during image capturing.

59. The apparatus of claim 58, further comprising an exposure control means for determining if the image was captured in a condition conducive to the red-eye phenomenon and for generating a red-eye signal in response thereto, wherein said red-eye filter is further enabled in response to the red-eye signal.

60. The apparatus of claim 49, wherein said red-eye filter further includes a false-detection avoidance apparatus which enables modification of the pixels indicative of the red-eye phenomenon in response to an absence of color indicative of the red-eye phenomenon with in a vicinity of and exclusive to the pixels.

61. The apparatus of claim 49, wherein said red-eye filter further includes a false-detection avoidance apparatus which enables modification of the pixels in response to one or more of a substantially white colored region, an iris ring and an eyebrow line within a vicinity of the pixels.

62. The apparatus of claim 49, wherein said red-eye filter detects said pixels within the image indicative of a red-eye phenomenon based on one or more of a substantially white colored region, an iris ring and an eyebrow line within a vicinity of the area.

63. The apparatus of claim 49, wherein said red-eye filter comprises:

a pixel locator for locating the pixels having a color indicative of the red-eye phenomenon;

a shape analyzer for determining if a grouping of at least a portion of the pixels located by said pixel locator comprise a shape indicative of the red-eye phenomenon; and

a pixel modifier for modifying the color of the pixels within the grouping.

64. The apparatus of claim 63, further comprising a false-detection analyzer for further processing the image in a vicinity of the grouping for details indicative of an eye, and for enabling said pixel modifier in response thereto.

65. The apparatus of claim 63, further comprising an exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon.

66. The apparatus of claim 63, wherein the red-eye filter further comprises an exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon.

67. The apparatus of claim 63, wherein said exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon including determining whether said light source was used during image recording.

68. The apparatus of claim 63, wherein said exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon including determining whether low ambient lighting conditions existed during image recording.

69. The apparatus of claim 63, wherein said exposure analyzer for determining if the image was recorded in a condition indicative of the red-eye phenomenon further comprises a distance analyzer for determining if the subject was at a relatively close distance to the apparatus during image recording.

70. The apparatus of claim 49, said meta-data information being analyzed in said red-eye phenomenon determination prior to a pixel analysis.

71. The apparatus of claim 49, said meta-data information being analyzed in sadi red-eye phenomenon determination following a pixel analysis.

72. The apparatus of claim 49, said meta-data information comprising digital apparatus-specific information.

73. The apparatus of claim 72, said meta-data information comprising information regarding conditions under which the image was acquired.

74. A method of filtering a red-eye phenomenon from an acquired digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

(a) analyzing meta-data information including information describing conditions under which the image was acquired; and

(b) determining, based at least in part on said meta-data analysis, whether one or more regions within said digital image are suspected as including red eye artifact.

75. The method of claim 74, further comprising analyzing pixel information within one or more regions suspected as including red eye artifact based on said meta-data analysis, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said meta-data analysis.

76. The method of claim 74, further comprising analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed before said meta-data analysis.

77. The method of claim 74, further comprising obtaining anthropometrical information of human faces and said determining, based at least in part on said meta-data analysis, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

78. The method of claim 74, said filtering method being executed within a portable image acquisition device, having no photographic film.

79. The method of claim 74, said filtered method being executed as a post-processing step on an external computation device.

80. The method of claim 74, said meta-data information describing the conditions under which the image was acquired comprising an indication of whether a flash was used when the image was acquired.

81. The method of claim 74, said determining whether said regions are actual red eye artifact being performed as a probability determination process based upon a plurality of criteria.

82. The method of claim 74, further comprising:

(i) adjusting a pixel color within any of said regions wherein red eye artifact is determined; and

(ii) outputting image data to a printer.

83. The method of claim 82, the pixel color being adjusted within the printer.