Detecting red eye filter and apparatus using meta-data

• US 9,025,054 B2
• Filed: 02/19/2013
• Issued: 05/05/2015
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: DeLuca, Michael J., Prilutsky, Yury, Steinberg, Eran, Corcoran, Peter, Bigioi, Petronel, Pososin, Alexei
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): Morehead, III, John H
• Application Number:
  US13/771,001
• Publication Number:
  US 20130162869A1
• Time in Patent Office: 805 Days
• Field of Search: None
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 25/68 applied to defects

Abstract

A method of filtering a red-eye phenomenon from an acquired digital image including a multiplicity of pixels indicative of color, the pixels forming various shapes of the image, includes analyzing meta-data information, determining one or more regions within the digital image suspected as including red eye artifact, and determining, based at least in part on the meta-data analysis, whether the regions are actual red eye artifact. The meta-data information may include information describing conditions under which the image was acquired, captured and/or digitized, acquisition device-specific information, and/film information.

Claims

1. Within a cell phone equipped with a digital camera, a method of filtering a red-eye phenomenon from a digital image acquired with said same cell phone equipped with said digital camera, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said cell phone equipped with said digital camera;

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said digital camera component of said cell phone, including f-stop, aperture, exposure, gain, white balance, or color transformation, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

2. The method of claim 1, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

3. The method of claim 1, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

4. The method of claim 1, further comprising analyzing pixel information within one or more regions suspected as including red eye artifact based on said analyzing of information describing conditions under which the image was acquired, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said analyzing of information describing conditions under which the image was acquired.

5. The method of claim 1, further comprising analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, and wherein said pixel analysis is performed after said analyzing of information describing conditions under which the image was acquired.

6. The method of claim 1, further comprising obtaining anthropometrical information of human faces and said determining, based at least in part on said analyzing of information describing conditions under which the image was acquired, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

7. The method of claim 1, wherein said filtering method is executed at least in part on said cell phone equipped with said digital camera.

8. The method of claim 1, wherein said filtering method is executed entirely on said cell phone equipped with said digital camera.

9. The method of claim 1, wherein said information describes conditions under which the image was acquired and comprises an indication of whether a flash was used when the image was acquired.

10. The method of claim 1, wherein said determining whether said regions are actual red eye artifact is performed within the cell phone equipped with the digital camera as a probability determination process based upon a plurality of criteria.

11. The method of claim 1, further comprising:

adjusting a pixel color within any of said regions wherein red eye artifact is determined;

outputting image data to an external device.

12. The method of claim 11, wherein said determining whether said regions are actual red eye artifact is performed within the cell phone equipped with the digital camera as a probability determination process based upon a plurality of criteria, and wherein the pixel color is adjusted within the external device.

13. The method of claim 1, wherein said performing shape analysis includes analyzing a combined shape of the red region and the first group of pixels in the vicinity of said red region.

14. The method of claim 1, wherein said first group of pixels substantially surrounds the red region.

15. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from a digital image that is acquired with a cell phone equipped with a digital camera and that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including f-stop, aperture, exposure, gain, white balance or color transformation, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

16. The one or more non-transitory processor-readable storage devices of claim 15, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

17. The one or more non-transitory processor-readable storage devices of claim 15, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

18. The one or more non-transitory processor-readable storage devices of claim 15, wherein the method further comprises analyzing pixel information within one or more regions suspected as including red eye artifact based on said analyzing of information describing conditions under which the image was acquired, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said analyzing of information describing conditions under which the image was acquired.

19. The one or more non-transitory processor-readable storage devices of claim 15, wherein the method further comprises analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, and wherein said pixel analysis is performed after said analyzing of information describing conditions under which the image was acquired.

20. The one or more non-transitory processor-readable storage devices of claim 15, wherein the method further comprises obtaining anthropometrical information of human faces and said determining, based at least in part on said analyzing of information describing conditions under which the image was acquired, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

21. The one or more non-transitory processor-readable storage devices of claim 15, wherein said filtering method is executed at least in part on said cell phone equipped with said digital camera.

22. The one or more non-transitory processor-readable storage devices of claim 15, wherein said filtering method is executed entirely on said cell phone equipped with said digital camera.

23. The one or more non-transitory processor-readable storage devices of claim 15, wherein said information describes conditions under which the image was acquired and comprises an indication of whether a flash was used when the image was acquired.

24. The one or more non-transitory processor-readable storage devices of claim 15, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria.

25. The one or more non-transitory processor-readable storage devices of claim 15, wherein the method further comprises:

adjusting a pixel color within any of said regions wherein red eye artifact is determined;

and outputting image data to an external device.

26. The one or more non-transitory processor-readable storage devices of claim 25, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria, and wherein the pixel color is adjusted within the printer.

27. The one or more non-transitory processor-readable storage devices of claim 15, wherein said performing shape analysis includes analyzing a combined shape of the red region and the first group of pixels in the vicinity of said red region.

28. The one or more non-transitory processor-readable storage devices of claim 15, wherein said first group of pixels substantially surrounds the red region.

29. A cell phone equipped with a digital camera, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image that is acquired with the cell phone equipped with said digital camera and that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said cell phone equipped with said digital camera;

analyzing within the cell phone equipped with said digital camera information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including f-stop, aperture, exposure, gain, white balance or color transformation, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

30. The cell phone of claim 29, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

31. The cell phone of claim 29, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

32. The cell phone of claim 29, wherein the method further comprises analyzing pixel information within one or more regions suspected as including red eye artifact based on said analyzing of information describing conditions under which the image was acquired, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said analyzing of information describing conditions under which the image was acquired.

33. The cell phone of claim 29, wherein the method further comprises analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, and wherein said pixel analysis is performed after said analyzing of information describing conditions under which the image was acquired.

34. The cell phone of claim 29, wherein the method further comprises obtaining anthropometrical information of human faces and said determining, based at least in part on said analyzing of information describing conditions under which the image was acquired, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

35. The cell phone of claim 29, wherein said filtering method is executed within a portable image acquisition device, having no photographic film.

36. The cell phone of claim 29, wherein said filtering method is executed as a post-processing step on an external computation device.

37. The cell phone of claim 29, wherein said information describes conditions under which the image was acquired and comprises an indication of whether a flash was used when the image was acquired.

38. The cell phone of claim 29, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria.

39. The cell phone of claim 29, wherein the method further comprises:

adjusting a pixel color within any of said regions wherein red eye artifact is determined; and

outputting image data to an external device.

40. The cell phone of claim 39, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria, and wherein the pixel color is adjusted within the external device.

41. The cell phone of claim 29, wherein said performing shape analysis includes analyzing a combined shape of the red region and the first group of pixels in the vicinity of said red region.

42. The cell phone of claim 29, wherein said first group of pixels substantially surrounds the red region.

43. Within a cell phone equipped with a digital camera, a method of filtering a red-eye phenomenon from a digital image acquired with said same cell phone equipped with said digital camera, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said cell phone equipped with said digital camera;

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including a spectral response curve of a sensor stored as a look-up table of an acquisition device with which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

44. The method of claim 43, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

45. The method of claim 43, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

46. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from a digital image that was acquired by a cell phone equipped with a digital camera and that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including a spectral response curve of a sensor stored as a look-up table of an acquisition device with which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

47. The one or more non-transitory processor-readable storage devices of claim 46, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

48. The one or more non-transitory processor-readable storage devices of claim 46, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

49. A cell phone equipped with a digital camera, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image that was acquired with the cell phone equipped with said digital camera and that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said cell phone equipped with said digital camera;

analyzing within the device information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including a spectral response curve of a sensor stored as a look-up table of the cell phone equipped with the digital camera with which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

50. The cell phone of claim 49, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

51. The cell phone of claim 49, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

52. Within a cell phone equipped with a digital camera, a method of filtering a red-eye phenomenon from a digital image acquired with said same cell phone equipped with said digital camera, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said cell phone equipped with said digital camera;

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired and meta-data information including information specific to said camera component of said cell phone, aperture, f-stop, or spectral response curve of sensor, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

53. The method of claim 52, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

54. The method of claim 52, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

55. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from a digital image acquired by a cell phone equipped with a digital camera that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, aperture, f-stop, or spectral response curve of sensor, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

56. The one or more non-transitory processor-readable storage devices of claim 55, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

57. The one or more non-transitory processor-readable storage devices of claim 55, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

58. A cell phone equipped with a digital camera, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image acquired with the digital camera component of said cell phone that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said cell phone equipped with said digital camera;

analyzing within the cell phone equipped with said digital camera information describing conditions under which the image was acquired, and meta-data information including information specific to said digital camera component of said cell phone, aperture, f-stop, or spectral response curve of sensor, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

59. The cell phone of claim 58, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

60. The cell phone of claim 58, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

61. Within a cell phone equipped with a digital camera, a method of filtering a red-eye phenomenon from a digital image acquired with said same cell phone equipped with said digital camera, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said cell phone equipped with said digital camera;

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, aperture, f-stop, color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

62. The method of claim 61, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

63. The method of claim 61, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

64. The method of claim 61, wherein said meta-data information comprises color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

65. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from a digital image acquired by a cell phone equipped with a digital camera that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

using a red eye filter stored within said cell phone equipped with said digital camera;

analyzing, with said red eye filter, information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including aperture, f-stop, color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

66. The one or more non-transitory processor-readable storage devices of claim 65, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

67. The one or more non-transitory processor-readable storage devices of claim 65, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

68. The one or more non-transitory processor-readable storage devices of claim 65, wherein said meta-data information comprises color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

69. A cell phone equipped with a digital camera, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image acquired with the device that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said cell phone equipped with said digital camera;

analyzing within the cell phone equipped with the digital camera information describing conditions under which the image was acquired, and meta-data information including information specific to said camera component of said cell phone, including aperture, f-stop, color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, or performing color analysis on the first group of pixels in the vicinity of the red region.

70. The cell phone of claim 69, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said shape analysis including determining whether one or both of an iris shape or an eyebrow line is found.

71. The cell phone of claim 69, wherein said determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact comprises performing said color analysis including determining whether a white area is found.

72. The cell phone of claim 69, wherein said meta-data information comprises color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.