Detecting red eye filter and apparatus using meta-data

• US 8,957,993 B2
• Filed: 03/06/2013
• Issued: 02/17/2015
• Est. Priority Date: 08/05/2003
• Status: Expired due to Fees

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: DeLuca, Michael, Prilutsky, Yury, Steinberg, Eran, Corcoran, Peter, Bigioi, Petronel, Pososin, Alexei
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): Morehead, III, John H
• Application Number:
  US13/787,783
• Publication Number:
  US 20130188077A1
• Time in Patent Office: 713 Days
• Field of Search: None
• US Class Current: 348/241
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/008 Local, e.g. shadow enhancement
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/6086 by scene illuminant, i.e. conditions at the time of picture capture, e.g. flash, optical filter used, evening, cloud, daylight, artificial lighting, white point measurement, colour temperature
• CPC Class Codes: H04N 1/62 Retouching, i.e. modification of isolated colours only or in isolated picture areas only
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise

Abstract

A method of filtering a red-eye phenomenon from an acquired digital image including a multiplicity of pixels indicative of color, the pixels forming various shapes of the image, includes analyzing meta-data information, determining one or more regions within the digital image suspected as including red eye artifact, and determining, based at least in part on the meta-data analysis, whether the regions are actual red eye artifact. The meta-data information may include information describing conditions under which the image was acquired, captured and/or digitized, acquisition device-specific information, and/film information.

Claims

1. Within a processor-based digital image acquisition device, a method of filtering a red-eye phenomenon from a digital image acquired with said same digital image acquisition device, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said processor-based digital image acquisition device;

using a red eye filter stored within the digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired and meta data information including image acquisition device specific information;

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found.

2. The method of claim 1, further comprising analyzing pixel information within one or more regions suspected as including red eye artifact based on said analyzing of information describing conditions under which the image was acquired, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said analyzing of information describing conditions under which the image was acquired.

3. The method of claim 1, further comprising analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, and wherein said pixel analysis is performed after said analyzing of information describing conditions under which the image was acquired.

4. The method of claim 1, further comprising obtaining anthropometrical information of human faces and said determining, based at least in part on said analyzing of information describing conditions under which the image was acquired, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

5. The method of claim 1, wherein said filtering method is executed within a portable image acquisition device, having no photographic film.

6. The method of claim 1, wherein said filtering method is executed as a post-processing step on an external computation device.

7. The method of claim 1, wherein said information describes conditions under which the image was acquired and comprises an indication of whether a flash was used when the image was acquired.

8. The method of claim 1, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria.

9. The method of claim 1, further comprising:

adjusting a pixel color within any of said regions wherein red eye artifact is determined;

outputting image data to a printer.

10. The method of claim 9, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria, and wherein the pixel color is adjusted within the printer.

11. The method of claim 1, wherein said performing shape analysis includes analyzing a combined shape of the red region and the first group of pixels in the vicinity of said red region.

12. The method of claim 1, wherein said first group of pixels substantially surrounds the red region.

37. The method of claim 1, wherein said analyzing meta-data information including image acquisition device-specific information comprises analyzing information relating to f-stop, aperture, exposure, gain, white balance or color transformation, or combinations thereof.

13. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from an acquired digital image that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

acquiring a digital image with a processor-based digital image acquisition device;

using a red eye filter stored within the processor-based digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired and meta-data information including image acquisition device-specific information;

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found.

14. The one or more non-transitory processor-readable storage devices of claim 13, wherein the method further comprises analyzing pixel information within one or more regions suspected as including red eye artifact based on said analyzing of information describing conditions under which the image was acquired, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said analyzing of information describing conditions under which the image was acquired.

15. The one or more non-transitory processor-readable storage devices of claim 13, wherein the method further comprises analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, and wherein said pixel analysis is performed after said analyzing of information describing conditions under which the image was acquired.

16. The one or more non-transitory processor-readable storage devices of claim 13, wherein the method further comprises obtaining anthropometrical information of human faces and said determining, based at least in part on said analyzing of information describing conditions under which the image was acquired, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

17. The one or more non-transitory processor-readable storage devices of claim 13, wherein said filtering method is executed within a portable image acquisition device, having no photographic film.

18. The one or more non-transitory processor-readable storage devices of claim 13, wherein said filtering method is executed as a post-processing step on an external computation device.

19. The one or more non-transitory processor-readable storage devices of claim 13, wherein said information describes conditions under which the image was acquired and comprises an indication of whether a flash was used when the image was acquired.

20. The one or more non-transitory processor-readable storage devices of claim 13, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria.

21. The one or more non-transitory processor-readable storage devices of claim 13, wherein the method further comprises:

adjusting a pixel color within any of said regions wherein red eye artifact is determined; and

outputting image data to a printer.

22. The one or more non-transitory processor-readable storage devices of claim 21, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria, and wherein the pixel color is adjusted within the printer.

23. The one or more non-transitory processor-readable storage devices of claim 13, wherein said performing shape analysis includes analyzing a combined shape of the red region and the first group of pixels in the vicinity of said red region.

24. The one or more non-transitory processor-readable storage devices of claim 13, wherein said first group of pixels substantially surrounds the red region.

38. The one or more non-transitory processor-readable storage devices of claim 13, wherein said analyzing meta-data information including image acquisition device-specific information comprises analyzing information relating to f-stop, aperture, exposure, gain, white balance or color transformation, or combinations thereof.

25. A portable processor-based image acquisition device, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image acquired with the device that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said processor-based digital image acquisition device;

analyzing within the device information describing conditions under which the image was acquired and meta-data information including image acquisition device-specific information; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found.

26. The device of claim 25, wherein the method further comprises analyzing pixel information within one or more regions suspected as including red eye artifact based on said analyzing of information describing conditions under which the image was acquired, and determining whether any of said one or more suspected regions continue to be suspected as including red eye artifact based on said pixel analysis, said pixel analysis being performed after said analyzing of information describing conditions under which the image was acquired.

27. The device of claim 25, wherein the method further comprises analyzing pixel information within said digital image, and determining whether said one or more regions are suspected as including red eye artifact based on said pixel analysis, and wherein said pixel analysis is performed after said analyzing of information describing conditions under which the image was acquired.

28. The device of claim 25, wherein the method further comprises obtaining anthropometrical information of human faces and said determining, based at least in part on said analyzing of information describing conditions under which the image was acquired, whether said regions are actual red eye artifact, being based further on said anthropometrical information.

29. The device of claim 25, wherein said filtering method is executed within a portable image acquisition device, having no photographic film.

30. The device of claim 25, wherein said filtering method is executed as a post-processing step on an external computation device.

31. The device of claim 25, wherein said information describes conditions under which the image was acquired and comprises an indication of whether a flash was used when the image was acquired.

32. The device of claim 25, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria.

33. The device of claim 25, wherein the method further comprises:

adjusting a pixel color within any of said regions wherein red eye artifact is determined; and

outputting image data to a printer.

34. The device of claim 33, wherein said determining whether said regions are actual red eye artifact is performed within a digital camera as a probability determination process based upon a plurality of criteria, and wherein the pixel color is adjusted within the printer.

35. The device of claim 25, wherein said performing shape analysis includes analyzing a combined shape of the red region and the first group of pixels in the vicinity of said red region.

36. The device of claim 25, wherein said first group of pixels substantially surrounds the red region.

39. The device of claim 25, wherein said analyzing meta-data information including image acquisition device-specific information comprises analyzing information relating to f-stop, aperture, exposure, gain, white balance or color transformation, or combinations thereof.

40. Within a processor-based digital image acquisition device, a method of filtering a red-eye phenomenon from a digital image acquired with said same digital image acquisition device, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said processor-based digital image acquisition device;

using a red eye filter stored within the digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including a spectral response curve of a sensor stored as a look-up table of an acquisition device with which the image was acquired.

41. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from an acquired digital image that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

using a red eye filter stored within the processor-based digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including a spectral response curve of a sensor stored as a look-up table of an acquisition device with which the image was acquired.

42. A portable processor-based image acquisition device, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image acquired with the device that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said processor-based digital image acquisition device;

analyzing within the device information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including a spectral response curve of a sensor stored as a look-up table of an acquisition device with which the image was acquired.

43. Within a processor-based digital image acquisition device, a method of filtering a red-eye phenomenon from a digital image acquired with said same digital image acquisition device, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said processor-based digital image acquisition device;

using a red eye filter stored within the digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including aperture, f-stop, or spectral response curve of sensor, or combinations thereof.

44. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from an acquired digital image that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

using a red eye filter stored within the processor-based digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including aperture, f-stop, or spectral response curve of sensor, or combinations thereof.

45. A portable processor-based image acquisition device, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image acquired with the device that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said processor-based digital image acquisition device;

analyzing within the device information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including aperture, f-stop, or spectral response curve of sensor, or combinations thereof.

46. Within a processor-based digital image acquisition device, a method of filtering a red-eye phenomenon from a digital image acquired with said same digital image acquisition device, said digital image comprising a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, the method comprising:

acquiring a digital image with said processor-based digital image acquisition device;

using a red eye filter stored within the digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including aperture, f-stop, color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

47. The method of claim 46, wherein said meta-data information comprises color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

48. One or more non-transitory processor-readable storage devices having embedded therein digital code for programming a processor to perform a method of filtering a red-eye phenomenon from an acquired digital image that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises:

acquiring a digital image with a processor-based digital image acquisition device;

using a red eye filter stored within the processor-based digital image acquisition device;

analyzing, with said red eye filter, information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including aperture, f-stop, color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

49. The one or more non-transitory processor-readable storage devices of claim 48, wherein said meta-data information comprises color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

50. A portable processor-based image acquisition device, comprising:

a lens;

an image sensor;

a processor; and

a memory having program code embodied therein for programming the processor to perform a method of filtering a red-eye phenomenon from a digital image acquired with the device that includes a multiplicity of pixels indicative of color, the pixels forming various shapes within the image, wherein the method comprises;

acquiring a digital image with said processor-based digital image acquisition device;

analyzing within the device information describing conditions under which the image was acquired; and

based at least in part on said analyzing, determining with said red eye filter whether one or more regions within said digital image are suspected as including red eye artifact, including one or both of performing shape analysis on a first group of pixels in the vicinity of a red region, including determining whether one or both of an iris shape or an eyebrow line is found, or performing color analysis on the first group of pixels in the vicinity of the red region, including determining whether a white area is found; and

analyzing meta-data information including image acquisition device-specific information including aperture, f-stop, color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.

51. The device of claim 50, wherein said meta-data information comprises color transformation due to spectral response curve of sensor, or overall CCD size, or combinations thereof.