Method and apparatus for the automatic real-time detection and correction of red-eye defects in batches of digital images or in handheld appliances

• US 6,873,743 B2
• Filed: 03/29/2002
• Issued: 03/29/2005
• Est. Priority Date: 03/29/2001
• Status: Expired due to Term

• Current Assignee: Cedar Lane Technologies Inc.
• Original Assignee: FotoNation Holdings, LLC (Adeia Inc.)
• Inventors: Steinberg, Eran
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): Alavi, Amir
• Application Number:
  US10/113,871
• Publication Number:
  US 20020176623A1
• Time in Patent Office: 1,096 Days
• Field of Search: 382/115, 382/117, 382/162, 382/164, 382/165, 382/167, 382/168, 382/171, 382/172, 382/199, 382/278, 382274-277, 358/518, 358/520, 358/522, 348/207, 348/239, 348/370, 348/576
• US Class Current: 382/275
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

An automatic, red-eye detection and correction system for digital images capable of real-time processing of images, including a red-eye detector module that determines without user intervention if a red-eye defect exists. If a defect is located in an image the portion of the image surrounding the defect is passed to a correction module that de-saturates the red components of the defect while preserving the other color characteristics of the defect region. The invention is designed to minimize the computational resources required to detect and correct red-eye defects and thus is particularly suited to applications requiring real-time processing of large volumes of digital images prior to acquisition or printing. This system can operate on images stored on personal computers, commercial printers or inside digital cameras as part of the acquisition process, or prior to display on personal digital assistants, mobile phones and other digital imaging appliances.

Claims

1. An automated method for detecting red-eye defects in a digital image comprising:

(a) identifying and labeling an image segment that includes a potential red-eye defect in the image based on a red chrominance component and a luminance component of a color map;

(b) eliminating a segment as a candidate for having a red-eye defect, said eliminating including testing said segment and a boundary region of said image surrounding said segment for a plurality of attributes to determine if each said attribute exceeds a pre-determined threshold value; and

(c) recording a location, size and member pixels of a segment that survives said eliminating and is confirmed to have a red-eye defect of said image.

2. An automated method as recited in claim 1 wherein said attributes include the area of the segment.

3. An automated method as recited in claim 1 wherein said attributes include an elongation ratio of the segment.

4. An automated method as recited in claim 1 wherein said attributes include a compaction ratio of the segment.

5. An automated method as recited in claim 1 wherein said attributes of said bounding region include a number of pixels in the bounding region of the segment matching a color of human skin.

6. An automated method as recited in claim 1 wherein said attributes of said bounding region include a number of pixels in the bounding region of the segment matching a whitish color of a corneal region of an eye.

7. An automated method as recited in claim 1 wherein said attributes of said bounding region include an internal contrast ratio of a combined segment and bounding region.

8. An automated method as recited in claim 1 wherein said threshold values include an upper threshold value and a lower threshold value and said testing threshold values is modified to allow for both said upper and lower threshold value, for allowing for both a maximum and a minimum threshold value for some of said attributes of a segment.

9. An automated method as recited in claim 1 wherein said threshold values include a plurality of upper threshold values and a plurality of lower threshold values associated with said attributes, wherein said eliminating depends on which of said thresholds is exceeded.

10. An automated method as recited in claim 9 wherein said eliminating further includes testing for an additional characteristic, said characteristic being dependent on which of said thresholds is exceeded.

11. An automated method as recited in claim 1 wherein, within a predetermined range, the value of a particular attribute of a segment may directly determine said threshold value of subsequent test attributes.

12. An automated method as in claim 1 wherein said eliminating is based on a fuzzy relationship between two or more said attributes of said segment.

13. An automated method as recited in claim 1 further comprising correcting a confirmed red-eye defect.

14. An automated method as recited in claim 13 wherein said correcting the confirmed red-eye defect includes the steps of

(a) creating a cumulative chrominance histogram of said confirmed red-eye defect of said image;

(b) determining a threshold value for red colors;

(c) clipping all red components of pixels in said confirmed red-eye defect which are larger than said threshold value;

(d) maintaining all other luminance and chrominance values of said pixels as they are thus retaining the original color of said confirmed red-eye region; and

(e) Repeating the above steps (a)-(d) for said each defect out of said defects.

15. An automated method for detecting a red-eye defect in a digital image as recited in claim 1 wherein said method is implemented in a digital camera as part of a post acquisition process.

16. An automated method for detecting a red-eye defect in a digital image as implemented in a digital camera as part of a post acquisition process as recited in claim 15 wherein a criteria for activating the said method is acquiring said image using a flash.

17. An automated method for detecting red-eye defects in a digital image as implemented in a digital camera as part of a post acquisition process as recited in claim 15 wherein a criteria for activating said method is a distance of a subject matter to the camera when said image is acquired.

18. An automated method for detecting red-eye defects in a digital image as implemented in a digital camera as part of a post acquisition process as recited in claim 15 further comprising the steps:

(a) notifying a user of the existence of said red-eye defects prior to saving the image; and

(b) providing the user with a choice wherein the user can take action to correct the red-eye prior to saving the image.

19. An automated method for detecting a red-eye defect in a digital image as recited in claim 1 wherein said method is implemented in high volume photographic printing machines as part of the printing process of images.

20. An automated method for detecting red-eye defects in a digital image as implemented in a high volume photographic printing machine as part of the printing process of images as recited in claim 19 further comprising the steps:

(a) notifying an operator of said photographic printing machine of the existence of said red-eye defects prior to printing the image; and

(b) providing the operator with a choice wherein the operator can take action to correct the red-eye prior to printing the image.

21. An automated method of correcting red-eye defects in detected red-eye regions in a digital image for each individual defect out of said red-eye defects in said detected regions comprising:

(a) creating a cumulative chrominance histogram of said detected red-eye region of said image;

(b) determining a threshold value for red colors;

(c) clipping all red components of pixels in said detected red-eye region which are larger than a threshold value; and

(d) maintaining all other luminance and chrominance values of said pixels as they are thus retaining an original color of said detected red-eye region.

22. A user interactive red-eye detection system of digital images comprising:

(a) apparatus for automatic detection of the existence of a red-eye region having a red-eye defect in an image including

(i) first apparatus to segment and label a potential red-eye region in said image based on a red chrominance component and a luminance component of a color map;

(ii) second apparatus to eliminate a segment which is considered a said potential red-eye region based on testing threshold values for a range of attributes of said segment and a region of the image immediately surrounding said segment, known as a bounding region;

(iii) third apparatus to record a location, size and member pixels of a segment that is not eliminated by the second apparatus and is thus considered to be a potential red-eye region;

(b) apparatus for indicating to a user that an area of said image has been designated as a potential red-eye region;

(c) apparatus providing a user facility to choose to direct said system to correct said defect or choose to direct said system not to correct said defect;

(d) apparatus for automatic correction of said defect including

(i) first apparatus for creating a cumulative chrominance histogram of said detected red-eye region of said image;

(ii) second apparatus for determining a threshold value for the red chrominance component for said red-eye region;

(iii) third apparatus for clipping all red components of pixels in said detected red-eye region which are larger than said threshold values; and

(iv) fourth apparatus for maintaining all other luminance and chrominance values of said pixels of said red-eye region as they are thus retaining the original colors of said detected red-eye region.

23. A user interactive red-eye detection system as recited in claim 22 wherein said apparatus for indicating to a user that an area of said image has been designated as a potential red-eye region provides a visual highlighting of said potential red-eye region.

24. A user interactive red-eye detection system as recited in claim 22 wherein said apparatus for indicating to a user that an area of said image has been designated as a potential red-eye region provides an audible signal.

25. A user interactive red-eye detection system as recited in claim 22 wherein said user is an operator of a digital printing machine.

26. A user interactive red-eye detection system as recited in claim 22 wherein said user is an operator of a computer image editing application.

27. A user interactive red-eye detection system as recited in claim 22 wherein said user is an operator of a digital camera wherein said digital image is an image captured by said digital camera.