Method, apparatus, and program for detecting red eye

• US 20060204052A1
• Filed: 03/13/2006
• Published: 09/14/2006
• Est. Priority Date: 03/11/2005
• Status: Abandoned Application

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fuji Photo Film Co Limited (Fujifilm Holdings Corporation)
• Inventors: Yokouchi, Kouji
• Application Number:
  US11/373,439
• Publication Number:
  US 20060204052A1
• Time in Patent Office: Days
• US Class Current: 382/117
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

A process for detecting red eyes within faces included within photographic images and the like includes the steps of: detecting red eye candidates, which may be estimated to be red eyes, by searching the entire image (red eye candidate detecting process); detecting a face that includes the detected red eye candidates, by searching the vicinity of the red eye candidates (face detecting process); estimating which of the red eye candidates are red eyes, by searching within search regions in the vicinities of the red eye candidates at a higher accuracy than that employed during detection of the red eye candidates (red eye estimating process); and confirming whether the results of the red eye estimating process are correct, by judging whether the red eye candidates estimated to be red eyes are the corners of eyes.

Claims

1. A red eye detecting method for detecting red eyes, comprising the steps of:

detecting red eye candidates, by discriminating characteristics inherent to pupils, of which at least a portion is displayed red, from within an image;

detecting faces that include the red eye candidates, by discriminating characteristics inherent to faces, from among characteristics of the image in the vicinities of the red eye candidates;

estimating that the red eye candidates included in the detected faces are red eyes; and

confirming the results of estimation, by judging whether the red eye candidates are the corners of eyes.

2. A red eye detecting method as defined in claim 1, wherein the estimating step is realized by:

discriminating characteristics inherent to pupils, of which at least a portion is displayed red, from the characteristics of the image in the vicinities of the red eye candidates at a higher accuracy than that employed during the detection of the red eye candidates; and

estimating that the red eye candidates having the characteristics are red eyes.

3. A red eye detecting method as defined in claim 1, wherein the red eye candidates are detected by:

setting judgment target regions within the image;

obtaining characteristic amounts that represent characteristics inherent to pupils having regions displayed red from within the judgment target regions;

calculating scores according to the obtained characteristic amounts; and

judging that the image within the judgment target region represents a red eye candidate when the score is greater than or equal to a first threshold value; and

confirming the results of estimation only for red eye candidates, of which the score is less than a second threshold value, which is greater than the first threshold value.

4. A red eye detecting method as defined in claim 3, further comprising the steps of:

defining characteristic amounts that represent likelihood of being a dark pupil, a score table, and a threshold value, by learning sample images of dark pupils and sample images of subjects other than dark pupils, with a machine learning technique;

calculating the characteristic amounts from within the judgment target regions;

calculating scores corresponding to the characteristic amounts according to the score table; and

detecting dark pupils, by judging that the image within the judgment target region represents a dark pupil when the score is greater than or equal to the threshold value.

5. A red eye detecting method as defined in claim 1, wherein:

a pixel value profile is obtained, of pixels along a straight line that connects two red eye candidates, which have been estimated to be red eyes; and

the judgment regarding whether the red eye candidates are the corners of eyes is performed employing the pixel value profile.

6. A red eye detecting method as defined in claim 5, wherein:

the judgment is performed by confirming which profile the pixel value profile is, from among;

a profile in the case that the two red eye candidates are true red eyes;

a case that the two red eye candidates are the inner corners of eyes; and

a case that the two red eye candidates are the outer corners of eyes.

7. A red eye detecting apparatus, comprising:

red eye candidate detecting means for detecting red eye candidates, by discriminating characteristics inherent to pupils, of which at least a portion is displayed red, from within an image;

face detecting means for detecting faces that include the red eye candidates, by discriminating characteristics inherent to faces, from among characteristics of the image in the vicinities of the red eye candidates;

red eye estimating means for estimating that the red eye candidates included in the detected faces are red eyes; and

result confirming means for confirming the results of estimation, by judging whether the red eye candidates are the corners of eyes.

8. A red eye detecting apparatus as defined in claim 7, wherein:

the red eye estimating means discriminates characteristics inherent to pupils, of which at least a portion is displayed red, from the characteristics of the image in the vicinities of the red eye candidates at a higher accuracy than that employed during the detection of the red eye candidates; and

estimates that the red eye candidates having the characteristics are red eyes.

9. A red eye detecting apparatus as defined in claim 7, wherein the red eye candidate detecting means detects red eye candidates by:

setting judgment target regions within the image;

obtaining characteristic amounts that represent characteristics inherent to pupils having regions displayed red from within the judgment target regions;

calculating scores according to the obtained characteristic amounts; and

judging that the image within the judgment target region represents a red eye candidate when the score is greater than or equal to a first threshold value; and

the result confirming means confirms the results of estimation only for red eye candidates, of which the score is less than a second threshold value, which is greater than the first threshold value.

10. A red eye detecting apparatus as defined in claim 9, wherein:

the result confirming means further comprises dark pupil detecting means for detecting dark pupils within the face region detected by the face detecting means; and

the judgment regarding whether the red eye candidates, which have been estimated to be red eyes, are the corners of eyes is judged in the case that dark pupils are detected.

11. A red eye detecting apparatus as defined in claim 10, wherein the dark pupil detecting means detects dark pupils by:

defining characteristic amounts that represent likelihood of being a dark pupil, a score table, and a threshold value, by learning sample images of dark pupils and sample images of subjects other than dark pupils, with a machine learning technique;

calculating the characteristic amounts from within the judgment target regions;

calculating scores corresponding to the characteristic amounts according to the score table; and

judging that the image within the judgment target region represents a dark pupil when the score is greater than or equal to the threshold value.

12. A red eye detecting apparatus as defined in claim 7, wherein:

the result confirming means comprises a profile obtaining means for obtaining a pixel value profile of pixels along a straight line between two red eye candidates, which have been estimated to be red eyes by the red eye estimating means; and

the judgment regarding whether the red eye candidates are the corners of eyes is performed employing the pixel value profile obtained by the profile obtaining means.

13. A red eye detecting apparatus as defined in claim 12, wherein:

the result confirming means judges whether the red eye candidates are the corners of eyes, by confirming which profile the pixel value profile is, from among;

a profile in the case that the two red eye candidates are true red eyes;

a case that the two red eye candidates are the inner corners of eyes; and

a case that the two red eye candidates are the outer corners of eyes.

14. A computer readable medium having a red eye detecting program recorded therein that causes a computer to execute:

a red eye candidate detecting procedure for detecting red eye candidates, by discriminating characteristics inherent to pupils, of which at least a portion is displayed red, from within an image;

a face detecting procedure for detecting faces that include the red eye candidates, by discriminating characteristics inherent to faces, from among characteristics of the image in the vicinities of the red eye candidates;

a red eye estimating procedure for estimating that the red eye candidates included in the detected faces are red eyes; and

a result confirming procedure for confirming the results of estimation, by judging whether the red eye candidates are the corners of eyes.

15. A computer readable medium as defined in claim 14, wherein:

the red eye estimating procedure discriminates characteristics inherent to pupils, of which at least a portion is displayed red, from the characteristics of the image in the vicinities of the red eye candidates at a higher accuracy than that employed during the detection of the red eye candidates; and

estimates that the red eye candidates having the characteristics are red eyes.

16. A computer readable medium as defined in claim 14, wherein the red eye candidate detecting procedure detects red eye candidates by:

setting judgment target regions within the image;

obtaining characteristic amounts that represent characteristics inherent to pupils having regions displayed red from within the judgment target regions;

calculating scores according to the obtained characteristic amounts; and

judging that the image within the judgment target region represents a red eye candidate when the score is greater than or equal to a first threshold value; and

the result confirming procedure confirms the results of estimation only for red eye candidates, of which the score is less than a second threshold value, which is greater than the first threshold value.

17. A computer readable medium as defined in claim 16, wherein:

the result confirming procedure detects dark pupils within the face region detected by the face detecting procedure; and

the judgment regarding whether the red eye candidates, which have been estimated to be red eyes, are the corners of eyes is judged in the case that dark pupils are detected.

18. A computer readable medium as defined in claim 17, wherein the result confirming procedure detects the dark pupils by:

defining characteristic amounts that represent likelihood of being a dark pupil, a score table, and a threshold value, by learning sample images of dark pupils and sample images of subjects other than dark pupils, with a machine learning technique;

calculating the characteristic amounts from within the judgment target regions;

calculating scores corresponding to the characteristic amounts according to the score table; and

judging that the image within the judgment target region represents a dark pupil when the score is greater than or equal to the threshold value.

19. A computer readable medium as defined in claim 14, wherein:

the result confirming procedure comprises the step of obtaining a pixel value profile of pixels along a straight line between two red eye candidates, which have been estimated to be red eyes by the red eye estimating means; and

the judgment regarding whether the red eye candidates are the corners of eyes is performed employing the obtained pixel value profile.

20. A computer readable medium as defined in claim 19, wherein:

the result confirming procedure judges whether the red eye candidates are the corners of eyes, by confirming which profile the pixel value profile is, from among;

a profile in the case that the two red eye candidates are true red eyes;

a case that the two red eye candidates are the inner corners of eyes; and

a case that the two red eye candidates are the outer corners of eyes.