Computer program product for redeye detection

• US 6,292,574 B1
• Filed: 08/29/1997
• Issued: 09/18/2001
• Est. Priority Date: 08/29/1997
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo, Schildkraut, Jay S., Gray, Robert T.
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US08/919,560
• Time in Patent Office: 1,481 Days
• Field of Search: 382/103, 382/117, 382/118, 382/159, 382/167, 382/190, 382/193, 382/173, 382/274
• US Class Current: 382/117
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: H04N 1/624 Red-eye correction control of camera based on recognised facial parts H04N23/611

Abstract

A computer program product for detecting eye color defects of a subject in an image due to flash illumination comprises: a computer readable storage medium having a computer program stored thereon for performing the steps of detecting skin colored regions in a digital image; searching the skin colored regions for groups of pixels with color characteristic of redeye defect; and correcting color of the pixels based on a location of redeye defect found in step (b).

Claims

1. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a Computer program stored thereon for performing the steps of;

(a) searching an entire digital image for detecting one or more skin colored regions in the digital image that have a characteristic of a face;

(b) resizing each skin colored region based on one or more predetermined facial dimensions to form one or more resized skin colored regions;

(c) searching the resized skin colored regions for groups of pixels with color characteristic of redeye defect; and

(d) correcting color of the pixels based on a location of redeye defect found in step (c).

2. The computer program product as in claim 1, wherein step (a) includes:

(a1) segmenting the digital image into continuous regions of uniform color and assigning a score indicating probability that the region corresponds to skin for forming a candidate skin region.

3. The computer program product as in claim 2, wherein step (a1) includes merging two or more candidate skin regions based on their similarity of color and degree of connectivity.

4. The computer program product as in claim 2, wherein step (a1) includes determining probability that the candidate region is a face based on its shape.

15. The computer program product as claimed in claim 1 wherein resizing in step (b) is based on a predetermined distance between the eyes of a typical face.

16. The computer program product as claimed in claim 15 wherein resizing in step (b) is based on a predetermined ratio between the width of a typical face and the distance between the eyes of a typical face.

17. The computer program product as claimed in claim 1 wherein step (c) further comprises comparing each redeye defect to one or more facial properties of the resized skin colored regions.

18. The computer program product as claimed in claim 17 wherein each redeye defect is compared to an eye size expected for the resized skin colored region.

19. The computer program product as claimed in claim 1 wherein step (c) further comprises detecting pairs of redeye defects by comparing each redeye defect in the pair to one or more facial properties of the resized skin colored regions.

20. The computer program product as claimed in claim 19 wherein the distance between each redeye defect in a pair is compared to a distance expected for the resized skin colored region.

21. The computer program product as claimed in claim 19 wherein each redeye defect in the pair is compared to an eye size expected for the resized skin colored region.

5. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to one or more properties of a face to determine whether a pair of the candidate redeye defects is consistent with the size expected of an eye based on distance between the pair of candidate redeye defects; and

(c) correcting color of the pixels based on a location of redeye defect found in step (b).

6. The computer program product as in claim 5, wherein the match of step (b) further includes determining for a pair of candidate redeye defects the tilt expected of an eye based on the tilt of the pair of candidate redeye defects.

7. The computer program product as in claim 5, wherein the match of step (b) includes correlating a region around the candidate redeye defects with an eye template.

8. The computer program product as in claim 7, wherein the match of step (b) includes correlating the region around the candidate redeye defects with individual zones of the eye template.

9. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) detecting skin colored regions in a digital image;

(b) resizing each skin colored region based on one or more predetermined facial dimensions to form one or more resized skin colored regions;

(c) searching the resized skin colored regions for groups of pixels with color characteristic of redeye defect for forming a candidate redeye defect;

(d) evaluating the groups of pixels relative to one or more properties of a face to determine whether a pair of the candidate redeye defects is consistent with a facial property of an eye pair; and

(e) correcting color of the pixels based on a location of the redeye defect.

10. The computer program product as in claim 9, wherein step (a) includes:

(a1) segmenting the digital image into continuous regions of uniform color and assigning a score indicating probability that the region corresponds to skin for forming a candidate skin region.

11. The computer program product as in claim 9, wherein step (a1) includes merging two or more candidate skin regions based on their similarity of color and degree of connectivity.

12. The computer program product as in claim 10, wherein step (a1) includes determining probability that the candidate region is a face based on its shape.

13. The computer program product as in claim 9, wherein the match of step (d) further includes determining for a pair of candidate redeye defects the size and tilt expected of an eye based on distance between and the tilt of the pair of candidate redeye defects.

14. The computer program product as in claim 13, wherein the match of step (d) includes correlating the region around the candidate redeye defects with individual zones of the eye template.

22. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to the determined size of a face to determine the extent to which the size of a candidate redeye defect matches an expected size of an eye pupil; and

(c) correcting color of the pixels based on the evaluation of redeye defect performed in step (b).

23. The computer program product as in claim 22, wherein the evaluation of step (b) includes correlating a region around the candidate redeye defects with an eye template.

24. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to the size of a face to determine the extent to which the distance between a pair of the candidate redeye defects matches an expected distance between a pair of eyes; and

(c) correcting color of the pixels based on a location of redeye defect found in step (b).

25. The computer program product as in claim 24, wherein the evaluation of step (b) includes correlating a region around the candidate redeye defects with an eye template.

26. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) detecting groups of pixels with color characteristic of redeye defect for forming a pair of candidate redeye defects;

(b) evaluating the groups of pixels relative to a distance between the redeye defects and a tilt of the redeye defects to determine the extent to which each candidate redeye defect is part of an eye; and

(c) correcting color of the pixels based on the evaluation of redeye defect found in step (b).

27. The computer program product as in claim 26, wherein the evaluation of step (b) includes correlating a region around the candidate redeye defects with an eye template.

28. A computer program product for detecting eye color defects of a subject in an image due to flash illumination, comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to one or more properties of a face to determine whether a pair of the candidate redeye defects is consistent with the tilt expected of an eye based on the tilt of the pair of candidate redeye defects; and

(c) correcting color of the pixels based on a location of redeye defect found in step (b).

29. The computer program product as in claim 28, wherein the match of step (b) further includes determining for a pair of candidate redeye defects the size expected of an eye based on distance between the pair of candidate redeye defects.

30. The computer program product as in claim 28, wherein the match of step (b) includes correlating a region around the candidate redeye defects with an eye template.

31. The computer program product as in claim 30, wherein the match of step (b) includes correlating the region around the candidate redeye defects with individual zones of the eye template.

32. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) searching an entire digital image for detecting one or more skin colored regions in the digital image that have a characteristic of a face;

(b) resizing each skin colored region based on one or more predetermined facial dimensions to form one or more resized skin colored regions;

(c) searching the resized skin colored regions for groups of pixels with color characteristic of redeye defect; and

(d) correcting color of the pixels based on a location of redeye defect found in step (c).

33. The method as in claim 32, wherein step (a) includes:

(a1) segmenting the digital image into continuous regions of uniform color and assigning a score indicating probability that the region corresponds to skin for forming a candidate skin region.

34. The method as in claim 33, wherein step (a1) includes merging two or more candidate skin regions based on their similarity of color and degree of connectivity.

35. The method of claim 33, wherein step (a1) includes determining probability that the candidate region is a face based on its shape.

46. The method as claimed in claim 32 wherein resizing in step (b) is based on a predetermined distance between the eyes of a typical face.

47. The method as claimed in claim 46 wherein resizing in step (b) is based on a predetermined ratio between the width of a typical face and the distance between the eyes of a typical face.

48. The method as claimed in claim 32 wherein step (c) further comprises comparing each redeye defect to one or more facial properties of the resized skin colored regions.

49. The method as claimed in claim 48 wherein each redeye defect is compared to an eye size expected for the resized skin colored region.

50. The method as claimed in claim 32 wherein step (c) further comprises detecting pairs of redeye defects by comparing each redeye defect in the pair to one or more facial properties of the resized skin colored regions.

51. The method as claimed in claim 50 wherein the distance between each redeye defect in a pair is compared to a distance expected for the resized skin colored region.

52. The method as claimed in claim 50 wherein each redeye defect in the pair is compared to an eye size expected for the resized skin colored region.

36. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to one or more properties of a face to determine whether a pair of the candidate redeye defects is consistent with the size expected of an eye based on distance between the pair of candidate redeye defects; and

(c) correcting color of pixels based on a location of redeye defect found in step (b).

37. The method as in claim 36, wherein the match of step (b) further includes determining for a pair of candidate redeye defects the tilt expected of an eye based on distance between and the tilt of the pair of candidate redeye defects.

38. The method as in claim 36, wherein the match of step (b) includes correlating a region around the candidate redeye defects with an eye template.

39. The method as in claim 38, wherein the match of step (b) includes correlating the region around the candidate redeye defects with individual zones of the eye template.

40. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) detecting skin colored regions in a digital image;

(b) resizing each skin colored region based on one or more predetermined facial dimensions to form one or more resized skin colored regions;

(c) searching the resized skin colored regions for groups of pixels with color characteristic of redeye defect for forming a candidate redeye defect;

(d) evaluating the groups of pixels relative to one or more properties of a face to determine whether a pair of the candidate redeye defects is consistent with a facial property of an eye pair; and

(e) correcting color of the pixels based on a location of the redeye defect.

41. The method as in claim 40, wherein step (a) includes:

(a1) segmenting the digital image into continuous regions of uniform color and assigning a score indicating probability that the region corresponds to skin for forming a candidate skin region.

42. The method as in claim 40, wherein step (a1) includes merging two or more candidate skin regions based on their similarity of color and degree of connectivity.

43. The method as in claim 41, wherein step (a1) includes determining probability that the candidate region is a face based on its shape.

44. The method as in claim 40, wherein the match of step (d) further includes determining for a pair of candidate redeye defects the size and tilt expected of an eye based on distance between and the tilt of the pair of candidate redeye defects.

45. The method as in claim 44, wherein the match of step (d) includes correlating the region around the candidate redeye defects with individual zones of the eye template.

53. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to the determined size of a face to determine the extent to which the size of a candidate redeye defect matches an expected size of an eye pupil; and

(c) correcting color of the pixels based on the evaluation of redeye defect performed in step (b).

54. The method as in claim 53, wherein the evaluation of step (b) includes correlating a region around the candidate redeye defects with an eye template.

55. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to the size of a face to determine the extent to which the distance between a pair of the candidate redeye defects matches an expected distance between a pair of eyes; and

(c) correcting color of the pixels based on a location of redeye defect found in step (b).

56. The method as in claim 55, wherein the evaluation of step (b) includes correlating a region around the candidate redeye defects with an eye template.

57. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) detecting groups of pixels with color characteristic of redeye defect forming a pair of candidate redeye defects;

(b) evaluating the groups of pixels relative to a distance between the redeye defects and a tilt of the redeye defects to determine the extent to which each candidate redeye defect is part of an eye; and

(c) correcting color of the pixels based on the evaluation of redeye defect found in step (b).

58. The method as in claim 57, wherein the evaluation of step (b) includes correlating a region around the candidate redeye defects with an eye template.

59. A method for detecting eye color defects of a subject in an image due to flash illumination, said method comprising the steps of:

(a) searching of an entire digital image for skin-colored regions having groups of pixels with color characteristic of redeye defect for forming candidate redeye defect, said skin colored regions having a characteristic of a face;

(b) evaluating the groups of pixels relative to one or more properties of a face to determine whether a pair of the candidate redeye defects is consistent with the tilt expected of an eye based on the tilt of the pair of candidate redeye defects; and

(c) correcting color of the pixels based on a location of redeye defect found in step (b).

60. The method as in claim 59, wherein the match of step (b) further includes determining for a pair of candidate redeye defects the size expected of an eye based on distance between the pair of candidate redeye defects.

61. The method as in claim 59, wherein the match of step (b) includes correlating a region around the candidate redeye defects with an eye template.

62. The method as in claim 61, wherein the match of step (b) includes correlating the region around the candidate redeye defects with individual zones of the eye template.