Detecting irises and pupils in images of humans

• US 20060098867A1
• Filed: 11/10/2004
• Published: 05/11/2006
• Est. Priority Date: 11/10/2004
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Gallagher, Andrew C.
• Granted Patent: US 7,444,017 B2
• Time in Patent Office: Days
• US Class Current: 382/167
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30216 Redeye defect
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

In an eye detection method, a plurality of candidate eyes are located within a digital image. Pixels of interest having a predetermined characteristic and a predetermined association to respective eyes are found. Pixels of interest associated with each eye are grouped. Parametric boundaries are fit on each of the groups to define a plurality of detected eye features. The boundaries have a predefined uniform shape and a size matched to a respective group. Each of the detected eye features is scored as to a geometric relationship between the respective boundary and pixels of interest associated with the respective eye to provide eye feature scores.

Claims

1. An eye detection method for use with a digital image having a plurality of pixels, said method comprising the steps of:

locating a plurality of candidate eyes within the image;

finding pixels of said digital image having a predetermined characteristic and a predetermined association to respective said eyes, to provide pixels of interest;

grouping said pixels of interest associated with each of said eyes into a plurality of different groups;

fitting parametric boundaries on each of said groups to define a plurality of detected eye features, said boundaries having a predefined uniform shape and a size matched to a respective said group;

scoring each of said detected eye features as to a geometric relationship between the respective one of said boundaries and said pixels of interest associated with the respective said eye to provide respective eye feature scores.

2. The method of claim 1 wherein said groups are of uniform size.

3. The method of claim 1 wherein said groups each have a number of pixels minimally sufficient to define said parametric boundaries.

4. The method of claim 3 wherein each of said boundaries is defined by a polynomial.

5. The method of claim 3 wherein said boundaries are circles.

6. The method of claim 5 wherein each of said groups consists of three pixels.

7. The method of claim 1 wherein each of said boundaries is defined by a polynomial.

8. The method of claim 1 wherein said boundaries are circles.

9. The method of claim 1 wherein said finding further comprises identifying a local neighborhood relative to each of said candidate eyes.

10. The method of claim 9 wherein said locating further comprises detecting a plurality of pairs of candidate eyes, and the size of said local neighborhoods is a function of the separation of members of respective said pairs of candidate eyes.

11. The method of claim 10 wherein said candidate eyes each have a redeye defect and said locating further comprises finding respective said redeye defects.

12. The method of claim 9 wherein said finding further comprises producing a map of values of a predetermined property for each of said local neighborhoods;

and identifying positions on each said map having said values of said property in a predefined range.

13. The method of claim 12 wherein said map includes at least one of:

edge values, color values, and color adjacency values.

14. The method of claim 9 wherein said finding further comprises producing an edge map of each of said local neighborhoods;

and identifying positions on each said edge map having an edge magnitude greater than a predefined threshold to define said pixels of interest.

15. The method of claim 9 wherein said scoring of each of said detected eye features further comprises ascertaining separations of said pixels of interest from the respective said boundary.

16. The method of claim 15 wherein said scoring of each of said detected eye features further comprises counting a number of said pixels of interest within a predetermined distance of the respective said boundary.

17. The method of claim 16 wherein said distance is 0.7 pixels.

18. An eye detection method for use with a digital image having a plurality of pixels, said method comprising the steps of:

locating one or more candidate eyes within the image;

identifying one or more local neighborhoods, each said local neighborhood being inclusive of and larger than a respective one of said candidate eyes;

determining a plurality of different groups of pixels of interest within each of said local neighborhoods;

fitting predefined parametric boundaries on each of said groups to define a plurality of detected eye features;

scoring each of said detected eye features as to a geometric relationship between pixels of interest in the local neighborhood and the respective said boundary to provide respective eye feature scores.

19. The method of claim 18 further comprising, in each said local neighborhood, selecting one of said detected eye features having a highest ranked one of said eye feature scores to provide selected eye features.

20. The method of claim 19 wherein said locating further comprises detecting redeye defects and said method further comprises modifying said redeye defects based upon said selected eye features.

21. The method of claim 20 wherein said modifying further comprises revising respective said redeye defects in accordance with respective said selected eye features.

22. The method of claim 18 wherein said scoring of each of said detected eye features further comprises ascertaining the number of said pixels of interest that are within a small distance of the respective said boundary.

23. The method of claim 18 wherein each of said boundaries is defined by a polynomial.

24. The method of claim 23 wherein said groups of pixels of interest each have a number of pixels minimally needed to define the respective said polynomial.

25. The method of claim 18 wherein said boundaries are circles.

26. The method of claim 18 wherein said groups of pixels of interest each have a number of pixels minimally needed to define the respective said parametric boundary.

27. The method of claim 26 wherein said parametric boundaries are circles and each of said groups consists of three pixels.

28. The method of claim 18 wherein said locating further comprises detecting one or more redeye defect pairs.

29. The method of claim 28 wherein the size of said local neighborhoods is a function of the separation of members of respective redeye defect pairs.

30. The method of claim 28 wherein each said local neighborhood has an upper left corner having pixel coordinates, which are the minimum of the x and y coordinates, respectively, of points B1, B2, B3, and B4, and has a lower right corner having pixel coordinates, which are the maximum of the x and y coordinates, respectively, of points B1, B2, B3, and B4;

wherein

point B1=D0+b₀D
point B2=D0+b₁D
point B3=D0+b₂D_perp
point B4=D0−

b₂D_perp

where;

D0 is the (x,y) position of a first defect of a respective said defect pair,

D1 is the (x,y) position of the second defect of the respective said defect pair,

D is a vector from D0 to D1, such that D=D1−

D0,

D_perp is a vector extending through D0 perpendicular to D and having the same length as D, and

b₀, b₁, and b₂ are coefficients.

31. The method of claim 30 wherein b₀=0.35, b₁=−

0.51, and b₂=0.2.

32. The method of claim 30 further comprising determining an age class associated with each of said redeye defect pairs and adjusting one or more of b₀, b₁, and b₂ based upon the associated age class.

33. The method of claim 30 further comprising determining a degree of out of plane rotation associated with each of said redeye defect pairs and adjusting one or more of b₀, b₁, and b₂ based upon the associated degree of out of plane rotation.

34. The method of claim 30 wherein said boundaries are circles, and said fitting further comprises excluding from consideration ones of said groups fit to circles having a radius beyond a range of maximum and minimum radii given by:

35. The method of claim 34 wherein B is 2 pixels, R_n is 0.06, and R_x is 0.14.

36. The method of claim 34 further comprising further comprising determining an age class of adult or child associated with each of said redeye defect pairs and wherein R_n=0.06 and R_x=0.12, when the respective age class is adult, and R_n=0.09 and R_x=0.14, when the respective age class is child.

37. The method of claim 18 wherein said determining further comprises producing an edge map of each of said local neighborhoods.

38. The method of claim 37 wherein said determining further comprises identifying positions on each said edge map having an edge magnitude greater than a predefined threshold to define said pixels of interest.

39. The method of claim 38 wherein said threshold is predefined as a dynamically selected value such that a predetermined percentage of the pixels of the local neighborhood are pixels of interest.

40. The method of claim 39 wherein said predetermined percentage is at least 2 percent of the pixels of the local neighborhood.

41. The method of claim 38 wherein said boundaries are circles.

42. The method of claim 41 wherein said groups each include two pixels and said fitting further comprises ascertaining from said edge map, local edge orientations at said two pixels.

43. The method of claim 42 wherein said scoring of each of said detected eye features further comprises ascertaining the number of said pixels of interest that are within a predetermined distance of the respective said boundary.

44. The method of claim 43 wherein said scoring further comprises excluding from consideration detected eye features having local edge orientations that are beyond π

/24 radians from tangency at one or both of said pixels of said group.

45. The method of claim 41 wherein said groups each include three pixels.

46. The method of claim 41 wherein said determining and fitting steps further comprise applying one or more stages of Hough transform to said pixels of interest within each of said local neighborhoods.

47. The method of claim 18 wherein said fitting further comprises excluding from consideration pixels of said groups at more than a predetermined distance from respective said boundaries.

48. The method of claim 47 wherein said distance is 0.7 pixels.

49. The method of claim 47 wherein said scoring further comprises, following said excluding, classifying non-excluded pixels of said groups into lateral pixels and longitudinal pixels and weighting said lateral pixels and said longitudinal pixels differently.

50. The method of claim 18 wherein:

said locating further comprises detecting a plurality of pairs of candidate eyes;

the size of said local neighborhoods is a function of the separation of members of respective said pairs of candidate eyes;

said boundaries are circles; and

said scoring of each of said detected eye features further comprises counting a number of said pixels of interest within a predetermined distance of the respective said boundary, and assigned a weight to each of said pixels of interest within said predetermined distance of the respective said boundary, each said weight being based upon an angle formed by a line joining the respective pixel of interest and the center of the respective boundary and a line joining the members of the respective pair of candidate eyes.

51. An eye detection method for use with a digital image having a plurality of pixels, said method comprising the steps of:

locating one or more redeye defect pairs in the digital image;

identifying a local neighborhood relative to each member of each of said redeye defect pairs;

ascertaining pixels of interest within each said local neighborhood, said pixels of interest each having a predetermined characteristic;

determining a circle best fitting said pixels of interest in each said local neighborhood.

52. The method of claim 51 further comprising using each said circle to correct the respective said redeye defect pairs.

53. The method of claim 52 wherein each said circle defines a detected iris and said using further comprises removing redeye only within respective said circles.

54. The method of claim 52 wherein each said circle defines a detected pupil said using further comprises removing redeye only within respective said circles.

55. The method of claim 51 wherein said predetermined characteristic is an edge magnitude and edge orientation.

56. The method of claim 51 wherein said determining further comprises producing an edge map of each of said local neighborhoods;

and identifying positions on each said edge map having an edge magnitude greater than a predefined threshold to define said pixels of interest.

57. The method of claim 51 wherein the size of said local neighborhoods is a function of the separation of members of respective said redeye defect pairs.

58. The method of claim 51 wherein said determining further comprises fitting a plurality of circles on different groups of said pixels of interest within each of said local neighborhoods and scoring said circles based upon relative positions of pixels of interest within respective said local neighborhoods.

59. A method of detecting a human iris in a digital image having a plurality of pixels and one or more redeye defects, said method comprising:

locating one or more redeye defect pairs in the digital image;

identifying a local neighborhood relative to each member of each of said redeye defect pairs, said local neighborhoods each having a size that is a function of the separation of members of the respective said redeye defect pair;

producing an edge map of each of said local neighborhoods;

identifying positions on each said edge map having an edge magnitude greater than a predefined threshold to define pixels of interest;

grouping said pixels of interest associated with each of said redeye defects into a plurality of different groups, each group having two pixels;

fitting circles on each of said groups, wherein said edge directions are tangent to the respective said circle at each of said pixels of each said group, to define a plurality of detected eye features;

counting a number of said pixels of interest within a predetermined distance of each of said circles; and

assigning a weight to each of said pixels of interest within said predetermined distance of respective said circles.

60. A method of correcting redeye defects in an image having pixels of a human subject, said method comprising:

locating a pair of redeye defects representing a right eye and a left eye, said defects each having one or more pixels;

identifying a local neighborhood associated with each of said eyes;

searching in each said local neighborhood to detect a candidate eye feature having a circular shape and provide a pair of detected eye features;

using said detected eye features and associated said redeye defects to correct the image.

61. The method of claim 60 wherein each said circle defines a detected iris and said using further comprises removing redeye only within respective said circles.

62. The method of claim 60 wherein each said circle defines a detected pupil said using further comprises removing redeye only within respective said circles.

63. A computer program product for use with a digital image having a plurality of pixels, the computer program product comprising computer readable storage medium having a computer program stored thereon for performing the steps of:

locating a plurality of candidate eyes within the image;

finding pixels of said digital image having a predetermined characteristic and a predetermined association to respective said eyes, to provide pixels of interest;

grouping said pixels of interest associated with each of said eyes into a plurality of different groups;

fitting parametric boundaries on each of said groups to define a plurality of detected eye features, said boundaries having a predefined uniform shape and a size matched to a respective said group;

scoring each of said detected eye features as to a geometric relationship between the respective one of said boundaries and said pixels of interest associated with the respective said eye to provide respective eye feature scores.

64. An eye detection apparatus for use with a digital image having a plurality of pixels, comprising:

means for locating a plurality of candidate eyes within the image;

means for finding pixels of said digital image having a predetermined characteristic and a predetermined association to respective said eyes, to provide pixels of interest;

means for grouping said pixels of interest associated with each of said eyes into a plurality of different groups;

means for fitting parametric boundaries on each of said groups to define a plurality of detected eye features, said boundaries having a predefined uniform shape and a size matched to a respective said group;

means for scoring each of said detected eye features as to a geometric relationship between the respective one of said boundaries and said pixels of interest associated with the respective said eye to provide respective eye feature scores.