Method for automatic detection of human eyes in digital images

• US 6,151,403 A
• Filed: 08/29/1997
• Issued: 11/21/2000
• Est. Priority Date: 08/29/1997
• Status: Expired due to Term

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Luo, Jiebo
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): CHAWAN, SHEELA C
• Application Number:
  US08/920,844
• Time in Patent Office: 1,180 Days
• Field of Search: 382/115, 382/118, 382/117, 382/116, 382/308, 382/257, 382/274, 382/209, 382/219
• US Class Current: 382/117
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A computer program product for locating first and second objects, each having substantially the same physical characteristics, and the ratio of the distance between the first and second objects and the size of each object is substantially invariant, the computer program product comprises: a computer readable storage medium having a computer program stored thereon for performing the steps of determining potential flesh regions in an image; determining valley regions in an image; performing template matching for determining a plurality of locations that give a desirable match of the object relative to the template; and performing verification for determining the likelihood of pairs of potential eye candidates at the locations determined in the performing template matching step.

Claims

1. A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) determining a potential flesh region in an intensity image;

(b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image;

(c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and

(d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.

2. The computer program product as in claim 1 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.

3. The computer program product as in claim 1 further comprising forming a mask image for searching for the first and second human eye objects;

the locations where a search is to be performed are determined by the valley regions within the determined flesh region.

4. The computer program product as in claim 1, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.

5. The computer program product as in claim 1, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.

6. The computer program product as in claim 1, wherein computing the symmetry includes computing the symmetry between first and second halves of a face window determined by the eye locations.

7. The computer program product as in claim 1, wherein computing the centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.

8. The computer program product as in claim 1 further comprising the step of normalizing the template for maximizing robustness of matching.

9. The computer program product as in claim 1 further comprising the step of creating an image block of a same size and orientation as the template.

10. The computer program product as in claim 1 further comprising the step of normalizing the image block for maximizing robustness of matching.

11. The computer program product as in claim 9 further comprising the step of extracting individual sub-zones in both the template and the image block, and performing cross-correlation for each corresponding sub-zone.

12. The computer product as in claim 11 further comprising the step of computing an overall cross-correlation score for the image block if the cross-correlation score for each sub-zone exceeds a threshold.

13. A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) determining a potential flesh region in an intensity image;

(b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image;

(c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and

(d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for symmetry includes computing the symmetry between first and second halves of a face window determined by the locations of the human eye objects.

14. The computer program product as in claim 13 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.

15. The computer program product as in claim 13 further comprising forming a mask image for searching for the first and second human eye objects;

the locations where a search is to be performed are determined by the valley regions within the determined flesh region.

16. The computer program product as in claim 13, wherein computing the orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.

17. The computer program product as in claim 13, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.

18. The computer program product as in claim 13, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.

19. The computer program product as in claim 13, wherein computing the centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.

20. The computer program product as in claim 13 further comprising the step of normalizing the template for maximizing robustness of matching.

21. The computer program product as in claim 13 further comprising the step of creating an image block of a same size and orientation as the template.

22. The computer program product as in claim 13 further comprising the step of normalizing the image block for maximizing robustness of matching.

23. The computer program product as in claim 21 further comprising the step of extracting individual sub-zones in both the template and the image block, and performing cross-correlation for each corresponding sub-zone.

24. The computer program product as in claim 23 further comprising the step of computing an overall cross-correlation score for the image block if the cross-correlation score for each sub-zone exceeds a threshold.

25. A computer program product for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, the computer program product comprising:

a computer readable storage medium having a computer program stored thereon for performing the steps of;

(a) determining a potential flesh region in an intensity image;

(b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image;

(c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and

(d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.

26. The computer program product as in claim 25 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.

27. The computer program product as in claim 25 further comprising forming a mask image for searching for the first and second human eye objects;

the locations where a search is to be performed are determined by the valley regions within the determined flesh region.

28. The computer program product as in claim 25, wherein computing the orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.

29. The computer program product as in claim 25, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.

30. The computer program product as in claim 25, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.

31. The computer program product as in claim 25, wherein computing the symmetry includes computing the symmetry between first and second halves of a face window determined by the eye locations.

32. The computer program product as in claim 25 further comprising the step of normalizing the template for maximizing robustness of matching.

33. The computer program product as in claim 25 further comprising the step of creating an image block of a same size and orientation as the template.

34. The computer program product as in claim 25 further comprising the step of normalizing the image block for maximizing robustness of matching.

35. The computer program product as in claim 33 further comprising the step of extracting individual sub-zones in both the template and the image block, and performing cross-correlation for each corresponding sub-zone.

36. The computer program product as in claim 35 further comprising the step of computing an overall cross-correlation score for the image block if the cross-correlation score for each sub-zone exceeds a threshold.

37. A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of:

(a) determining a potential flesh region in an intensity image;

(b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image;

(c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and

(d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit for orientation includes measuring the difference between an orientation of a line connecting the first and second human eye objects, and an average orientation of the first and second human eye objects.

38. The method as in claim 37 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on a shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.

39. The method as in claim 37 further comprising forming a mask image for searching for the first and second human eye objects;

the locations where a search is to be performed are determined by the valley regions within the determined flesh region.

40. The method as in claim 37, wherein computing the proportion includes computing a distance between the first and second human eye objects for satisfying an anthropological prior model.

41. The method as in claim 37, wherein computing the profile includes pre-determining a model of an eye-to-eye profile and determining an actual profile from the image and computing a goodness-of-fit between the actual profile and the model profile.

42. The method as in claim 37, wherein computing the symmetry includes computing the symmetry between first and second halves of a face window determined by the eye locations.

43. The method as in claim 37, wherein computing the centrality includes computing a distance from a middle point between the first and second human eye objects to a major axis of the determined flesh region.

44. A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ratio of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of:

(a) determining a potential flesh region in an intensity image;

(b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image;

(c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and

(d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit symmetry includes computing the symmetry between first and second halves of a face window determined by the locations of the human eye objects.

45. The method as in claim 44 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.

46. The method in claim 44 further comprising forming a mask image for searching for the first and second human eye objects;

the locations where a search is to be performed are determined by the valley regions within the determined flesh region.

47. A method for locating first and second human eye objects each having substantially the same physical characteristics, and the ration of the distance between the first and second human eye objects and the size of each human eye object is substantially invariant, said method comprising the steps of:

(a) determining a potential flesh regions in an intensity image;

(b) determining valley regions in the intensity image for determining substantially non-flat regions with lower intensity values in a local area of the intensity image;

(c) performing matching in the intensity image using an intensity-based template in a neighborhood of the valley regions within the flesh regions for determining a plurality of locations that give a desirable match of the human eye object relative to the template, said step of matching using cross-correlation to identify desirable locations; and

(d) performing verification by mating a pair of potential human eye object candidates with desirable matching response to the template by using a plurality of verification criteria selected from the group including the orientation, proportion, profile, symmetry, and centrality of the paired human eye objects matched to the intensity-based template, wherein the verification criteria comprise finding the best pair of locations of human eye objects by computing figures of merit individually or in combination for the plurality of verification criteria, and wherein the figure of merit centrality includes computing a distance from a middle point between the first and second human objects to a major axis of the determined flesh region.

48. The method as in claim 47 further comprising either or both (e) determining an estimate size of both the first and second human eye objects based on shape and size of a determined flesh region or (f) determining an estimate orientation of both the first and second human eye objects based on the shape and orientation of the determined flesh region.

49. The method as in claim 47 further comprising forming a mask image for searching for the first and second human eye objects;

the locations where a search is to be performed are determined by the valley regions within the determined flesh region.