Automatic detection of vertical gaze using an embedded imaging device

• US 8,836,777 B2
• Filed: 02/25/2011
• Issued: 09/16/2014
• Est. Priority Date: 02/25/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: DigitalOptics Corporation Europe Limited (Adeia Inc.)
• Inventors: Mehra, Sumat
• Primary Examiner(s): Rao, Andy S.
• Assistant Examiner(s): Owens, Tsion B
• Application Number:
  US13/035,907
• Publication Number:
  US 20120218398A1
• Time in Patent Office: 1,299 Days
• Field of Search: 348/78, 345/7
• US Class Current: 348/78
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G09C 5/00 Ciphering apparatus or methods not provided for in the preceding groups, e.g. involving the concealment or deformation of graphic data such as designs, written or printed messages

Abstract

A method of detecting and applying a vertical gaze direction of a face within a digital image includes analyzing one or both eyes of a face within an acquired image, including determining a degree of coverage of an eye ball by an eye lid within the digital image. Based on the determined degree of coverage of the eye ball by the eye lid, an approximate direction of vertical eye gaze is determined. A further action is selected based on the determined approximate direction of vertical eye gaze.

Claims

1. A method for detecting a vertical gaze direction of a face within a digital image, the method comprising performing, by a computing device, the steps of:

acquiring a digital image including at least part of a face including one or both eyes;

analyzing at least one of the eyes for which at least a portion of an eye ball is visible within the digital image,

wherein analyzing the at least one of the eyes includes determining whether the eye ball is covered by an eye lid within the digital image by more than a threshold degree;

based, at least in part, on whether the eye ball is covered by the eye lid by more than the threshold degree, determining an approximate direction of vertical eye gaze by the at least one of the eyes; and

initiating a further action based on the determined approximate direction of vertical eye gaze,

wherein the method is performed by one or more computing devices.

2. The method of claim 1, further comprising determining that the at least one of the eyes is looking at a particular display screen of multiple display screens, including concluding that the determined approximate direction of vertical eye gaze lies within a directional range of said particular display screen.

3. The method of claim 1, wherein the initiating a further action comprises controlling an avatar of a gaming application.

4. The method of claim 1, wherein the analyzing at least one of the eyes comprises determining an approximate direction of horizontal gaze, and wherein said initiating a further action or initiating a different action, or both, are based at least in part on the determined approximate direction of horizontal gaze.

5. The method of claim 4, wherein the analyzing at least one of the eyes comprises spectrally analyzing a reflection of light from the at least one of the eyes.

6. The method of claim 4, wherein the analyzing at least one of the eyes comprises analyzing an amount of sclera visible on at least one side of the iris.

7. The method of claim 1, further comprising determining an angular offset of the face from normal, and determining said approximate direction of vertical eye gaze based in part on the angular offset and in part on the degree of coverage of the eye ball by the eye lid.

8. The method of claim 1, wherein when the determined approximate direction of vertical gaze is determined not to lie within a directional range of a digital device, then the further action comprises a power save routine of the digital device.

9. The method of claim 8, wherein when the determined approximate direction of vertical gaze is determined to lie within a directional range of a digital device, then the further action comprises a power on launch routine of the digital device.

10. The method of claim 1 wherein determining whether the eye ball is covered by the eye lid includes determining a degree of coverage of the eye ball by the eye lid.

11. The method of claim 10, further comprising:

determining that the at least one of the eyes is looking at a particular display screen of multiple display screens, wherein the multiple display screens comprise a top screen and a bottom screen;

concluding that the approximate direction lies within the directional range of the bottom screen when determined degree of coverage exceeds the threshold degree; and

concluding that the approximate direction lies within the directional range of the top screen when the determined degree of coverage is less than the threshold degree.

12. The method of claim 1 wherein determining whether the eye ball is covered by the eye lid includes determining a percentage of coverage of an iris of the eye ball by the eye lid.

13. An image acquisition device, comprising:

a lens and image sensor for acquiring digital images;

a processor;

a memory having embedded therein code to program the processor to perform a method of detecting a vertical gaze direction of a face within a digital image, wherein the method comprises;

acquiring a digital image including at least part of a face including one or both eyes;

analyzing at least one of the eyes for which at least a portion of an eye ball is visible within the digital image,

wherein analyzing the at least one of the eyes includes determining whether the eye ball is covered by an eye lid within the digital image by more than a threshold degree;

based, at least in part, on whether the eye ball is covered by the eye lid by more than a threshold degree, determining an approximate direction of vertical eye gaze by the at least one of the eyes; and

initiating a further action based on the determined approximate direction of vertical eye gaze.

14. The device of claim 13, wherein the method further comprises determining that the at least one of the eyes is looking at a particular display screen of multiple display screens, including concluding that the determined approximate direction of vertical eye gaze lies within a directional range of said particular display screen.

15. The device of claim 13, wherein the initiating a further action comprises controlling an avatar of a gaming application.

16. The device of claim 13, wherein the analyzing at least one of the eyes comprises determining an approximate direction of horizontal gaze, and wherein said initiating a further action or initiating a different action, or both, are based at least in part on the determined approximate direction of horizontal gaze.

17. The device of claim 16, wherein the analyzing at least one of the eyes comprises spectrally analyzing a reflection of light from the at least one of the eyes.

18. The device of claim 16, wherein the analyzing at least one of the eyes comprises analyzing an amount of sclera visible on at least one side of the iris.

19. The device of claim 13, wherein the method further comprises determining an angular offset of the face from normal, and determining said approximate direction of vertical eye gaze based in part on the angular offset and in part on the degree of coverage of the eye ball by the eye lid.

20. The device of claim 13, wherein when the determined approximate direction of vertical gaze is determined not to lie within a directional range of a digital device, then the further action comprises a power save routine of the digital device.

21. The device of claim 20, wherein when the determined approximate direction of vertical gaze is determined to lie within a directional range of a digital device, then the further action comprises a power on launch routine of the digital device.

22. The device of claim 13, wherein determining whether the eye ball is covered by the eye lid includes determining a degree of coverage of the eye ball by the eye lid.

23. The device of claim 22, the method further comprising:

determining that the at least one of the eyes is looking at a particular display screen of multiple display screens, wherein the multiple display screens comprise a top screen and a bottom screen; and

concluding that the approximate direction lies within the directional range of the bottom screen when determined degree of coverage exceeds the threshold degree; and

concluding that the approximate direction lies within the directional range of the top screen when the determined degree of coverage is less than the threshold degree.

24. The device of claim 13, wherein determining whether the eye ball is covered by the eye lid includes determining a percentage of coverage of an iris of the eye ball by the eye lid.

25. One or more non-transitory processor-readable media having code embedded therein for programming a processor to perform a method of detecting a vertical gaze direction of a face within a digital image, wherein the method comprises:

acquiring a digital image including at least part of a face including one or both eyes;

analyzing at least one of the eyes for which at least a portion of an eye ball is visible within the digital image,

wherein analyzing the at least one of the eyes includes determining whether the eye ball is covered by an eye lid within the digital image by more than a threshold degree;

based, at least in part, on whether the eye ball is covered by the eye lid by more than the threshold degree, determining an approximate direction of vertical eye gaze by the at least one of the eyes; and

initiating a further action based on the determined approximate direction of vertical eye gaze.

26. The one or more non-transitory processor-readable media of claim 25, wherein the method further comprises determining that the at least one of the eyes is looking at a particular display screen of multiple display screens, including concluding that the determined approximate direction of vertical eye gaze lies within a directional range of said particular display screen.

27. The one or more non-transitory processor-readable media of claim 25, wherein the initiating a further action comprises controlling an avatar of a gaming application.

28. The one or more non-transitory processor-readable media of claim 25, wherein the analyzing at least one of the eyes comprises determining an approximate direction of horizontal gaze, and wherein said initiating a further action or initiating a different action, or both, are based at least in part on the determined approximate direction of horizontal gaze.

29. The one or more non-transitory processor-readable media of claim 28, wherein the analyzing at least one of the eyes comprises spectrally analyzing a reflection of light from the at least one of the eyes.

30. The one or more non-transitory processor-readable media of claim 28, wherein the analyzing at least one of the eyes comprises analyzing an amount of sclera visible on at least one side of the iris.

31. The one or more non-transitory processor-readable media of claim 28, wherein the method further comprises determining an angular offset of the face from normal, and determining said approximate direction of vertical eye gaze based in part on the angular offset and in part on the degree of coverage of the eye ball by the eye lid.

32. The one or more non-transitory processor-readable media of claim 25, wherein when the determined approximate direction of vertical gaze is determined not to lie within a directional range of a digital device, then the further action comprises a power save routine of the digital device.

33. The one or more non-transitory processor-readable media of claim 32, wherein when the determined approximate direction of vertical gaze is determined to lie within a directional range of a digital device, then the further action comprises a power on launch routine of the digital device.

34. The one or more non-transitory processor-readable media of claim 25, wherein determining whether the eye ball is covered by the eye lid includes determining a degree of coverage of the eye ball by the eye lid.

35. The one or more non-transitory processor-readable media of claim 34, the method further comprising:

determining that the at least one of the eyes is looking at a particular display screen of multiple display screens, wherein the multiple display screens comprise a top screen and a bottom screen;

concluding that the approximate direction lies within the directional range of the bottom screen when determined degree of coverage exceeds the threshold degree; and

concluding that the approximate direction lies within the directional range of the top screen when the determined degree of coverage is less than the threshold degree.

36. The one or more non-transitory processor-readable media of claim 25, wherein determining whether the eye ball is covered by the eye lid includes determining a percentage of coverage of an iris of the eye ball by the eye lid.