Using eye detection for providing control and power management of electronic devices

• US 20050199783A1
• Filed: 03/15/2004
• Published: 09/15/2005
• Est. Priority Date: 03/15/2004
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Inventors: Wenstrand, John Stewart, Sachs, Todd Stephen
• Granted Patent: US 7,091,471 B2
• Time in Patent Office: Days
• US Class Current: 250/214.1
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Eye detection is used as the basis for providing power management or other variations of operational parameters of a controlled device. The system includes an imager and eye detection processing which cooperate to detect light received from a person'"'"'s eye. The eye detection processing generates an output that can be used for determinations regarding presence of one or more persons or presence of a specific person. This output is used as the basis for switching the controlled device between at least two power consumption states. Eye detection may also be used to allow a person to actively adjust operational conditions of the controlled device, since unnatural eye blinking patterns may be associated with particular commands.

Claims

1. A control system for a device comprising:

an imager and eye detection processing for detecting light received by said imager from a human eye, said eye detection processing being configured to generate an output indicative of determinations of detecting said human eye; and

a controller enabled to switch said device among a plurality of power consumption states in response to said output of said eye detection processing, said controller being configured to execute at least one of a first conditional response and a second conditional response, wherein

(a) said first conditional response is one in which said device is switched from a power-up state to a lower power consumption state upon passage of a first time period without detecting a human eye; and

(b) said second conditional response is one in which said device is switched to said power-up state upon detection of a human eye.

2. The control system of claim 1 wherein said controller is configured to execute both of said first and second conditional responses.

3. The control system of claim 1 wherein said device has at least three said power consumption states, including a power-off state, a sleep state and said power-up state, wherein said controller is set to switch said device from said power-up state to said sleep state upon passage of said first time period without detecting a human eye.

4. The control system of claim 3 wherein said controller is further set to switch said device from said sleep state to said power-off state upon passage of a second time period without detecting a human eye.

5. The control system of claim 4 wherein said controller is configured to switch said device from said sleep state to said power-up state upon detection of a human eye.

6. The control system of claim 3 wherein said controller is further configured to switch said device from said power-off state to said power-up state upon detection of a human eye.

7. The control system of claim 1 wherein said controller enables adjustment of said first time period by a user.

8. The control system of claim 1 wherein said controller includes a control algorithm which adaptively adjusts said first time period on a basis of factors that include historical data of occurrences of switching by said controller.

9. The control system of claim 1 further comprising first and second light sources for illuminating an area being monitored for detecting said human eye, said first light source being positioned to emit light at a first illumination angle relative to an axis of said imager, said second light source being positioned to emit light at a second illumination angle relative to said axis, said second illumination angle being substantially greater than said first illumination angle.

10. The control system of claim 9 wherein said imager is a two-dimensional array of pixels controlled to generate frames of image information, said eye detection processing including programming for generating a differential image frame representative of differences in two said frames of image information, said differential image frame being a basis for identifying said human eye in at least one said frame.

11. The control system of claim 1 wherein said controller is connected to an electronic device that includes a display screen, said electronic device being said device having said plurality of power consumption states, said imager being in a fixed location relative to said device being controlled.

12. The control system of claim 11 wherein said electronic device is one of a computer system and a video display system.

13. The control system of claim 1 wherein said controller is further configured to switch said device among said power consumption states in response to detecting predetermined eye blinking patterns, said patterns being interpreted by said controller as commands.

14. The control system of claim 13 wherein said controller is connected to provide switching for illumination of a specific area.

15. The control system of claim 13 wherein said predetermined eye blinking patterns include patterns interpreted by said controller to trigger switching between an “

on”

state and an “

off”

state.

16. The control system of claim 1 further comprising storage of correlations between specific commands and predetermined eye blinking patterns, said commands including device-adjustment commands unrelated to said switching among said power consumption states, said eye detection processing being enabled to recognize said patterns and said controller being configured to implement said commands upon recognitions of said patterns.

17. The control system of claim 16 wherein said eye detection processing is programmed such that an eye closure is detected using at least one frame of image information and such that detection of said eye closure triggers a change from eye recognition via analysis of frames to blink pattern detection via analysis of streaming image data.

18. The control system of claim 1 further comprising at least one stored correlation between a particular person and image information that is specific to an eye of said particular person, wherein operations of said eye detection processing are dependent upon recognizing a person.

20. The control system of claim 1 further comprising first and second light sources directed to illuminate a field of view of said imager, said eye detection processing being configured to identify a pupil of said human eye on a basis of differences between reflected light of said first light source and reflected light of said second light source.

19. The control system of clam 1 wherein said imager is sampled in a non-continuous basis, with a sampling frequency selected to conserve battery charge.

21. A control system for operating a device comprising:

a controller enabled to vary operational parameters of said device in response to detections that are specific to eyes; and

an eye detector connected to said controller to output signals indicative of said detections specific to eyes, said eye detector including;

(a) an imager for receiving reflected light from said eyes;

(b) a first light source positioned to emit first light at a first illumination angle relative to said imager; and

(c) a second light source positioned to emit second light at a second illumination angle relative to said imager, said second illumination angle being substantially greater than said first illumination angle, wherein said eyes are detectable using the difference between reflected said first light and reflected said second light received at said imager.

22. The control system of claim 21 wherein said imager is positioned relative to said device such that the field of view of said imager captures persons having a line of sight to said device, said imager being in a fixed location relative to said device.

23. The control system of claim 22 wherein said device is a television and said operational parameters that are varied by said controller include at least one of a channel setting and a volume setting.

24. The control system of claim 21 wherein said first illumination angle ranges from zero to three degrees and said second illumination angle ranges from three to fifteen degrees.

25. The control system of claim 21 wherein said eye detector includes a processor configured to identify a pupil on a basis of said difference between said reflected first light and said reflected second light.

26. The control system of claim 21 wherein said eye detector further includes a storage of correlations between preselected blink patterns and operational commands, said processor and controller being programmed to detect occurrences of each of said blink patterns and to generate the correlated operational command upon each said occurrence, said operational commands being specific to varying said operational parameters of said device.

27. The control system of claim 21 wherein said imager and said first and second light sources are controlled to generate a succession of image frames in which first image frames are acquired while said first light source is activated and said second image frames are acquired while said second light source is activated.

28. The control system of claim 21 wherein said eye detector further includes a storage of at least one correlation between a particular person and image information that is specific to an eye of said particular person, wherein said controller is responsive to identifications of each said particular person in enabling variations in said operational parameters of said device.

29. A control system for a device comprising:

a controller enabled to vary operational parameters of said device in response to detections that are specific to eyes; and

an eye detector connected to said controller to output signals indicative of said detections specific to eyes, said eye detector including;

(a) an imager for receiving light from said eyes; and

(b) eye detection processing configured to form first and second frames from image information acquired from said imager and to generate a differential image frame representative of differences in said first and second frames, said differential image frame being indicative of a presence or absence of an eye.

30. The control system of claim 29 wherein said eye detection processing includes data sampling circuitry for generating said first and second frames at different sampling times of accessing data from said imager.

31. The control system of claim 29 wherein said eye detection processing includes data sampling and processing circuitry in which said first and second frames are formed by separating pixel image data acquired in a single sampling of said imager, said imager being a two-dimensional array of pixels.

32. The control system of claim 31 wherein pixel image data used in forming said first frame is data acquired from those said pixels of said imager associated with first reflectivity characteristics with respect to imaging an eye, said second frame being formed from pixel image data acquired from those said pixels of said imager associated with second reflectivity characteristics different from said first reflectivity characteristics.

33. The control system of claim 29 wherein said eye detector further includes first and second light sources that are sequenced such that said first frame is acquired during activation of said first light source and said second frame is acquired during activation of said second light source.

34. The control system of claim 29 wherein said eye detector further includes first and second light sources, said first light source having a first illumination angle relative to an axis of said imager and said second light source having a second illumination angle relative to said axis, said second illumination angle being substantially greater than said first illumination angle.

35. The control system of claim 29 wherein said eye detector further includes first and second light sources, said first light source emitting light at a different wavelength from light emitted by said second light source.

36. The control system of claim 29 wherein said eye detector further includes first and second light sources, said first light source emitting light at a different polarization from light emitted by said second light source.

37. The control system of claim 29 wherein said eye detector further includes first and second light sources, wherein light from at least one of said first and second light sources is modulated to enable said light emitted by said first light source to be distinguished from light emitted by said second light source after reflection to said imager.

38. The control system of claim 29 wherein said controller and eye detector are integrated into said device being controlled.

39. The control system of claim 38 wherein said device being controlled includes a display screen.

40. The control system of claim 39 wherein said device being controlled is a television, said operational parameters including at least one of a volume setting, a channel setting, and a selection of a power consumption state.

41. The control system of claim 29 further including a storage of correlations between blinking patterns and commands for varying said operational parameters, said eye detector generating one of said commands in response to recognizing one of said blinking patterns.

42. The control system of claim 41 wherein said eye detector is programmed to utilize frame analysis to detect a potential start of one of said blinking patterns by a person and to switch to data stream analysis as a response to detecting said potential start.

43. The control system of claim 29 wherein said eye detector further includes a storage of at least one correlation between a particular person and image information that is specific to an eye of said particular person, wherein said controller is responsive to identifications of each said particular person in enabling variations in said operational parameters of said device.

44. A method of controlling a device comprising:

detecting occurrences of receiving light reflected from an eye;

switching said device among a plurality of power consumption states on a basis of at least one of a first conditional response and a second conditional response, wherein

(a) said first conditional response is one in which said device is switched from a power-up state to a lower power consumption state upon passage of a first time period without detecting an eye; and

(b) said second conditional response is one in which said device is switched to said power-up state upon detection of an eye.

45. The method of claim 44 wherein both said first and second conditional responses are implemented.

46. The method of claim 44 wherein said device has at least three said power consumption states, including a power-off state, a sleep state and said power-up state, said lower power consumption state of said first conditional response being said sleep state.

47. The method of claim 46 further comprising switching said device from said sleep state to said power-off state as a response to detecting passage of a second time period without detecting an eye.

48. The method of claim 47 wherein said second conditional response is one in which said device is switched from said sleep state to said power-up state upon detection of an eye.

49. The method of claim 44 further comprising:

recognizing predetermined eye blinking patterns by imaging at least one eye;

correlating each said pattern to a command for varying an operational parameter of said device; and

varying said operational parameter of said device based upon said recognizing.

50. The method of claim 49 wherein said predetermined eye blinking patterns are correlated to commands to vary at least one of a current said power consumption state, a volume setting and a channel setting.

51. The method of claim 44 further comprising:

recognizing a correlation between a particular person and a detected eye;

limiting variations to said device on a basis of recognizing said correlation.

52. A method of controlling a device comprising:

locating a detector to have a field of view which is relevant to operation of said device;

illuminating said field of view by activating a first light source on one of a continuous or timed basis, said first light source being positioned to provide a first illumination angle with respect to an axis of said detector;

illuminating said field of view by activating a second light source on one of a continuous or timed basis, said second light source being positioned to provide a second illumination angle with respect to said axis, said second illumination angle being substantially greater than said first illumination angle;

detecting eyes within said field of view; and

varying operational parameters of said device in response to said detecting.

53. The method of claim 52 wherein said device is a video display system and wherein varying said operational parameters includes varying settings for at least one of a channel setting, a volume setting, and a current power consumption state.

54. The method of claim 52 wherein said illuminating establishes said first illumination angle within the range of zero to three degrees and establishes said second illumination angle in the range of three to fifteen degrees.

55. The method of claim 52 further comprising:

recognizing predetermined eye blinking patterns by imaging at least one eye;

correlating each said pattern to a command for varying an operational parameter of said device; and

varying said operational parameter of said device based upon said recognizing.