ELECTRONIC DEVICES WITH GAZE DETECTION CAPABILITIES

• US 20100079508A1
• Filed: 09/30/2008
• Published: 04/01/2010
• Est. Priority Date: 09/30/2008
• Status: Abandoned Application

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Rosenblatt, Michael, Hodge, Andrew
• Application Number:
  US12/242,251
• Publication Number:
  US 20100079508A1
• Time in Patent Office: Days
• US Class Current: 345/697
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2330/021 Power management, e.g. power saving
• CPC Class Codes: H04N 21/4436 Power management, e.g. shutting down unused components of the receiver power management in computer systems G06F1/3203; hibernate or awake process in computer systems G06F9/4418
• CPC Class Codes: H04W 52/027 by controlling a display operation or backlight unit
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

An electronic device may have gaze detection capabilities that allow the device to detect when a user is looking at the device. The electronic device may implement a power management scheme using the results of gaze detection operations. When the device detects that the user has looked away from the device, the device may dim a display screen and may perform other suitable actions. The device may pause a video playback operation when the device detects that the user has looked away from the device. The device may resume the video playback operation when the device detects that the user is looking towards the device. Gaze detector circuitry may be powered down when sensor data indicates that gazed detection readings will not be reliable or are not needed.

Claims

1. A method for minimizing power consumption in a portable electronic device that has a display and gaze detection circuitry, the method comprising:

performing a video playback operation in which video is displayed on the display;

with the gaze detection circuitry, determining whether a user'"'"'s gaze is directed towards the portable electronic device; and

when it is determined that the user'"'"'s gaze is not directed towards the portable electronic device, pausing the video playback operation.

2. The method defined in claim 1 wherein determining whether the user'"'"'s gaze is directed towards the portable electronic device comprises determining whether the user'"'"'s gaze is directed towards the display.

3. The method defined in claim 1 further comprising:

after it has been determined that the user'"'"'s gaze is not directed towards the portable electronic device, determining whether the user'"'"'s gaze has returned to the portable electronic device; and

when it has been determined that the user'"'"'s gaze has returned to the portable electronic device, resuming the video playback operation.

4. The method defined in claim 1 further comprising:

after it has been determined that the user'"'"'s gaze is not directed towards the portable electronic device, determining whether the user'"'"'s gaze has returned to the portable electronic device; and

when it has been determined that the user'"'"'s gaze has returned to the portable electronic device, presenting an on-screen selectable option to resume the video playback operation.

5. The method defined in claim 1 further comprising:

when it has been determined that the user'"'"'s gaze is not directed towards the portable electronic device, placing the display in an operating mode that reduces power consumption by the display.

6. The method defined in claim 5 wherein the display is at a first brightness level during the video playback operation and wherein placing the display in the operating mode comprises reducing the brightness of the display to a second brightness level.

7. The method defined in claim 6 further comprising:

after it has been determined that the user'"'"'s gaze is not directed towards the portable electronic device, determining whether the user'"'"'s gaze has returned to the portable electronic device; and

when it has been determined that the user'"'"'s gaze has returned to the portable electronic device, increasing the brightness of the display to the first brightness level.

8. The method defined in claim 5 wherein placing the display in the operating mode that reduces power consumption comprises turning off the display.

9. A method for using a portable electronic device having an accelerometer and gaze detection circuitry, the method comprising:

with the accelerometer, determining whether a measured acceleration level for the portable electronic device has exceeded a given threshold value; and

when it has been determined that the acceleration level of the portable electronic device has exceeded the given threshold value, disabling the gaze detection circuitry.

10. The method defined in claim 9 further comprising:

after it has been determined that the acceleration level of the portable electronic device has exceeded the given threshold value, determining whether the acceleration level of the portable electronic device has dropped below a second threshold value; and

when it has been determined that the acceleration level of the portable electronic device has dropped below the second threshold value, enabling the gaze detection circuitry.

11. The method defined in claim 9 wherein the gaze detection circuitry comprises a camera that is used to determine whether the user'"'"'s gaze is directed towards the portable electronic device and wherein disabling the gaze detection circuitry comprises turning off the camera.

12. A portable electronic device comprising:

an image sensor that captures images of a user; and

circuitry that processes the captured images to determine whether the user is looking at the portable electronic device, wherein the circuitry is configured to suspend captured image processing operations when user activity with the portable electronic device is detected.

13. The portable electronic device defined in claim 12 wherein the circuitry comprises at least one button and wherein the user activity comprises presses of the button by the user.

14. The portable electronic device defined in claim 12 wherein the user activity includes movement of the portable electronic device and wherein the circuitry comprises a sensor that detects the movement of the portable electronic device.

15. The portable electronic device defined in claim 12 further comprising:

a proximity sensor that detects whether any objects are within a given distance of the portable electronic device, wherein the circuitry is configured to suspend captured image processing operations when the proximity sensor detects an object within the given distance.

16. The portable electronic device defined in claim 12 further comprising:

a light sensor that measures a brightness level of ambient light, wherein the circuitry is configured to suspend captured image processing operations when the measured brightness level of ambient light is less than a given brightness level.

17. A portable electronic device comprising:

a display;

an image sensor that captures images of a user;

a user input device that receives input from the user; and

circuitry that processes the captured images to determine whether the user is looking at the portable electronic device, wherein the circuitry is configured to;

power the display at a first brightness level when it is determined that the user is looking at the portable electronic device; and

turn off the display after a given period of time has elapsed in which no input has been received from the user by the user input device, wherein the given period of time begins when it is determined that the user has looked away from the portable electronic device.

18. The portable electronic device defined in claim 17 wherein the circuitry is configured to reduce power to the display so that the display is at a second brightness level during the given period of time in which no input has been received from the user by the user input device.

19. The portable electronic device defined in claim 17 wherein the circuitry is configured to once again power the display at the first brightness level when the user input device receives input from the user.

20. The portable electronic device defined in claim 17 wherein the user input device comprises at least one button and wherein the input from the user comprises presses of the button by the user.

21. The portable electronic device defined in claim 17 wherein the circuitry is configured to suspend captured image processing operations after the given period of time has elapsed.

22. The portable electronic device defined in claim 21 wherein the circuitry is configured to resume captured image processing operations when the user input device receives input from the user.

23. A method for using a portable electronic device having a display, a user input device, and gaze detection circuitry, the method comprising:

with the gaze detection circuitry, determining whether a user'"'"'s gaze is directed towards the portable electronic device;

when it is determined that the user'"'"'s gaze is not directed towards the portable electronic device, reducing power to the display; and

when the user input device receives input from a user and power to the display has been reduced, increasing power to the display so that the display is at a first brightness level.

24. The method defined in claim 23 wherein reducing power to the display comprises reducing power to the display so that the display is at a second brightness level that is less than the first brightness level.

25. The method defined in claim 23 wherein reducing power to the display comprises turning off the display.

26. The method defined in claim 23 wherein reducing power to the display comprises reducing power to the display so that the display is at a second brightness level that is less than the first brightness level, the method further comprising:

after a given period of time has elapsed in which no input has been received by the user input device, turning off the display.

27. A portable electronic device comprising:

a display;

an image sensor that captures images of a user; and

circuitry that processes the captured images to determine whether the user is looking at the portable electronic device, wherein the portable electronic device is configured to operate in;

an active mode in which the display is powered at a first brightness level;

a partial standby mode in which the display is powered at a second brightness level that is less than the first brightness level; and

a full standby mode in which the display screen and at least part of the circuitry that processes the captured images are powered down.

28. The portable electronic device defined in claim 27 wherein the portable electronic device is further configured to switch from operating in the active mode to operating in the partial standby mode when the circuitry determines that the user has looked away from the portable electronic device.

29. The portable electronic device defined in claim 27 wherein the portable electronic device is further configured to switch from operating in the partial standby mode to operating in the full standby mode after a given period of time has elapsed in which no input has been received from the user by the user input device.

30. The portable electronic device defined in claim 29 wherein the first mode is an active mode, wherein the second mode is a full standby mode in the display is powered down, and wherein the portable electronic device is further configured to operate in a partial standby mode in which the display is powered at a second brightness level that is less than the first brightness level.

31. The portable electronic device defined in claim 29 wherein the processor includes a given number of processing cores, wherein, in the first mode, the processor is configured to operate using the given number of the processing cores, and wherein, in the second mode, the processor is configured to operate using fewer than the given number of the processing cores.

32. The portable electronic device defined in claim 29 wherein the processor is configured to receive a first clock in the first mode and a second clock in the second mode and wherein the first clock has a frequency that is greater than the second clock.

29-1. A portable electronic device comprising:

a display;

an image sensor that captures images of a user;

a processor; and

circuitry that processes the captured images to determine whether the user is looking at the portable electronic device, wherein the portable electronic device is configured to operate in;

a first mode in which the display is powered at a first brightness level; and

a second mode in which power to the display and the processor is reduced.