Dynamic device adjustments based on determined user sleep state

• US 9,094,539 B1
• Filed: 09/22/2011
• Issued: 07/28/2015
• Est. Priority Date: 09/22/2011
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Noble, Isaac S.
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Rahman, Mohammad J
• Application Number:
  US13/240,709
• Time in Patent Office: 1,405 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04N 21/00 Selective content distribution, e.g. interactive television or video on demand [VOD] real-time bi-directional transmission of motion video data H04N7/14 ; broadcast or conference over packet switching networks H04L12/18
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 7/00 Television systems details H04N3/00, H04N5/00; methods or arrangements, for coding, decoding, compressing or decompressing digital video signals H04N19/00; selective content distribution H04N21/00
• CPC Class Codes: Y02D 30/70 in wireless communication networks

Abstract

An electronic device performs functionality adjustments based on a determined attention level of a user of the device. In some embodiments, the device automatically enters a sleep mode upon determining that the user has fallen asleep. The device may automatically modify functionality of the device such that disturbances to the user is minimized or power consumption is reduced. For example, the device may lower the volume of a ring tone or deactivate a communication channel. In some instances, the device may determine that the user is half-awake or paying an intermediate level of attention and activate an intermediate mode where at least some functionality is modified or adjusted. A device subsequently determining that the user is fully awake or actively engaged with the device causes the device to return to a standard mode of operation, where functionality of the device is adjusted to one appropriate for the user'"'"'s attention level.

Claims

1. A method of enabling a mobile electronic device to determine when to modify functionality of the mobile electronic device, the method comprising:

activating a standard mode of operation of the mobile electronic device;

capturing first image information using at least one camera of the mobile electronic device, the first image information including image information of at least a portion of a user of the mobile electronic device;

determining whether the user is asleep by, at least in part, analyzing the first image information;

determining a brightness level of at least an area near the user;

causing, using a processor of the electronic device, the mobile electronic device to automatically enter a sleep mode when the user is determined to be asleep and when the brightness level of the at least an area near the user is determined to be at or below a brightness threshold, the mobile electronic device having less active functionality in the sleep mode than when the electronic device is in the standard mode;

capturing second image information using the at least one imaging element of the electronic device when the mobile electronic device is in the sleep mode;

determining that the user is awake by, at least in part, analyzing the second image information;

causing, using the processor of the mobile electronic device, the mobile electronic device to enter an intermediate mode when the user is determined to be awake, the mobile electronic device having more active functionality in the intermediate mode relative to when the mobile electronic device is in the sleep mode and less active functionality in the intermediate mode relative to when the mobile electronic device is in the standard mode; and

causing, using the processor of the mobile electronic device, the mobile electronic device to return to the standard mode of operation upon detecting a trigger from the user.

2. The method of claim 1, further comprising:

illuminating the user with infrared (IR) light, wherein determining whether the user is asleep comprises analyzing the first image information to determine a reflection by at least one retina of the user.

3. The method of claim 1, further comprising:

illuminating the user with infrared (IR) light; and

detecting a face from the first image information, wherein determining whether the user is asleep comprises;

analyzing the detected face to determine whether the detected face is of the user; and

determining, from the detected face, whether eyes of the user are closed for a period of time when the detected face is determined to be of the user.

4. The method of claim 1, wherein determining whether the user is asleep comprises determining whether the first image information indicates at least one of a breathing rate decrease, the user looking at the mobile device, a body temperature decrease, or a breathing pattern change.

5. The method of claim 1, further comprising:

illuminating the user with infrared (IR) light, wherein determining whether the user is asleep comprises analyzing the first image information to determine whether an amount of user movement is below a certain threshold.

6. The method of claim 1, wherein the user is determined to be awake when the second image information indicates that the user is looking directly at the mobile electronic device.

7. A method of enabling a mobile electronic device to adjust one or more functional aspects of the mobile electronic device, the method comprising:

operating, while a user is determined to be asleep, in a sleep mode using a processor of the mobile electronic device;

capturing, while in the sleep mode, image information using at least one imaging element of the mobile electronic device, the image information including a brightness level of a surrounding area of the mobile electronic device;

determining whether the user is awake by, at least in part, analyzing the image information; and

causing, by the processor of the mobile electronic device, the mobile electronic device to operate in an intermediate mode, the mobile electronic device having more active functionality in the intermediate mode relative to when the mobile electronic device is in the sleep mode and less active functionality in the intermediate mode relative to when the mobile electronic device is in a standard mode.

8. The method of claim 7, wherein the at least one imaging element of the mobile electronic device includes at least one of a camera, an infrared (IR) sensor, and a thermal imaging sensor.

9. The method of claim 7, wherein analyzing the captured image information includes:

detecting a face from the captured image information;

performing facial recognition of the user using the detected face; and

upon recognizing the face, determining whether the user is looking at the mobile electronic device.

10. The method of claim 7, wherein the image information includes infrared (IR) light that was emitted by an infrared source of the mobile electronic device and reflected by at least one retina of the user.

11. The method of claim 7, wherein the intermediate mode includes at least one of a current time and calendar information is displayed to the user.

12. The method of claim 7 further comprising:

detecting the brightness level of a the surrounding area of the mobile electronic device using an ambient light sensor, wherein the one or more functional aspects adjusted includes at least one of the brightness level of a display element of the mobile electronic device and the brightness level of one or more LEDs on the mobile electronic device.

13. The method of claim 7, wherein the one or more functional aspects adjusted includes at least one of a ring tone of the mobile electronic device from a loud ring to a low-volume ring and a vibration mode of the mobile electronic device.

14. The method of claim 7, wherein a set of filters are activated upon entering the intermediate mode such that the mobile electronic device notifies the user only upon receiving phone calls or emails from a pre-selected set of contacts.

15. The method of claim 7, wherein a set of filters are activated upon entering the intermediate mode such that the mobile electronic device notifies the user only upon alerts and notifications from a pre-selected set of applications.

16. The method of claim 7 further comprising:

detecting a particular gesture of the user; and

upon detecting the particular gesture, modifying the mobile electronic device from operating in the intermediate mode to operating in a third mode that enables the mobile electronic device to activate a standard mode of operation.

17. The method of claim 7, wherein the functional aspects adjusted as the mobile electronic device is modified from operating in one mode to another mode is configurable by the user of the device.

18. The method of claim 7, wherein analyzing the captured image information includes determining whether the user is looking at the mobile electronic device.

27. The method of claim 7, wherein the mobile electronic device enters the intermediate mode automatically without user intervention when the mobile electronic device determines that the user is awake.

19. A method of enabling a mobile electronic device to switch between different operational modes that correspond to an attention level of a user of the mobile electronic device, the method comprising:

activating a standard mode that adjusts a set of functionalities of the mobile electronic device;

capturing first image information of a user using at least one imaging element on the mobile electronic device;

determining, based at least in part on the first image information, that the user is asleep;

determining a brightness level of at least an area near the user;

causing the mobile electronic device to enter a sleep mode when the user is determined to be asleep and when the brightness level of the at least an area near the user is determined to be at or below a brightness threshold, the mobile electronic device having less active functionality in the sleep mode relative to when the mobile electronic device is in the standard mode;

capturing, while in the sleep mode, second image information;

determining, based at least in part on the second image information, that the user is awake; and

causing the mobile electronic device to enter an intermediate mode, the mobile electronic device having more active functionality in the intermediate mode relative to when the mobile electronic device is in the sleep mode and less active functionality in the intermediate mode relative to when the mobile electronic device is in the standard mode.

20. The method of claim 19, wherein the sleep mode that is activated upon a selection of a user-selectable user interface (UI) item on a display element of the mobile electronic device.

21. The method of claim 19, wherein the intermediate mode is an information mode that displays information including at least a current time, a notification of a number of new emails, a notification of a number of text messages, a current news, and a calendar information.

22. The method of claim 19 further comprising:

detecting at least one of an environmental attribute and a user attribute;

analyzing, using a processor of the mobile electronic device, the at least one of the environmental attribute and the user attribute and a current time to determine a probability that the user of the mobile electronic device is asleep; and

determining whether the probability exceeds a predetermined level,

wherein the sleep mode is activated when the probability exceeds the predetermined level.

23. The method of claim 19, wherein the set of functionalities in the sleep mode are adjusted to notification alerts to notifications from a specified set of contacts.

24. A non-transitory computer-readable storage medium storing a computing program for enabling mobile electronic device to adjust one or more functional aspects of the mobile electronic device, the computer program comprising sets of instructions that, when executed by a processor, cause the processor to:

operate, while a user is determined to be asleep, in a sleep mode of the mobile electronic device;

capture, while in the sleep mode, image information using at least one imaging element of the mobile electronic device, the image information including a brightness level of a surrounding area of the mobile electronic device;

determine whether the user is awake by, at least in part, analyzing the image information; and

modify the mobile electronic device from operating in the sleep mode to operating in an intermediate mode, the mobile electronic device having more active functionality in the intermediate mode relative to when the mobile electronic device is in the sleep mode and less active functionality in the intermediate mode relative to when the mobile electronic device is in a standard mode.

25. The non-transitory computer-readable storage medium of claim 24, wherein the sets of instructions when executed further cause the processor to:

determine an appropriate brightness level of the mobile electronic device using pupil detection and ambient light measurements; and

displaying information on the mobile electronic device at the determined brightness level when the mobile electronic device is modified to operate in the intermediate mode.

26. The non-transitory computer-readable storage medium of claim 24, wherein the sets of instructions when executed further cause the processor to:

detect a face from the captured image information;

perform facial recognition for the user using the detected face; and

upon recognizing the face, determine whether the user is looking at the mobile electronic device.

28. The non-transitory computer-readable storage medium of claim 24, wherein the mobile electronic device enters the intermediate mode automatically without user intervention when the mobile electronic device determines that the user is awake.