Waking a device in response to user gestures

• US 9,933,833 B2
• Filed: 07/10/2015
• Issued: 04/03/2018
• Est. Priority Date: 07/18/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Tu, Xiaoyuan, Kandangath, Anil K.
• Primary Examiner(s): Ma, Calvin C
• Application Number:
  US14/796,718
• Publication Number:
  US 20160018900A1
• Time in Patent Office: 998 Days
• Field of Search: 345156, 345173, 345174
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3218 of display devices
• CPC Class Codes: G06F 1/3234 Power saving characterised by the action undertaken
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A wearable computing device can detect device-raising gestures. For example, onboard motion sensors of the device can detect movement of the device in real time and infer information about the spatial orientation of the device. Based on analysis of signals from the motion sensors, the device can detect a raise gesture, which can be a motion pattern consistent with the user moving the device'"'"'s display into his line of sight. In response to detecting a raise gesture, the device can activate its display and/or other components. Detection of a raise gesture can occur in stages, and activation of different components can occur at different stages.

Claims

1. A method executed in a wearable device, the method comprising:

establishing a null state wherein an applications processor of the device is in a sleep state and a user interface component of the device is in an inactive state;

executing, in a coprocessor of the device, a plurality of gesture detection algorithms including one or more raise-gesture detection algorithms to detect a completed raise gesture and a preheat detection algorithm to detect an initiation of the raise gesture;

wherein executing the raise-gesture detection algorithm includes;

selecting criteria for identifying a raise gesture based on a starting pose determined from a set of motion sensor data samples; and

detecting the completed raise gesture based at least in part on whether the selected criteria are satisfied;

generating, by the preheat detection algorithm, a preheat event in the event that the initiation of the raise gesture is detected;

generating, by at least one of the raise-gesture detection algorithms, a raise-gesture event in the event that the completed raise gesture is detected;

transitioning the applications processor from the sleep state to a wake state in response to the preheat event; and

transitioning the user interface component from the inactive state to an active state in response to the raise-gesture event.

2. The method of claim 1 further comprising, in the event that the raise-gesture detection algorithm generates the raise-gesture event before the preheat detection algorithm generates the preheat event, transitioning the applications processor from the sleep state to a wake state in response to the raise-gesture event.

3. The method of claim 1 further comprising:

disabling the preheat detection algorithm when the applications processor is in the wake state; and

reenabling the preheat detection algorithm in the event that the applications processor transitions from the wake state to the sleep state.

4. The method of claim 1 further comprising:

disabling all of the one or more raise-gesture detection algorithms when the user interface component is in the active state; and

reenabling at least one of the raise-gesture detection algorithm in the event that the user interface component transitions from the active state to the inactive state.

5. The method of claim 1 wherein the one or more raise-gesture detection algorithms include:

one or more natural raise-gesture detection algorithms; and

a deliberate raise-gesture detection algorithm.

6. The method of claim 5 wherein the one or more natural raise-gesture detection algorithms include:

a first natural raise-gesture detection algorithm to detect a raise gesture in a low-dynamic environment; and

a second natural raise-gesture detection algorithm to detect a raise gesture in a high-dynamic environment.

7. The method of claim 6 further comprising:

selecting one of the first or second natural raise-gesture detection algorithms to be executed, the selection being based at least in part on a current user activity.

8. The method of claim 1 further comprising:

in response to the raise-gesture event, executing a loss-of-focus detection algorithm to detect a loss-of-focus event; and

in response to detecting the loss-of-focus event, transitioning the user interface component from the active state to the inactive state.

9. The method of claim 1 further comprising, after detecting the raise-gesture event:

determining a dwell time of the device in a focus pose; and

modifying a presentation of information at the user interface component based at least in part on the dwell time.

10. The method of claim 9 wherein modifying the presentation of information includes one or more of:

changing a presented information item from a first information item to a second information item;

presenting additional information about an initially presented information item;

or

ceasing to present an initially presented information item.

11. The method of claim 1 wherein the one or more raise-gesture detection algorithms includes an algorithm to:

determine, based on a set of motion-sensor data samples corresponding to a time interval, a starting pose corresponding to a beginning of the time interval;

determine, based at least in part on the starting pose, a set of criteria for identifying a possible raise gesture using the set of motion-sensor data samples;

determine a possible raise gesture based at least in part on whether the set of motion-sensor data samples satisfies the selected set of criteria; and

determine a completed raise gesture in the event that the possible raise gesture is followed by at least a minimum period of dwell time in a focus pose.

12. The method of claim 1 wherein the one or more raise-gesture detection algorithms includes an algorithm to:

collect a set of motion-sensor data samples from a motion sensor of the device, the set of motion-sensor data samples corresponding to a time interval;

detect a first impulse peak and a second impulse peak in the motion-sensor data samples, the first and second impulse peaks each indicating movement in a direction corresponding to a rotation of a wrist of a user wearing the device, the first and second impulse peaks being separated in time by a period less than a timeout period; and

detect a completed raise gesture based at least in part on detecting the first and second impulse peaks.

13. The method of claim 1 wherein the preheat detection algorithm includes an algorithm to:

operate a motion sensor of the device to collect data samples at a sampling rate;

detect a possible initiation of a raise gesture based at least in part on analyzing one or more of the data samples including a most recent data sample;

in the event that a possible initiation of a raise gesture is not detected, repeat the analyzing for a subsequent data sample; and

in the event that a possible initiation of a raise gesture is detected;

evaluate each of a sequence of subsequent data samples as either consistent or inconsistent with a raise gesture; and

detect a preheat event in the event that a first threshold number of subsequent data samples consistent with a raise gesture is reached before a second threshold number of subsequent data samples inconsistent with a raise gesture is reached.

14. The method of claim 1 wherein the preheat algorithm generates the preheat event at a first time, the at least one of the raise-gesture detection algorithms generates the raise-gesture event at a second time, and the first time precedes the second time by an amount sufficient that the applications processor is ready to process user input at the second time.

15. The method of claim 1 wherein:

the one or more raise-gesture detection algorithms include a first algorithm that selects criteria for identifying the completed raise gesture, the selection being based on a starting pose, and compares a set of motion sensor data samples to the criteria to detect the completed raise gesture; and

the preheat detection algorithm is based on evaluating a succession of motion sensor data samples as being either consistent or inconsistent with a raise gesture in progress.

16. The method of claim 15 wherein the criteria correspond to a likely amount and direction of motion to bring a display of the device from the starting pose into a user'"'"'s line of sight.

17. The method of claim 15 wherein the preheat detection algorithm generates the preheat event at a first time, the at least one of the raise-gesture detection algorithms generates the raise-gesture event at a second time, and the first time precedes the second time by an amount sufficient that the applications processor is ready to process user input at the second time.

18. The method of claim 1 wherein the criteria correspond to a likely amount and direction of motion to bring a display of the device from the starting pose into a user'"'"'s line of sight.

19. The method of claim 1 wherein executing the preheat gesture detection algorithm includes:

detecting a possible initiation of a raise gesture based on one or more motion-sensor data samples; and

detecting a preheat event based on an evaluation of each of a set of subsequent motion-sensor data samples as being either consistent with a raise gesture or inconsistent with a raise gesture.

20. The method of claim 19 wherein detecting the preheat event includes determining that a first threshold number of subsequent motion sensor data samples consistent with a raise gesture is reached before a second threshold number of subsequent motion sensor data samples inconsistent with a raise gesture is reached.

21. A device comprising:

a user interface component having an active state and an inactive state;

an applications processor coupled to the user interface component, the applications processor having a sleep state and a wake state;

a coprocessor coupled to the applications processor; and

a motion sensor to detect motion of the device, the motion sensor coupled to the coprocessor,

wherein the coprocessor is configured to;

process motion-sensor data received from the motion sensor using a plurality of processing algorithms to detect gesture-related events including preheat events corresponding to initiation of a raise gesture and raise-gesture events corresponding to a completed raise gesture,

wherein the plurality of processing algorithms includes a raise-gesture detection algorithm that includes selecting criteria for identifying a raise gesture based on a starting pose determined from a set of motion sensor data samples and detecting the completed raise gesture based at least in part on whether the selected criteria are satisfied; and

notify the applications processor of the preheat events and raise-gesture events, and

wherein the applications processor is configured to;

transition from the sleep state to the wake state in response to a notification of a preheat event; and

instruct the user interface component to transition from the inactive state to the active state in response to a notification of a raise-gesture event.

22. The device of claim 21 wherein the device is a wrist-wearable device.

23. The device of claim 21 wherein the applications processor is further configured to transition from the sleep state to the wake state in the event that a notification of a raise gesture is received while the applications processor is in the sleep state.

24. The device of claim 21 wherein at least two of the plurality of processing algorithms are concurrently operable.

25. The device of claim 21 wherein the motion sensor includes one or more of an accelerometer or a gyroscopic sensor.

26. The device of claim 21 wherein the plurality of processing algorithms includes a loss-of-focus algorithm to detect a loss-of-focus event corresponding to a user moving a device out of a focus pose, wherein the coprocessor is further configured to notify the applications processor when the loss-of-focus event is detected, and wherein the applications processor is further configured to transition the user interface to the inactive state in response to the notification of the loss-of-focus event.

27. The device of claim 26 wherein the coprocessor is further configured to:

enable the loss-of-focus algorithm and disable all of the gesture detection algorithms in response to detecting the raise gesture; and

enable at least two of the gesture detection algorithms and disable the loss-of-focus algorithm in response to detecting the loss-of-focus event.

28. The device of claim 21 wherein the user interface component includes a display and wherein the display is powered off when the user interface component is in the inactive state and powered on when the user interface component is in the active state.

29. The device of claim 21 wherein the user interface component includes a touch-screen display and wherein the touch-screen display is powered off when the user interface component is in the inactive state and powered on when the user interface component is in the active state.

30. The device of claim 21 wherein the plurality of processing algorithms includes on or more of:

an algorithm to detect a natural raise gesture;

an algorithm to detect a preheat event, or

an algorithm to detect a deliberate raise gesture.