Raise gesture detection in a device

• US 10,303,239 B2
• Filed: 09/13/2018
• Issued: 05/28/2019
• Est. Priority Date: 07/18/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Tu, Xiaoyuan, Kandangath, Anil K., Howell, Adam S.
• Primary Examiner(s): Lin, Chun-Nan
• Application Number:
  US16/130,725
• Publication Number:
  US 20190011973A1
• Time in Patent Office: 257 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3218 of display devices
• CPC Class Codes: G06F 1/3234 Power saving characterised by the action undertaken
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A wearable computing device can detect device-raising gestures. For example, onboard motion sensors of the device can detect movement of the device in real time and infer information about the spatial orientation of the device. Based on analysis of signals from the motion sensors, the device can detect a raise gesture, which can be a motion pattern consistent with the user moving the device'"'"'s display into his line of sight. In response to detecting a raise gesture, the device can activate its display and/or other components. Detection of a raise gesture can occur in stages, and activation of different components can occur at different stages.

Claims

1. A method executed in a wearable device, the method comprising:

obtaining, by a motion coprocessor of the wearable device, a set of motion sensor data samples from a motion sensor of the wearable device, the set of motion sensor data samples being obtained while an applications processor of the wearable device is in a sleep state and a user interface component of the wearable device is in an inactive state;

analyzing, by the motion coprocessor, the set of motion sensor data samples using one or more detection processes to detect a raise gesture;

in response to detecting the raise gesture, waking, by the motion coprocessor, the applications processor from the sleep state;

executing, by the applications processor, a confirmation process on at least a portion of the set of motion sensor data samples to confirm the detected raise gesture;

determining, by the applications processor, whether the user interface component should be activated, the determination being based at least in part on a result of the confirmation process; and

in response to determining that the user interface component should be activated, activating, by the applications processor, the user interface component.

2. The method of claim 1 wherein analyzing the set of motion sensor data samples to identify the raise gesture includes concurrently executing at least two different detection processes to detect a natural raise gesture.

3. The method of claim 2 wherein the at least two different detection processes include:

a preheat detection process that detects an initiation of a raise gesture; and

a raise-gesture detection process that detects a complete raise gesture,

wherein the motion coprocessor wakes the applications processor from the sleep state in response to detecting either the initiation of the raise gesture or the complete raise gesture.

4. The method of claim 3 wherein the raise-gesture detection process includes:

selecting a set of criteria for detecting a raise gesture based on a starting pose determined from the set of motion sensor data samples; and

detecting the raise gesture based on determining that the set of motion sensor data samples satisfies the selected set of criteria.

5. The method of claim 3 wherein the preheat detection process includes:

detecting a possible initiation of a raise gesture based at least in part on a first one of the motion sensor data samples; and

in response to detecting the possible initiation of the raise gesture;

sequentially evaluating each motion sensor data sample subsequent to the first motion sensor data sample as either consistent or inconsistent with a raise gesture; and

identifying the initiation of the raise gesture in the event that a first threshold number of subsequent motion sensor data samples consistent with a raise gesture is reached before a second threshold number of subsequent motion sensor data samples inconsistent with a raise gesture is reached.

6. The method of claim 1 wherein the confirmation process includes applying a gesture classifier that has been trained using a machine-learning process to distinguish a raise gesture from one or more other motion types.

7. The method of claim 6 wherein the gesture classifier is a Bayesian classifier.

8. A wearable device comprising:

a motion sensor to detect motion of the wearable device;

a motion coprocessor coupled to the motion sensor;

an applications processor coupled to the motion coprocessor, the applications processor having a sleep state and a wake state;

a user interface component coupled to the applications processor, the user interface component having an active state and an inactive state,

wherein the motion coprocessor is configured to;

obtain, from the motion sensor, a set of motion sensor data samples while the applications processor is in the sleep state and the user interface component is in the inactive state;

analyze the set of motion sensor data samples using one or more detection processes to detect a raise gesture; and

wake the applications processor from the sleep state in response to detecting the raise gesture; and

wherein the applications processor is configured to;

execute a confirmation process on at least a portion of the set of motion sensor data samples to confirm the detected raise gesture;

determine, based at least in part on a result of the confirmation process, whether the user interface component should be activated; and

activate the user interface component in response to determining that the user interface component should be activated.

9. The wearable device of claim 8 wherein the user interface component includes a display that is powered off when the user interface component is in the inactive state and powered on when the user interface component is in the active state.

10. The wearable device of claim 8 wherein the user interface component includes a touch-screen display that is powered off when the user interface component is in the inactive state and powered on when the user interface component is in the active state.

11. The wearable device of claim 8 wherein the motion sensor includes an accelerometer and a gyroscope and wherein the motion coprocessor is further configured to:

determine whether a user of the wearable device is engaged in a high-dynamic activity or a low-dynamic activity;

obtain the set of motion sensor data samples from both the accelerometer and the gyroscope in response to determining that the user of the wearable device is engaged in a high-dynamic activity; and

obtain the set of motion sensor data samples from only the accelerometer in response to determining that the user of the wearable device is engaged in a low-dynamic activity.

12. The wearable device of claim 8 wherein the motion coprocessor is further configured such that analyzing the set of motion sensor data samples to identify the raise gesture includes concurrently executing at least two different detection processes to detect a natural raise gesture.

13. The wearable device of claim 12 wherein the at least two different detection processes include:

a preheat detection process that detects initiation of a raise gesture; and

a raise-gesture detection process that detects a complete raise gesture,

wherein the motion coprocessor is further configured to wake the applications processor from the sleep state in response to detecting either the initiation of the raise gesture or the complete raise gesture.

14. A computer-readable storage medium having stored therein program instructions that, when executed by a motion coprocessor and an applications processor in a wearable device, cause the wearable device to perform a method comprising:

obtaining, by the motion coprocessor, a set of motion sensor data samples from a motion sensor of the wearable device, the set of motion sensor data samples being obtained while the applications processor is in a sleep state and a user interface component of the wearable device is in an inactive state;

analyzing, by the motion coprocessor, the set of motion sensor data samples using one or more detection processes to detect a raise gesture;

in response to detecting the raise gesture, waking, by the motion coprocessor, the applications processor from the sleep state;

executing, by the applications processor, a confirmation process on at least a portion of the set of motion sensor data samples to confirm the detected raise gesture;

determining, by the applications processor, whether the user interface component should be activated, the determination being based at least in part on a result of the confirmation process; and

in response to determining that the user interface component should be activated, activating, by the applications processor, the user interface component.

15. The computer-readable storage medium of claim 14 wherein analyzing the set of motion sensor data samples to identify the raise gesture includes concurrently executing at least two different detection processes to detect a natural raise gesture.

16. The computer-readable storage medium of claim 15 wherein the at least two different detection processes include:

a preheat detection process that detects initiation of a raise gesture; and

a raise-gesture detection process that detects a complete raise gesture,

wherein the motion coprocessor wakes the applications processor from the sleep state in response to detecting either the initiation of the raise gesture or the complete raise gesture.

17. The computer-readable storage medium of claim 16 wherein the raise-gesture detection process includes:

selecting a set of criteria for identifying a raise gesture based on a starting pose determined from the set of motion sensor data samples; and

detecting the raise gesture based on determining that the set of motion sensor data samples satisfies the selected set of criteria.

18. The computer-readable storage medium of claim 16 wherein the preheat detection process includes:

detecting a possible initiation of a raise gesture based at least in part on a first one of the motion sensor data samples; and

in response to detecting the possible initiation of the raise gesture;

sequentially evaluating each motion sensor data sample subsequent to the first motion sensor data sample as either consistent or inconsistent with a raise gesture; and

identifying the initiation of the raise gesture in the event that a first threshold number of subsequent motion sensor data samples consistent with a raise gesture is reached before a second threshold number of subsequent motion sensor data samples inconsistent with a raise gesture is reached.

19. The computer-readable storage medium of claim 14 wherein the confirmation process includes applying a gesture classifier that has been trained using a machine-learning process to distinguish a raise gesture from one or more other motion types.

20. The computer-readable storage medium of claim 19 wherein the gesture classifier is a Bayesian classifier.