Raise gesture detection in a device

• US 10,101,793 B2
• Filed: 07/10/2015
• Issued: 10/16/2018
• Est. Priority Date: 07/18/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Tu, Xiaoyuan, Kandangath, Anil K., Howell, Adam S.
• Primary Examiner(s): Lin, Chun-Nan
• Application Number:
  US14/796,675
• Publication Number:
  US 20160018872A1
• Time in Patent Office: 1,194 Days
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/3206 Monitoring of events, devices or parameters that trigger a change in power modality
• CPC Class Codes: G06F 1/3218 of display devices
• CPC Class Codes: G06F 1/3234 Power saving characterised by the action undertaken
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A wearable computing device can detect device-raising gestures. For example, onboard motion sensors of the device can detect movement of the device in real time and infer information about the spatial orientation of the device. Based on analysis of signals from the motion sensors, the device can detect a raise gesture, which can be a motion pattern consistent with the user moving the device'"'"'s display into his line of sight. In response to detecting a raise gesture, the device can activate its display and/or other components. Detection of a raise gesture can occur in stages, and activation of different components can occur at different stages.

Claims

1. A method for detecting a raise gesture in a wrist-wearable device, the method comprising:

obtaining, at a motion coprocessor in the wrist-wearable device, a set of motion-sensor data samples from an internal motion sensor of the wrist-wearable device, the set of motion-sensor data samples corresponding to a time interval;

determining, by the motion coprocessor, based on the set of motion-sensor data samples, a starting pose corresponding to a beginning of the time interval;

selecting, by the motion coprocessor, based at least in part on the starting pose, a set of criteria for identifying a possible raise gesture using the set of motion-sensor data samples;

identifying, by the motion coprocessor, the possible raise gesture based at least in part on whether the set of motion-sensor data samples satisfies the selected set of criteria;

waking, by the motion coprocessor, an applications processor of the wrist-wearable device from a sleep state in response to identifying the possible raise gesture;

identifying, by the motion coprocessor, a detected raise gesture in the event that the identified possible raise gesture is followed by at least a minimum period of dwell time in a focus pose; and

activating a display of the wrist-wearable device in response to identifying the detected raise gesture.

2. The method of claim 1 further comprising:

transitioning a user interface component of the wrist-wearable device from an inactive state to a partially active state in response to identifying the possible raise gesture and from the partially active state to an active state in response to identifying the detected raise gesture.

3. The method of claim 1 wherein the motion-sensor data includes accelerometer data and the selected set of criteria for identifying a possible raise gesture includes one or more of:

a criterion based on an accrued tilt determined from the accelerometer data during the time interval;

a criterion based on an amount of lifting of the wrist-wearable device, the amount of lifting determined from the accelerometer data during the time interval;

or

a criterion based on smoothness of the motion during the time interval.

4. The method of claim 1 wherein the motion-sensor data includes gyroscope data from a gyroscopic sensor and the selected set of criteria for identifying a possible raise gesture includes a criterion based on an accrued rotation about an axis substantially parallel to a user'"'"'s forearm, the accrued rotation being determined from the gyroscope data during the time interval.

5. The method of claim 1 wherein the motion-sensor data includes gyroscope data from a gyroscopic sensor and accelerometer data from an accelerometer and wherein:

the gyroscope data is used to identify the possible raise gesture; and

the accelerometer data is used to identify the detected raise gesture.

6. The method of claim 1 further comprising:

subsequently to identifying the detected raise gesture, using a gesture classifier to confirm the detected raise gesture,

wherein a user interface component of the wrist-wearable device is activated in response to confirmation by the gesture classifier.

7. The method of claim 1 further comprising:

executing, by the motion coprocessor, a preheat detection algorithm to detect initiation of the raise gesture, wherein the preheat detection algorithm executes concurrently with performing the determining, selecting, and identifying acts; and

waking the applications processor from the sleep state in the event that initiation of the raise gesture is detected while the applications processor is in the sleep state.

8. The method of claim 1 further comprising:

obtaining, by the motion coprocessor, an indication of an activity in which a user of the wrist-wearable device is engaged; and

determining, by the motion coprocessor, whether the activity is a high-dynamic activity or a low-dynamic activity, wherein;

in response to determining that the activity is a high-dynamic activity, obtaining the set of motion-sensor data samples includes obtaining both accelerometer and gyroscope data samples; and

in response to determining that the activity is a low-dynamic activity, obtaining the set of motion-sensor data samples includes obtaining only accelerometer data samples.

9. A wearable device comprising:

a user interface component having an active state and an inactive state;

a motion sensor to detect motion of the wearable device;

an applications processor; and

a motion coprocessor configured to;

obtain a set of motion-sensor data samples from the motion sensor, the set of motion-sensor data samples corresponding to a time interval;

determine, based on the set of motion-sensor data samples, a starting pose corresponding to a beginning of the time interval;

select, based at least in part on the starting pose, a set of criteria for identifying a possible raise gesture using the set of motion-sensor data samples;

identify the possible raise gesture based at least in part on whether the set of motion-sensor data samples satisfies the selected set of criteria;

wake the applications processor from a sleep state in response to identifying the possible raise gesture;

identify a detected raise gesture in the event that the identified possible raise gesture is followed by at least a minimum period of dwell time in a focus pose; and

activate the user interface component in response to identifying the detected raise gesture.

10. The wearable device of claim 9 wherein the user interface component includes one or more of:

a display;

a touch-screen display;

a microphone;

a speaker;

or

a button.

11. The wearable device of claim 9 wherein the motion sensor includes one or more of:

an accelerometer;

or

a gyroscopic sensor.

12. The wearable device of claim 9 wherein the motion sensor includes an accelerometer and a gyroscopic sensor, and wherein the motion coprocessor is further configured to:

obtain an indication of an activity in which a user of the wearable device is engaged; and

determine whether the activity is a high-dynamic activity or a low-dynamic activity,

wherein the motion coprocessor is further configured such that obtaining the set of motion sensor data samples includes;

in response to determining that the activity is a high-dynamic activity, obtaining both accelerometer and gyroscope data samples; and

in response to determining that the activity is a low-dynamic activity, obtaining only accelerometer data samples.

13. A computer-readable storage medium having stored therein program instructions that, when executed by a motion coprocessor in a wrist-wearable device, cause the motion coprocessor to perform a method comprising:

obtaining a set of motion-sensor data samples from an internal motion sensor of the wrist-wearable device, the set of motion-sensor data samples corresponding to a time interval;

determining, based on the set of motion-sensor data samples, a starting pose corresponding to a beginning of the time interval;

selecting, based at least in part on the starting pose, a set of criteria for identifying a possible raise gesture using the set of motion-sensor data samples;

identifying the possible raise gesture based at least in part on whether the set of motion-sensor data samples satisfies the selected set of criteria;

waking an applications processor of the wrist-wearable device from a sleep state in response to identifying the possible raise gesture;

identifying a detected raise gesture in the event that the identified possible raise gesture is followed by at least a minimum period of dwell time in a focus pose; and

activating a display of the wrist-wearable device in response to identifying the detected raise gesture.

14. The computer-readable storage medium of claim 13 wherein the method further comprises:

detecting a first impulse peak and a second impulse peak in the motion-sensor data samples, the first and second impulse peaks each indicating movement in a direction corresponding to a rotation of a wrist of a user wearing the wrist-wearable device, the first and second impulse peaks being separated in time by a period less than a timeout period;

detecting a deliberate raise gesture based at least in part on detecting the first and second impulse peaks; and

waking the applications processor and the display in the event that a deliberate raise gesture is detected.

15. The computer-readable storage medium of claim 14 wherein the method further comprises:

determining that, following the second impulse peak, the wrist-wearable device is held in an approximately stable position for at least a minimum dwell time,

wherein the deliberate raise gesture is detected based in part on detecting the first and second impulse peaks and in part on determining that the wrist-wearable device is held in the approximately stable position.

16. The computer-readable storage medium of claim 14 wherein the motion sensor includes an accelerometer and wherein detecting the first and second impulse peaks is based at least in part on a jerk determined from accelerometer data samples.