Efficient gesture processing

• US 9,535,506 B2
• Filed: 03/11/2014
• Issued: 01/03/2017
• Est. Priority Date: 07/13/2010
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Raffa, Giuseppe, Nachman, Lama, Lee, Jinwon
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): FINK, THOMAS ANDREW
• Application Number:
  US14/205,210
• Publication Number:
  US 20140191955A1
• Time in Patent Office: 1,029 Days
• US Class Current: 1/1
• CPC Class Codes: G01C 19/00 Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects
• CPC Class Codes: G01P 15/18 in two or more dimensions
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion

Abstract

Embodiments of the invention describe a system to efficiently execute gesture recognition algorithms. Embodiments of the invention describe a power efficient staged gesture recognition pipeline including multimodal interaction detection, context based optimized recognition, and context based optimized training and continuous learning. Embodiments of the invention further describe a system to accommodate many types of algorithms depending on the type of gesture that is needed in any particular situation. Examples of recognition algorithms include but are not limited to, HMM for complex dynamic gestures (e.g. write a number in the air), Decision Trees (DT) for static poses, peak detection for coarse shake/whack gestures or inertial methods (INS) for pitch/roll detection.

Claims

1. At least one non-transitory computer readable storage medium having instructions stored thereon that, when executed on a machine, cause the machine to:

receive data from a motion sensor;

select a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on an amplitude of the data;

determine an energy magnitude of the data based on the amplitude of the data; and

determine a gesture from the data based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data.

3. The at least one computer readable storage medium of claim 1, wherein the machine is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of minimum/maximum energy magnitude values of the data with minimum/maximum energy magnitude values associated with each of the plurality of gesture algorithms.

4. The at least one computer readable storage medium of claim 1, the machine to further:

determine a frequency spectrum of the data based, at least in part, on the amplitude of the data and a phase of the data.

5. The at least one computer readable storage medium of claim 4, wherein the machine is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of the frequency spectrum of the data with one or more spectrum patterns associated with each of the plurality of gesture algorithms.

6. The at least one computer readable storage medium of claim 1, wherein the motion sensor comprises at least one of an accelerometer or a gyroscope.

2. The at least one computer readable storage medium 1, wherein the machine is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of a total energy magnitude of the data with a total energy magnitude value associated with each of the plurality of gesture algorithms.

7. A mobile computing device comprising:

a motion sensor;

a memory;

at least one processor;

an algorithm selection module, stored in the memory and executed via the at least one processor, to select a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on an amplitude of a data from the motion sensor and determine an energy magnitude of the data from the motion sensor based on the amplitude of the data from the motion sensor; and

a gesture recognition module, stored in the memory and executed via the at least one processor, to determine a gesture from the data from the motion sensor based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data from the motion sensor.

8. The mobile computing device of claim 7, wherein the at least one processor comprises a low power processing unit to execute the algorithm selection module, and a main processing unit to execute the gesture recognition module.

9. The mobile computing device of claim 7, wherein the algorithm selection module is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of a total energy magnitude of the data from the motion sensor with a total energy magnitude value associated with each of the plurality of gesture algorithms.

10. The mobile computing device of claim 7, wherein the algorithm selection module is to select the subset of gesture recognition algorithm(s) based, at least in part, on a comparison of minimum/maximum energy magnitude values of the data from the motion sensor with minimum/maximum energy magnitude values associated with each of the plurality of gesture algorithms.

11. The mobile computing device of claim 7, wherein the algorithm selection module is to further:

determine a frequency spectrum of the data from the motion sensor based, at least in part, on the amplitude of the data from the motion sensor and a phase of the data from the motion sensor.

12. The mobile computing device of claim 11, wherein the algorithm selection module is to select the subset of gesture recognition algorithm(s) is based, at least in part, on a comparison of the frequency spectrum of the data from the motion sensor with one or more spectrum patterns associated with each of the plurality of gesture algorithms.

13. The mobile computing device of claim 7, wherein the motion sensor comprises an accelerometer.

14. The mobile computing device of claim 7, wherein the motion sensor comprises a gyroscope.

15. The mobile computing device of claim 7, further comprising:

a wearable housing including the motion sensor, the at least one processor, and the memory.

16. A machine-implemented method comprising:

receiving data from a motion sensor;

selecting a subset of one or more gesture recognition algorithms from a plurality of gesture recognition algorithms based, at least in part, on an amplitude of the data;

determining an energy magnitude of the data based on the amplitude of the data; and

determining a gesture from the data based, at least in part, on applying the subset of gesture recognition algorithm(s) to the data.

17. The machine-implemented method of claim 16, wherein selecting the subset of gesture recognition algorithm(s) is based, at least in part, on at least one of:

comparing a total energy magnitude of the data with a total energy magnitude value associated with each of the plurality of gesture algorithms;

or

comparing minimum/maximum energy magnitude values of the data with minimum/maximum energy magnitude values associated with each of the plurality of gesture algorithms.

18. The machine-implemented method of claim 16, further comprising:

determining a frequency spectrum of the data based, at least in part, on the amplitude of the data and a phase of the data;

wherein selecting the subset of gesture recognition algorithm(s) is based, at least in part, on comparing the frequency spectrum of the data with one or more spectrum patterns associated with each of the plurality of gesture algorithms.