Method and system for gesture recognition

• US 9,442,570 B2
• Filed: 03/13/2013
• Issued: 09/13/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Slonneger, Andrew M
• Primary Examiner(s): PARDO, THUY N
• Application Number:
  US13/800,743
• Publication Number:
  US 20140282270A1
• Time in Patent Office: 1,280 Days
• Field of Search: 345/156, 345/173, 345/169, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method and system for recognizing gestures on an electronic device, such as a mobile device (e.g., watch), are disclosed. In one example embodiment, the method includes obtaining a gesture template, determining a first mean value based upon the gesture template, obtaining gesture data by way of a motion sensing component of the electronic device, and calculating (by way of a processing device) a correlation metric based at least indirectly upon the gesture data and the gesture template, where the correlation metric is calculated based at least in part upon the first mean value. The method also includes determining based at least in part upon the correlation metric that a first of the gestures has occurred, and taking at least one additional action based at least in part upon the determining.

Claims

1. A method of recognizing spatial gestures of an electronic device, the method comprising:

obtaining a gesture template that includes movement data representative of a particular gesture type;

obtaining, by one or more processors of the electronic device and from a motion sensing component of the electronic device, motion data indicative of motion of the electronic device;

calculating, by the one or more processors, a correlation metric based at least part on a mean value of the movement data of the gesture template, a variance of the movement data of the gesture template that is based in part on the mean value of the movement data of the gesture template, the motion data, and the gesture template;

determining, based at least in part on the correlation metric, that a gesture of the particular gesture type has occurred, and

performing, in response to determining that the gesture of the particular gesture type has occurred, at least one action.

2. The method of claim 1, wherein the movement data for the particular gesture type includes a first number of values and the gesture data includes a second number of values.

3. The method of claim 2, wherein the first number equals the second number.

4. The method of claim 1, wherein the correlation metric is calculated based at least in part on a modified variance of the motion data and a modified covariance of the motion data, wherein each of the modified variance and modified covariance is calculated at least in part based on the mean value.

5. The method of claim 4, wherein the correlation metric (M_corr) is calculated based on the following equation:

6. The method of claim 1, wherein the determination that the gesture has occurred is further based on a peak-to-peak metric of the motion data.

7. The method of claim 6, further comprising:

determining, based at least in part on the motion data, that the motion data is appropriate for an occurrence of a gesture of the particular gesture type.

8. The method of claim 7, wherein determining that the motion data is appropriate for the occurrence of a gesture of the particular gesture type is further based on one or more of a position of the electronic device, a tilt of the electronic device, or an orientation of the electronic device as indicated by at least some of the motion data.

9. The method of claim 1, further comprising:

responsive to determining that the motion data is inconsistent with a gesture type of the particular gesture, discarding an old value included in the motion data and obtaining a new value to so as to update the motion data.

10. The method of claim 1, wherein the electronic device is a wearable mobile device, and wherein the at least one action includes outputting a signal by way of an output device of the wearable mobile device.

11. A method of recognizing gestures, the method comprising:

obtaining a gesture template that includes movement data representative of a particular gesture type;

determining, by one or more processors of a mobile device, that a status of the mobile device is appropriate for an occurrence of a gesture of the particular gesture type based on one or more of a detected position of the mobile device, a detected tilt of the mobile device, and a detected orientation of the mobile device;

responsive to determining that the status is appropriate for the occurrence of a gesture of the particular gesture type, obtaining, by the one or more processors and from an accelerometer of the mobile device, motion data;

determining, by the one or more processors, that the motion data includes a sufficient number of samples to allow for recognition of a gesture of the particular gesture type;

calculating, by the one or more processors, a correlation metric based at least on the motion data and the gesture template; and

responsive to determining, based at least in part on the correlation metric, that a gesture of the particular gesture type has occurred taking at least one action.

12. The method of claim 11, wherein determining the correlation metric comprises:

determining the correlation metric based at least on the motion data, the gesture template, a mean value of the movement data of the gesture template, and a variance of the movement data of the gesture template that is based in part on the mean value of the movement data of the gesture template.

13. The method of claim 12, wherein the determining that the gesture of the particular gesture type has occurred is further based at least in part on a peak-to-peak metric of the motion data.

14. The method of claim 12, wherein the correlation metric (M_corr) is calculated based on the following equation:

15. The method of claim 12, wherein the calculating of the correlation metric is not based on a mean value determined based on the motion data.

16. A mobile device comprising:

a motion sensing component configured to generate motion data indicative of motion of the mobile device;

at least one memory device configured to store a gesture template that includes movement data representative of a particular gesture type;

at least one processing device coupled to the motion sensing component and the memory device, wherein the processing device is configured to;

calculate a correlation metric based at least in part on the motion data, the gesture template, and a mean value of the movement data of the gesture template, and a variance of the movement data of the gesture template that is based in part on the mean value of the movement data of the gesture template;

determine, based at least in part on the correlation metric, that a gesture of the particular gesture type has occurred; and

perform, in response to determining that the gesture of the particular gesture type has occurred, at least one action.

17. The mobile device of claim 16, wherein the motion sensing component includes one or more of an accelerometer, a gyroscope, and a barometer.

18. The mobile device of claim 16, wherein the at least one processing device is further configured to:

calculate a peak-to-peak metric of the motion data, and

determine that the gesture of the particular gesture type has occurred further based at least in part on the peak-to-peak metric of the motion data.

19. The mobile device of claim 16, wherein the at least one processing device operates in accordance with a state machine having a first state that involves not collecting the motion data, a second state that does involve collecting the motion data, and a third state that involves assessing the motion data, wherein the assessing includes the calculating of the correlation metric.

20. The mobile device of claim 16, wherein the at least one processing device is further configured to:

determine that a status of the mobile device is appropriate for an occurrence of a gesture of the particular gesture type based on one or more of a detected position of the mobile device, a detected tilt of the mobile device, and a detected orientation of the mobile device; and

calculate the correlation metric in response to determining that the status is appropriate for the occurrence of a gesture of the particular gesture type.