Wearable computing device with gesture recognition

• US 9,164,588 B1
• Filed: 02/05/2013
• Issued: 10/20/2015
• Est. Priority Date: 02/05/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Balez, Mat, Sparks, David, Starner, Thad Eugene, Johnson, Michael Patrick
• Primary Examiner(s): Sitta, Grant
• Assistant Examiner(s): HERMANN, KIRK W
• Application Number:
  US13/760,048
• Time in Patent Office: 987 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0185 Displaying image at variable distance
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus

Abstract

Methods, apparatus, and computer-readable media are described herein related to recognizing a look up gesture. Level-indication data from at least an accelerometer associated with a wearable computing device (WCD) can be received. The WCD can be worn by a wearer. The WCD can determine whether a head of the wearer is level based on the level-indication data. In response to determining that the head of the wearer is level, the WCD can receive lookup-indication data from at least the accelerometer. The WCD can determine whether the head of the wearer is tilted up based on the lookup-indication data. In response to determining that the head of the wearer is tilted up, the WCD can generate a gesture-recognition trigger, where the gesture-recognition trigger indicates that the head of the wearer has moved up from level.

Claims

1. A method, comprising

receiving level-indication data at a wearable computing device (WCD) from at least an accelerometer, wherein the WCD is worn by a wearer;

determining whether a head of the wearer is level relative to gravity based on the level-indication data using the WCD, wherein determining whether the head of the wearer is level comprises comparing a magnitude of the level-indication data to a minimum magnitude threshold and a maximum magnitude threshold; and

in response to determining that the head of the wearer is level, the WCD;

receiving lookup-indication data from at least the accelerometer,

determining whether the head of the wearer is tilted up relative to gravity based on the lookup-indication data, and

in response to determining that the head of the wearer is tilted up,

generating a gesture-recognition trigger, wherein the gesture-recognition trigger indicates that the head of the wearer has moved up from level.

2. The method of claim 1, wherein determining whether the head of the wearer is level comprises:

determining whether the magnitude of the level-indication data is greater than the maximum magnitude threshold;

in response to determining that the magnitude of the level-indication data is greater than the maximum magnitude threshold, determining that the head of the wearer is not level; and

in response to determining that the magnitude of the level-indication data is less than the minimum magnitude threshold, determining that the head of the wearer is not level.

3. The method of claim 1, wherein determining whether the head of the wearer is level comprises:

defining an X-axis, a Y-axis, and a Z-axis for the accelerometer;

determining an accelerometer vector based on the level-indication data, the accelerometer vector comprising an X component along the X-axis, a Y component along the Y-axis, and a Z component along the Z-axis;

determining a level cone;

determining whether the Y component of the accelerometer vector is within the level cone;

in response to determining that the Y component of the accelerometer vector is within the level cone, determining that the head of the wearer is level; and

in response to determining that the Y component of the accelerometer vector is not within the level cone, determining that the head of the wearer is not level.

4. The method of claim 1, wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining a second accelerometer vector based on the lookup-indication data, the second accelerometer vector comprising an X component along the X-axis, a Y component along the Y-axis, and a Z component along the Z-axis;

determining a roll cone about the X-axis;

determining whether the X component of the second accelerometer vector is outside of the roll cone; and

in response to determining that the X component of the second accelerometer vector is outside of the roll cone, determining that the head of the wearer is not tilted up.

5. The method of claim 4, wherein determining whether the head of the wearer is tilted up based on the lookup-indication data further comprises:

in response to determining that the X component of the second accelerometer vector is inside of the roll cone;

determining a lookup-gesture-trigger cone about the Y-axis;

determining whether the Y component of the second accelerometer vector is outside of the lookup-gesture-trigger cone;

in response to determining that the Y component of the second accelerometer vector is not outside of the lookup-gesture-trigger cone, determining that the head of the wearer is not tilted up; and

in response to determining that the Y component of the second accelerometer vector is outside of the lookup-gesture-trigger cone;

determining whether the Z component of the second accelerometer vector corresponds to an upwards tilt,

in response to determining that the Z component of the second accelerometer vector does not correspond to the upwards tilt, determining that the head of the wearer is not tilted up, and

in response to determining that the Z component of the second accelerometer vector does correspond to the upwards tilt, determining that the head of the wearer is tilted up.

6. The method of claim 1, wherein the lookup-indication data comprises accelerometer data from the accelerometer and gyroscope data from a gyroscope, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining an accelerometer vector based on the accelerometer data;

determining whether the head of the wearer has apparently tilted up based on the accelerometer vector;

determining whether the head of the wearer has rotated based on the rotation data;

in response to either determining that the head of the wearer has not apparently tilted up based on the accelerometer vector or determining that the head of the wearer has not rotated based on the rotation data, determining that the head of the wearer is not tilted up; and

in response to determining that the head of the wearer has apparently tilted up based on the accelerometer vector and determining that the head of the wearer has rotated based on the rotation data, determining that the head of the wearer is tilted up.

7. The method of claim 1, wherein the lookup-indication data comprises first accelerometer data from the accelerometer and second accelerometer data from a second accelerometer, wherein the second accelerometer is located closer to a pivot point of the head of the wearer than the first accelerometer, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

defining an X-axis, a Y-axis, and a Z-axis for the WCD;

determining a first accelerometer vector based on the first accelerometer data, the first accelerometer vector comprising a first Y-component along the Y-axis;

determining a second accelerometer vector based on the second accelerometer data, the second accelerometer vector comprising a second Y-component along the Y-axis;

determining an absolute value of an accelerometer vector difference (AVD) between the first and second accelerometer vectors,

determining whether the absolute value of the AVD exceeds an AVD threshold;

determining a Y-component difference between the first Y-component and the second Y-component;

in response to either determining that the Y-component difference is not positive or determining that the absolute value of the AVD does not exceed the AVD threshold, determining that the head of the wearer is not tilted up; and

in response to determining that the Y-component difference is positive and determining that the absolute value of the AVD does exceed the AVD threshold, determining that the head of the wearer is tilted up.

8. The method of claim 1, further comprising:

in response to the gesture-recognition trigger, generating a gesture-recognition interrupt.

9. The method of claim 1, wherein the lookup-indication data comprises accelerometer data from the accelerometer and magnetic-field data from a magnetometer, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining an accelerometer vector based on the accelerometer data;

determining whether the head of the wearer has apparently tilted up based on the accelerometer vector;

verifying whether the head of the wearer has tilted up based on the magnetic-field data;

in response to either determining that the head of the wearer has not apparently tilted up based on the accelerometer vector or not verifying that the head of the wearer has tilted up based on the magnetic-field data, determining that the head of the wearer is not tilted up; and

in response to determining that the head of the wearer has apparently tilted up based on the accelerometer vector and verifying that the head of the wearer has tilted up based on the magnetic-field data, determining that the head of the wearer is tilted up.

10. A wearable computing device (WCD), comprising:

an accelerometer;

a processor; and

a computer-readable storage medium having stored thereon program instructions that, upon execution by the processor, cause the WCD to perform operations comprising;

receiving level-indication data from the accelerometer,

determining whether a head of a wearer of the WCD is level relative to gravity based on the level-indication data, wherein determining whether the head of the wearer is level comprises comparing a magnitude of the level-indication data to a minimum magnitude threshold and a maximum magnitude threshold, and

in response to determining that the head of the wearer is level,

receiving lookup-indication data from the accelerometer,

determining whether the head of the wearer is tilted up relative to gravity based on the lookup-indication data, and

in response to determining that the head of the wearer is tilted up,

generating a gesture-recognition trigger, wherein the gesture-recognition trigger indicates that the head of the wearer has moved up from level.

11. The WCD of claim 10, wherein determining whether the head of the wearer is level comprises:

determining whether the magnitude of the level-indication data is greater than the maximum magnitude threshold;

in response to determining that the magnitude of the level-indication data is greater than the maximum magnitude threshold, determining that the head of the wearer is not level; and

in response to determining that the magnitude of the level-indication data is less than the minimum magnitude threshold, determining that the head of the wearer is not level.

12. The WCD of claim 10, wherein determining whether the head of the wearer is level comprises:

defining an X-axis, a Y-axis, and a Z-axis for the accelerometer;

determining an accelerometer vector based on the level-indication data, the accelerometer vector comprising an X component along the X-axis, a Y component along the Y-axis, and a Z component along the Z-axis;

determining a level cone;

determining whether the Y component of the accelerometer vector is within the level cone;

in response to determining that the Y component of the accelerometer vector is within the level cone, determining that the head of the wearer is level; and

in response to determining that the Y component of the accelerometer vector is not within the level cone, determining that the head of the wearer is not level.

13. The WCD of claim 10, wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining a second accelerometer vector based on the lookup-indication data, the second accelerometer vector comprising an X component along the X-axis, a Y component along the Y-axis, and a Z component along the Z-axis;

determining a roll cone about the X-axis;

determining whether the X component of the second accelerometer vector is outside of the roll cone; and

in response to determining that the X component of the second accelerometer vector is outside of the roll cone, determining that the head of the wearer is not tilted up.

14. The WCD of claim 13, wherein determining whether the head of the wearer is tilted up based on the lookup-indication data further comprises:

in response to determining that the X component of the second accelerometer vector is inside of the roll cone;

determining a lookup-gesture-trigger cone about the Y-axis;

determining whether the Y component of the second accelerometer vector is outside of the lookup-gesture-trigger cone;

in response to determining that the Y component of the second accelerometer vector is not outside of the lookup-gesture-trigger cone, determining that the head of the wearer is not tilted up; and

in response to determining that the Y component of the second accelerometer vector is outside of the lookup-gesture-trigger cone;

determining whether the Z component of the second accelerometer vector corresponds to an upwards tilt;

in response to determining that the Z component of the second accelerometer vector does not correspond to the upwards tilt, determining that the head of the wearer is not tilted up; and

in response to determining that the Z component of the second accelerometer vector does correspond to the upwards tilt, determining that the head of the wearer is tilted up.

15. The WCD of claim 10, further comprising a gyroscope, wherein the lookup-indication data comprises accelerometer data from the accelerometer and gyroscope data from the gyroscope, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining an accelerometer vector based on the accelerometer data;

determining whether the head of the wearer has apparently tilted up based on the accelerometer vector;

determining whether the head of the wearer has rotated based on the rotation data;

in response to either determining that the head of the wearer has not apparently tilted up based on the accelerometer vector or determining that the head of the wearer has not rotated based on the rotation data, determining that the head of the wearer is not tilted up; and

in response to determining that the head of the wearer has apparently tilted up based on the accelerometer vector and determining that the head of the wearer has rotated based on the rotation data, determining that the head of the wearer is tilted up.

16. The WCD of claim 10, further comprising a second accelerometer, wherein the lookup-indication data comprises first accelerometer data from the accelerometer and second accelerometer data from the second accelerometer, wherein the second accelerometer is located closer to a pivot point of the head of the wearer than the first accelerometer, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

defining an X-axis, a Y-axis, and a Z-axis for the WCD;

determining a first accelerometer vector based on the first accelerometer data, the first accelerometer vector comprising a first Y-component along the Y-axis;

determining a second accelerometer vector based on the second accelerometer data, the second accelerometer vector comprising a second Y-component along the Y-axis;

determining an absolute value of an accelerometer vector difference (AVD) between the first and second accelerometer vectors;

determining whether the absolute value of the AVD exceeds an AVD threshold;

determining a Y-component difference between the first Y-component and the second Y-component;

in response to either determining that the Y-component difference is not positive or determining that the absolute value of the AVD does not exceed the AVD threshold, determining that the head of the wearer is not tilted up; and

in response to determining that the Y-component difference is positive and determining that the absolute value of the AVD does exceed the AVD threshold, determining that the head of the wearer is tilted up.

17. The WCD of claim 10, further comprising a magnetometer, wherein the lookup-indication data comprises accelerometer data from the accelerometer and magnetic-field data from the magnetometer, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining an accelerometer vector based on the accelerometer data;

determining whether the head of the wearer has apparently tilted up based on the accelerometer vector;

verifying whether the head of the wearer has tilted up based on the magnetic-field data;

in response to either determining that the head of the wearer has not apparently tilted up based on the accelerometer vector or not verifying that the head of the wearer has tilted up based on the magnetic-field data, determining that the head of the wearer is not tilted up; and

in response to determining that the head of the wearer has apparently tilted up based on the accelerometer vector and verifying that the head of the wearer has tilted up based on the magnetic-field data, determining that the head of the wearer is tilted up.

18. The WCD of claim 10, wherein the operations further comprise:

in response to the gesture-recognition trigger, generating a gesture-recognition interrupt.

19. An apparatus, including a non-transitory computer-readable storage medium having stored thereon program instructions that, upon execution by a computing device, cause the apparatus to perform operations comprising:

receiving level-indication data from an accelerometer;

determining whether a head of a wearer of the apparatus is level relative to gravity based on the level-indication data, wherein determining whether the head of the wearer is level comprises comparing a magnitude of the level-indication data to a minimum magnitude threshold and a maximum magnitude threshold; and

in response to determining that the head of the wearer is level, receiving lookup-indication data from the accelerometer, determining whether the head of the wearer is tilted up relative to gravity based on the lookup-indication data, and in response to determining that the head of the wearer is tilted up, generating a gesture-recognition trigger, wherein the gesture-recognition trigger indicates that the head of the wearer has moved up from level.

20. The apparatus of claim 19, wherein determining whether the head of the wearer is level comprises:

determining whether the magnitude of the level-indication data is greater than the maximum magnitude threshold;

in response to determining that the magnitude of the level-indication data is greater than the maximum magnitude threshold, determining that the head of the wearer is not level; and

in response to determining that the magnitude of the level-indication data is less than the minimum magnitude threshold, determining that the head of the wearer is not level.

21. The apparatus of claim 19, wherein determining whether the head of the wearer is level comprises:

defining an X-axis, a Y-axis, and a Z-axis for the accelerometer;

determining an accelerometer vector based on the level-indication data, the accelerometer vector comprising an X component along the X-axis, a Y component along the Y-axis, and a Z component along the Z-axis;

determining a level cone;

determining whether the Y component of the vector is within the level cone;

in response to determining that the Y component of the accelerometer vector is within the level cone, determining that the head of the wearer is level; and

in response to determining that the Y component of the accelerometer vector is not within the level cone, determining that the head of the wearer is not level.

22. The apparatus of claim 19, wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining a second accelerometer vector based on the lookup-indication data, the second accelerometer vector comprising an X component along the X-axis, a Y component along the Y-axis, and a Z component along the Z-axis;

determining a roll cone about the X-axis;

determining whether the X component of the second accelerometer vector is outside of the roll cone; and

in response to determining that the X component of the second accelerometer vector is outside of the roll cone, determining that the head of the wearer is not tilted up.

23. The apparatus of claim 22, wherein determining whether the head of the wearer is tilted up based on the lookup-indication data further comprises:

in response to determining that the X component of the second accelerometer vector is inside of the roll cone;

determining a lookup-gesture-trigger cone about the Y-axis;

determining whether the Y component of the second accelerometer vector is outside of the lookup-gesture-trigger cone;

in response to determining that the Y component of the second accelerometer vector is not outside of the lookup-gesture-trigger cone, determining that the head of the wearer is not tilted up; and

in response to determining that the Y component of the second accelerometer vector is outside of the lookup-gesture-trigger cone;

determining whether the Z component of the second accelerometer vector corresponds to an upwards tilt;

in response to determining that the Z component of the second accelerometer vector does not correspond to the upwards tilt, determining that the head of the wearer is not tilted up; and

in response to determining that the Z component of the second accelerometer vector does correspond to the upwards tilt, determining that the head of the wearer is tilted up.

24. The apparatus of claim 19, wherein the lookup-indication data comprises accelerometer data from the accelerometer and gyroscope data from a gyroscope, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining an accelerometer vector based on the accelerometer data;

determining whether the head of the wearer has apparently tilted up based on the accelerometer vector;

determining whether the head of the wearer has rotated based on the rotation data;

in response to either determining that the head of the wearer has not apparently tilted up based on the accelerometer vector or determining that the head of the wearer has not rotated based on the rotation data, determining that the head of the wearer is not tilted up; and

in response to determining that the head of the wearer has apparently tilted up based on the accelerometer vector and determining that the head of the wearer has rotated based on the rotation data, determining that the head of the wearer is tilted up.

25. The apparatus of claim 19, wherein the lookup-indication data comprises first accelerometer data from the accelerometer and second accelerometer data from a second accelerometer, wherein the second accelerometer is located closer to a pivot point of the head of the wearer than the accelerometer, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

defining an X-axis, a Y-axis, and a Z-axis for the WCD;

determining a first accelerometer vector based on the first accelerometer data, the first accelerometer vector comprising a first Y-component along the Y-axis;

determining a second accelerometer vector based on the second accelerometer data, the second accelerometer vector comprising a second Y-component along the Y-axis;

determining an absolute value of an accelerometer vector difference (AVD) between the first and second accelerometer vectors,

determining whether the absolute value of the AVD exceeds an AVD threshold;

determining a Y-component difference between the first Y-component and the second Y-component;

in response to either determining that the Y-component difference is not positive or determining that the absolute value of the AVD does not exceed the AVD threshold, determining that the head of the wearer is not tilted up; and

in response to determining that the Y-component difference is positive and determining that the absolute value of the AVD does exceed the AVD threshold, determining that the head of the wearer is tilted up.

26. The apparatus of claim 19, wherein the lookup-indication data comprises accelerometer data from the accelerometer and magnetic-field data from a magnetometer, and wherein determining whether the head of the wearer is tilted up based on the lookup-indication data comprises:

determining an accelerometer vector based on the accelerometer data;

determining whether the head of the wearer has apparently tilted up based on the accelerometer vector;

verifying whether the head of the wearer has tilted up based on the magnetic-field data;

in response to either determining that the head of the wearer has not apparently tilted up based on the accelerometer vector or not verifying that the head of the wearer has tilted up based on the magnetic-field data, determining that the head of the wearer is not tilted up; and

in response to determining that the head of the wearer has apparently tilted up based on the accelerometer vector and verifying that the head of the wearer has tilted up based on the magnetic-field data, determining that the head of the wearer is tilted up.

27. The apparatus of claim 19, wherein the operations further comprise:

in response to the gesture-recognition trigger, generating a gesture-recognition interrupt.