Use of comparative sensor data to determine orientation of head relative to body

• US 9,557,152 B2
• Filed: 02/15/2016
• Issued: 01/31/2017
• Est. Priority Date: 09/28/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Starner, Thad Eugene, Johnson, Michael Patrick
• Primary Examiner(s): Ghebretinsae, Temesghen
• Assistant Examiner(s): Jansen, II, Michael J
• Application Number:
  US15/044,053
• Publication Number:
  US 20160161240A1
• Time in Patent Office: 351 Days
• Field of Search: 345/156
• US Class Current: 1/1
• CPC Class Codes: G01B 7/00 Measuring arrangements characterised by the use of electric or magnetic techniques
• CPC Class Codes: G01C 17/02 Magnetic compasses
• CPC Class Codes: G01C 19/00 Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects
• CPC Class Codes: G01P 15/00 Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0179 Display position adjusting means not related to the information to be displayed
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface

Abstract

Methods and systems are described that involve a wearable computing device or an associated device determining the orientation of a person'"'"'s head relative to their body. To do so, example methods and systems may compare sensor data from the wearable computing device to corresponding sensor data from a tracking device that is expected to move in a manner that follows the wearer'"'"'s body, such a mobile phone that is located in the wearable computing device'"'"'s wearer'"'"'s pocket.

Claims

1. A computer-implemented method comprising:

detecting, by a computing device, sensor data that is indicative of an association between movement of a tracking device and body movement;

in response to detecting the sensor data that is indicative of the association;

determining a forward-backward body axis of a body corresponding to a wearable computing device; and

determining a base orientation of a tracking device relative to the forward-backward body axis (θ

_TD-B);

determining a first orientation of the wearable computing device relative to the tracking device (θ

_{HMD-TD_1}); and

determining a first head orientation relative to the body (θ

_{H-B_1}) based on both;

the base orientation (θ

_TD-B) and the first orientation (θ

_{HMD-TD_1}), wherein determining the first head orientation relative to the body (θ

_{H-B_1}) comprises offsetting the first orientation (θ

_{HMD-TD_1}) by the base orientation of the tracking device relative to the forward-backward body axis (θ

_TD-B).

2. The method of claim 1, wherein the computing device is the wearable computing device.

3. The method of claim 1, wherein the tracking device is a mobile phone.

4. The method of claim 1, wherein detecting the sensor data that is indicative of the association between movement of the tracking device and body movement comprises:

receiving sensor data from at least one of the wearable computing device and the tracking device, wherein the sensor data is indicative of movement; and

determining that the sensor data is characteristic of movement along the forward-backward body axis.

5. The method of claim 4, wherein determining that the sensor data is characteristic of movement along the forward-backward body axis comprises determining that the sensor data is characteristic of walking.

6. The method of claim 4, wherein the sensor data comprises at least one of:

(a) gyroscope data associated with the wearable computing device, (b) accelerometer data associated with the wearable computing device, (c) gyroscope data associated with the tracking device, and (d) accelerometer data associated with the tracking device.

7. The method of claim 1, wherein determining a forward-backward body axis of a body comprises determining a direction of forward body movement.

8. The method of claim 1, wherein determining a base orientation of the tracking device relative to the forward-backward body axis comprises determining an angle between a forward-facing direction of the tracking device and the directional component along the forward-backward body axis.

9. The method of claim 1, wherein the first orientation of the wearable computing device relative to the tracking device is determined based on both:

(a) magnetometer data associated with the wearable computing device and (b) magnetometer data associated with the tracking device.

10. The method of claim 9, wherein determining the first orientation of the wearable computing device relative to the tracking device comprises:

determining a first orientation of the wearable computing device relative to magnetic north based on the magnetometer data associated with the wearable computing device;

determining a first orientation of the tracking device relative to magnetic north based on the magnetometer data associated with the tracking device; and

determining the first orientation of the wearable computing device relative to the tracking device based on a difference between (a) the first orientation of the wearable computing device relative to magnetic north and (b) the first orientation of the tracking device relative to magnetic north.

11. The method of claim 1, further comprising initiating a computing action based on the first head orientation relative to the body.

12. The method of claim 1, wherein the determined first head orientation relative to the body comprises determining indicates a rotation of the head relative to the forward-backward body axis.

13. The method of claim 1, wherein the determined first head orientation relative to the body comprises two or more of:

(a) a rotation of the head relative to the forward-backward body axis, (b) a pitch of the head relative to an upward-downward body axis and (c) a yaw of the head relative to the forward-backward body axis and the upward-downward body axis.

14. A non-transitory computer readable medium having stored therein instructions that are executable to cause a computing device to perform functions comprising:

detecting sensor data that is indicative of an association between movement of a tracking device and body movement;

in response to detecting the sensor data that is indicative of the positional association;

determining a forward-backward body axis of a body associated with a wearable computing device; and

determining a base orientation of a tracking device relative to the forward-backward body axis (θ

_TD-B);

determining a first orientation of the wearable computing device relative to the tracking device (θ

_{HMD-TD_1}); and

determining a first head orientation relative to the body (θ

_{H-B_1}) based on both;

the base orientation (θ

_TD-B) and the first orientation (θ

_{HMD-TD_1}), wherein determining the first head orientation relative to the body (θ

_{H-B_1}) comprises offsetting the first orientation (θ

_{HMD-TD_1}) by the base orientation of the tracking device relative to the forward-backward body axis (θ

_TD-B).

15. A computing system comprising:

a non-transitory computer readable medium;

program instructions stored on the non-transitory computer readable medium and executable by at least one processor to;

detect sensor data that is indicative of an association between movement of a tracking device and body movement;

in response to detecting the sensor data that is indicative of the positional association;

determine a forward-backward body axis of a body associated with a wearable computing device; and

determine a base orientation of a tracking device relative to the forward-backward body axis (θ

_TD-B);

determine a first orientation of a wearable computing device relative to the tracking device (θ

_{HMD-TD_1}); and

determine a first head orientation relative to the body (θ

_{H-B_1}) based on both;

the base orientation (θ

_TD-B) and the first orientation (θ

_{HMD-TD_1}), wherein determining the first head orientation relative to the body (θ

_{H-B_1}) comprises offsetting the first orientation (θ

_{HMD-TD_1}) by the base orientation of the tracking device relative to the forward-backward body axis (θ

_TD-B).

16. The computing system of claim 15, wherein the computing system is implemented in or takes the form of the wearable computing device.

17. The computing system of claim 15, wherein the first orientation of the wearable computing device relative to the tracking device is determined based on both:

(a) magnetometer data associated with the wearable computing device and (b) magnetometer data associated with the tracking device.

18. The computing system of claim 15, wherein the determined first head orientation relative to the body comprises two or more of:

(a) a rotation of the head relative to the forward-backward body axis, (b) a pitch of the head relative to an upward-downward body axis and (c) a yaw of the head relative to the forward-backward body axis and the upward-downward body axis.