Position and orientation determination for a mobile computing device

• US 8,253,684 B1
• Filed: 11/02/2010
• Issued: 08/28/2012
• Est. Priority Date: 11/02/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lloyd, Matthew I.
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): PATEL, PREMAL R
• Application Number:
  US12/938,043
• Time in Patent Office: 665 Days
• Field of Search: 345156-169, 345/698, 715/702, 715/863, 704/270, 455/414.2, 455/456.1, 455/550.1
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

For multiple times in a time period, multiple data points can be received from an accelerometer and from a magnetometer that are included in a mobile computing device. For each of the data points, an orientation and a position of the mobile computing device can be determined based on an acceleration output and a magnetometer output that corresponds to the particular time. A trajectory is determined that represents movement of the mobile computing device during the time period based on the determined orientations and positions of the mobile computing device at the multiple times. Information that characterizes the trajectory is compared to stored information that characterizes a set of one or more base trajectories. Based on the comparison, an operation of the mobile computing device is identified that is associated with a trajectory included in the set of one or more base trajectories.

Claims

1. A computer-implemented method comprising:

receiving for a plurality of times in a time period, a plurality of data points from an accelerometer and a plurality of data points from a magnetometer, the accelerometer and the magnetometer included in a mobile computing device;

for each of the data points, determining an orientation and a position of the mobile computing device based on an acceleration output and a magnetometer output that corresponds to the particular time;

determining a trajectory that represents movement of the mobile computing device during the time period based on the determined orientations and positions of the mobile computing device at the plurality of times;

comparing information that characterizes the trajectory to stored information that characterizes a set of one or more base trajectories;

based on the comparison, identifying an operation of the mobile computing device that is associated with a trajectory included in the set of one or more base trajectories, wherein identifying the operation of the mobile computing device comprises recognizing that a user holding the mobile computing device has made a gesture that corresponds to the trajectory and that is associated with operation of the mobile computing device, wherein the recognized gesture comprises a gesture associated with the operation of voice recording that comprises the user holding the mobile computing device to the user'"'"'s face such that a microphone on the mobile computing device is close to the user'"'"'s mouth and a speaker on the mobile computing device is away from the user'"'"'s face; and

automatically initiating a voice recording operation in response to recognizing that the user has made the gesture.

2. The method of claim 1, wherein:

each accelerometer output comprises a gravity component and a positional component; and

determining the orientation and the position of the mobile computing device comprises;

determining the orientation of the mobile computing device based on the magnetometer output;

determining the gravity component of the accelerometer output based on the determined orientation of the mobile computing device;

determining the positional component of the acceleration output based on the gravity component; and

determining the position of the mobile computing device based on the positional component of the accelerometer output.

3. The method of claim 1, further comprising receiving optical signals from the mobile computing device, wherein determining the trajectory that represents movement of the mobile computing device further comprises processing the optical signals to the camera to derive optical flow information.

4. The method of claim 1, further comprising sending a signal to the user that the computing device is in voice recording operation, wherein the signal is selected from the group consisting of an audible signal, a tactile signal, and a visual signal.

5. The method of claim 1, further comprising, after initiating voice recording:

receiving for a second plurality of times in a second time period, a second plurality of data points from the accelerometer and a second plurality of data points from the magnetometer;

for each of the second plurality of data points from the accelerometer and second plurality of data points from the magnetometer, determining a second orientation and a second position of the mobile computing device based on an acceleration output and a magnetometer output that corresponds to the particular time;

determining a second trajectory that represents movement of the mobile computing device during the second time period based on the determined orientations and positions of the mobile computing device at the second plurality of times;

comparing information that characterizes the second trajectory to the stored information that characterizes the set of one or more base trajectories;

based on the comparison, identifying a second operation of the mobile computing device that is associated with a trajectory included in the set of one or more base trajectories, wherein identifying the second operation of the mobile computing device comprises recognizing that a user holding the mobile computing device has made a second gesture that corresponds to the second trajectory and that is associated with operation of the mobile computing device, wherein the recognized second gesture comprises a gesture associated with a non-operating position; and

automatically terminating the voice recording operation in response to recognizing that the user has made the second gesture.

7. The method of claim 1, wherein automatically initiating the voice recording operation is in response to identifying both the position and orientation of the mobile computing device.

6. A computer-readable storage medium storing instructions encoded thereon, which, when executed by a processor, cause the processor to perform operations comprising:

receiving for a plurality of times in a time period, a plurality of data points from an accelerometer and a plurality of data points from a magnetometer, the accelerometer and the magnetometer included in a mobile computing device;

for each of the data points, determining an orientation and a position of the mobile computing device based on an acceleration output and a magnetometer output that corresponds to the particular time;

determining a trajectory that represents movement of the mobile computing device during the time period based on the determined orientations and positions of the mobile computing device at the plurality of times;

comparing information that characterizes the trajectory to stored information that characterizes a set of one or more base trajectories; and

based on the comparison, identifying an operation of the mobile computing device that is associated with a trajectory included in the set of one or more base trajectories, wherein identifying the operation of the mobile computing device comprises recognizing that a user holding the mobile computing device has made a gesture that corresponds to the trajectory and that is associated with the operation of the mobile computing device, wherein the recognized gesture comprises a gesture associated with the operation of voice recording that comprises the user holding the mobile computing device to the user'"'"'s face such that a microphone on the mobile computing device is close to the user'"'"'s mouth and a speaker on the mobile computing device is away from the user'"'"'s face.

13. The computer-readable storage medium of claim 6, the operations further comprising:

automatically initiating a voice recording operation in response to recognizing that the user has made the gesture.

14. The computer-readable storage medium of claim 6, wherein:

each accelerometer output comprises a gravity component and a positional component; and

determining the orientation and the position of the mobile computing device comprises;

determining the orientation of the mobile computing device based on the magnetometer output;

determining the gravity component of the accelerometer output based on the determined orientation of the mobile computing device;

determining the positional component of the acceleration output based on the gravity component; and

determining the position of the mobile computing device based on the positional component of the accelerometer output.

15. The computer-readable storage medium of claim 6, the operations further comprising:

receiving optical signals from the mobile computing device, wherein determining the trajectory that represents movement of the mobile computing device further comprises processing the optical signals to derive optical flow information.

8. A mobile computing device comprising:

an accelerometer configured to provide an accelerometer output that indicates an acceleration of the mobile computing device;

a magnetometer configured to provide a magnetometer output that indicates a direction the mobile computing device is pointing relative to magnetic North;

a processor;

a storage device coupled to the processor and configurable for storing instructions, which, when executed by the processor cause the processor to perform operations comprising;

receive for a plurality of times in a time period, a plurality of data points from the accelerometer and a plurality of data points from the magnetometer;

for each of the plurality of times in the time period, determine an orientation and a position of the mobile computing device based on the acceleration output and the magnetometer output that corresponds to the particular time;

determine a trajectory that represents movement of the mobile computing device during the time period based on the determined orientations and positions of the mobile computing device at the plurality of times in the time period;

based on a comparison of information that characterizes the trajectory to a set of one or more base trajectories, identifying the operation of the mobile computing device that is associated with the trajectory included in the set of one or more base trajectories, wherein identifying the operation of the mobile computing device comprises recognizing that a user holding the mobile computing device has made a gesture that corresponds to the trajectory and that is associated with the operation of the mobile computing device; and

based on the comparison, automatically launching the operation of the mobile computing device that is associated with a trajectory included in the set of one or more base trajectories,

wherein the recognized gesture comprises a gesture associated with the operation of voice recording that comprises the user holding the mobile computing device to the user'"'"'s face such that a microphone on the mobile computing device is close to the user'"'"'s mouth and a speaker on the mobile computing device is away from the user'"'"'s face.

9. The mobile computing device of claim 8, wherein the operations further comprise:

comparing the information characterizing the trajectory to the set of one or more base trajectories.

10. The mobile computing device of claim 8, wherein:

each accelerometer output comprises a gravity component and a positional component; and

determining the orientation and the position of the mobile computing device comprises;

determining the orientation of the mobile computing device based on the magnetometer output;

determining the gravity component of the accelerometer output based on the determined orientation of the mobile computing device;

determining the positional component of the acceleration output based on the gravity component; and

determining the position of the mobile computing device based on the positional component of the accelerometer output.

11. The mobile computing device of claim 8, wherein the operations further comprise:

sending a signal to the user that the computing device is in voice recording operation, wherein the signal is selected from the group consisting of an audible signal, a tactile signal, and a visual signal.

12. The mobile computing device of claim 8, wherein the operations further comprise:

receiving optical signals from a camera, the camera included from the mobile computing device, wherein determining the trajectory that represents movement of the mobile computing device further comprises processing the optical signals to derive optical flow information.