Calibrating Intertial Sensors Using an Image Sensor

• US 20170089948A1
• Filed: 06/24/2016
• Published: 03/30/2017
• Est. Priority Date: 11/04/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Rhodes, Charles Campbell, Ho, Harvey
• Granted Patent: US 9,678,102 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G01P 21/00 Testing or calibrating of apparatus or devices covered by the preceding groups

Abstract

A method is provided for calibrating an inertial sensing unit of a device utilizing a vision sensing unit integral to the device. The method includes receiving inertial sensing input data from the inertial sensing unit, receiving vision sensing input data from the vision unit, and determining when the received vision sensing input data represents a predetermined input state of the vision sensing unit. The method includes estimating an error value in the inertial sensing input data received from the inertial sensing unit based on the received vision sensing input data upon determination that the received vision sensing input data represents the predetermined vision sensing input state. The method further includes adjusting first subsequent received inertial sensing input data from the inertial sensing unit based on the estimated error value, thereby calibrating the inertial sensing unit.

Claims

1. A method comprising:

during a session in which a first application is open on a wearable device;

receiving inertial data indicating movement of the wearable device from an inertial sensing unit integral to the wearable device;

receiving input image data from an image sensing unit integral to the wearable device;

determining when the received input image data represents a predetermined input state of the image sensing unit;

upon determination that the received input image data represents the predetermined image sensing input state;

(a) comparing, at the wearable device, the inertial sensing input data and the input image data; and

(b) estimating, at the wearable device, an error value in the inertial sensing input data based on the comparison of the inertial sensing input data and the input image data;

determining an inertial-data adjustment for subsequently received inertial data from the inertial sensing unit based on the estimated error value; and

storing the inertial-data adjustment in a memory of the wearable device;

ending the session of the first application, wherein the inertial-data adjustment in the memory of the wearable device after ending the first session;

subsequently initiating a session of a second application on the wearable device, wherein the second application utilizes inertial data; and

in response to initiating the session of the second application that utilizes inertial data, retrieving the inertial-data adjustment from memory and applying the inertial-data adjustment to at least some inertial data generated by the inertial sensing unit during the session of the second application.

2. The method of claim 1, wherein determining when the received input image data represents the predetermined image sensing input state comprises:

determining when the predetermined image sensing input state corresponds to a predetermined input data threshold.

3. The method of claim 2, wherein the predetermined input data threshold is indicative of a substantial stillness of the input image data.

4. The method of claim 2, wherein the predetermined input data threshold is indicative of a predetermined movement threshold of the device.

5. The method of claim 1, further comprising:

receiving a second subsequent inertial sensing input data from the inertial sensing unit;

adjusting the received second subsequent inertial sensing input data based on the estimated error value;

receiving first subsequent input image data; and

determining a subsequent estimated error value of the adjusted second subsequent inertial sensing input data based on the first subsequent input image data.

6. The method of claim 1, wherein the inertial sensing unit is one of a gyroscope, an accelerometer, and a magnetometer.

7. The method of claim 1, wherein the computing system comprises:

a memory unit storing program instructions executable to perform the method; and

a processor unit operatively coupled to the memory unit, the inertial sensing unit, and the image sensing unit, and configured to execute the method.

8. The method of claim 1, wherein:

the inertial sensing unit is operable to provide the inertial sensing input data, which represents sensed inertial movements of the wearable device; and

the image sensing unit is operable to provide the input image data, which represents captured still images or video images.

9. The method of claim 1, wherein the device is a head-wearable display device, and includes a display element for displaying graphical images, and an image presenting unit for presenting the graphical images on the display element.

10. A method comprising:

determining that no application running on a wearable device is utilizing data generated by an inertial sensing unit integral to the wearable device;

in response to determining that no application is utilizing data generated by the inertial sensing unit;

receiving input image data from an image sensing unit integral to the wearable device, and determining when the received input image data represents a predetermined input state of the image sensing unit;

operating the inertial sensing unit to receive inertial data indicating movement of the wearable device;

comparing, at the wearable device, the inertial data and the input image data, and estimating, at the wearable device, an error value in the inertial data based on the comparison of the inertial data and the received input image data; and

adjusting first subsequently received inertial sensing input data from the inertial sensing unit based on the estimated error value, thereby reducing accumulated error of the inertial sensing unit.

11. The system of claim 10, wherein determining when the received input image data represents the predetermined image sensing input state comprises:

determining when the predetermined image sensing input state corresponds to one of a predetermined input data threshold.

12. The system of claim 10, wherein the predetermined input data threshold is indicative of a substantial stillness of the input image data.

13. The system of claim 10, wherein the predetermined input data threshold is indicative of a predetermined movement threshold of the device.

14. The system of claim 10, wherein the inertial sensing unit is one of a gyroscope, an accelerometer, and a magnetometer.

15. The system of claim 10, wherein the computing system comprises:

a memory unit for storing program instructions executable to perform the method; and

a processor unit operatively coupled to the memory unit, the inertial sensing unit, and the image vision sensing unit, and configured to execute the method.

16. The system of claim 10, wherein:

the inertial sensing unit is operable to provide the inertial sensing input data, which represents sensed inertial movements of the device; and

the image sensing unit is operable to provide the input image data, which represents captured still images or video images.

17. The system of claim 10, wherein the device is a head-wearable display device, and includes a display element for displaying graphical images, and an image presenting unit for presenting the graphical images on the display element.

18. A non-transitory computer readable storage medium containing program instructions that cause a computing system to perform a method comprising:

determining that no application running on a wearable device is utilizing data generated by an inertial sensing unit integral to the wearable device;

in response to determining that no application is utilizing data generated by the inertial sensing unit;

receiving input image data from an image sensing unit integral to the wearable device, and determining when the received input image data represents a predetermined input state of the image sensing unit;

operating the inertial sensing unit to receive inertial data indicating movement of the wearable device;

comparing, at the wearable device, the inertial data and the input image data, and estimating, at the wearable device, an error value in the inertial data based on the comparison of the inertial data and the received input image data; and

adjusting first subsequently received inertial sensing input data from the inertial sensing unit based on the estimated error value, thereby reducing accumulated error of the inertial sensing unit.