Using object to align and calibrate inertial navigation system

• US 8,855,929 B2
• Filed: 01/18/2010
• Issued: 10/07/2014
• Est. Priority Date: 01/18/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zhang, Gengsheng, Garin, Lionel J.
• Primary Examiner(s): Olszewski, John R
• Assistant Examiner(s): Do, Truc M
• Application Number:
  US12/689,085
• Publication Number:
  US 20110178708A1
• Time in Patent Office: 1,723 Days
• Field of Search: 701/480, 701/494, 701/500, 701/501, 701/503, 701/504, 701/505, 701/523
• US Class Current: 701/480
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/188 for accumulated errors, e.g. by coupling inertial systems with absolute positioning systems
• CPC Class Codes: G01C 25/005 initial alignment, calibration or starting-up of inertial devices
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

The initialization of an inertial navigation system is performed using information obtained from an image of an object. Positional and orientational information about the object in a global reference frame and positional and orientational information about the camera relative to the object are obtained from the image. Positional and orientational information for the camera in the global reference frame is determined along with a transformation matrix between inertial sensor reference frame and a navigation coordinate frame. The inertial navigation system is initialized using the positional and orientational information for the camera, the transformation matrix and the velocity of the camera when the object was imaged, i.e., zero. Using the initialized data with measurements from the inertial sensors the position of the mobile platform may be updated during navigation and provided, e.g., on a digital map. Inertial navigation errors may be corrected using information obtained from images of different objects.

Claims

1. A method comprising:

capturing an image of an object with a camera in a mobile platform while stationary, the mobile platform having an inertial navigation system including inertial sensors, wherein an orientation between the inertial sensors and the camera is known;

obtaining position and orientation information of the object in a global reference frame;

estimating a position and orientation of the camera relative to the object from the image of the object;

determining a camera position and orientation in the global reference frame from the estimated position and orientation of the camera relative to the object and the obtained position and orientation of the object in the global reference frame;

determining with a processor a transformation matrix between a navigation coordinate frame and an inertial sensor reference frame for the inertial navigation system using the determined camera position and orientation in the global reference frame and the orientation between the inertial sensors and the camera, wherein determining the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame comprises using a determined first transformation matrix between a camera reference frame and an object reference frame and a second transformation matrix between the global reference frame and the object reference frame;

setting a velocity of the inertial navigation system to zero;

setting the position of the inertial navigation system to the estimated position of the camera in the global reference frame;

using measurements from the inertial sensors, the determined transformation matrix, the set velocity and the set position of the inertial navigation system to update the inertial navigation system position, velocity, and orientation; and

providing the updated estimate of the position of the inertial navigation system as a position of the mobile platform.

2. The method of claim 1, further comprising:

capturing an image of a different object with the camera in the mobile platform while stationary;

obtaining position and orientation information of the different object in the global reference frame;

estimating a new position and orientation of the camera relative to the different object from the image of the different object;

determining a new camera position and orientation in the global reference frame from the estimated new position and orientation of the camera relative to the object and the obtained position and orientation of the different object in the global reference frame; and

using at least one of the determined new camera position and/or orientation in the global reference frame to estimate and correct errors in the position, velocity and orientation of the inertial navigation system, and the inertial sensors.

3. The method of claim 2, wherein the errors in the position, velocity, and orientation of the inertial navigation system, and the inertial sensors are estimated and corrected using a Kalman filter.

4. The method of claim 1, wherein determining the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame further comprises using a third transformation matrix between the camera reference frame and the inertial sensor reference frame.

5. The method of claim 4, wherein determining the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame further comprises using a fourth transformation matrix between the global reference frame and the navigation coordinate frame.

6. The method of claim 1, wherein the object is a data code label.

7. The method of claim 1, wherein the object is a Quick Response code.

8. The method of claim 1, wherein the object is encoded with a Uniform Resource Identifier (URI) and wherein obtaining position and orientation information about the object comprises:

decoding the object to determine the URI; and

accessing and downloading the position and orientation information about the object using the URI.

9. The method of claim 1, wherein providing the updated estimate of the position of the inertial navigation system as a position of the mobile platform comprises displaying the updated estimate of the position on a digital map on the mobile platform.

10. A mobile platform comprising:

a camera being operable to image an object;

an inertial navigation system comprising inertial sensors that provides motion and rotation data;

a processor connected to the camera and the inertial sensors;

memory connected to the processor; and

software held in the memory and run in the processor to cause the processor to obtain position and orientation information about the object in a global reference frame, estimate a position and orientation of the camera relative to the object from an image of the object taken by the camera;

determine a camera position and orientation in the global reference frame from the estimated position and orientation and the obtained position and orientation of the object in the global reference frame;

determine a transformation matrix between a navigation coordinate frame and an inertial sensor reference frame for the inertial navigation system using the determined camera position and orientation in the global reference frame and a known orientation between the inertial sensors and the camera, wherein the software causes the processor to determine a first transformation matrix between a camera reference frame and an object reference frame and a second transformation matrix between the global reference frame and the object reference frame to determine the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame;

initializing the inertial navigation system with a velocity of zero, the determined position of the camera in the global reference frame and the determined transformation matrix between the navigation coordinate frame and the inertial sensor reference frame;

using data from the inertial sensors and the initialized velocity, position and transformation matrix to update the inertial navigation system position;

providing the updated inertial navigation system position as a position of the mobile platform; and

wherein the software causes the processor to determine a first transformation matrix between a camera reference frame and an object reference frame and a second transformation matrix between the global reference frame and the object reference frame to determine the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame.

11. The mobile platform of claim 10, wherein the software causes the processor to further cause the processor to obtain position and orientation information about a different imaged object in the global reference frame, estimate a new position and orientation of the camera relative to the different imaged object from an image taken by the camera;

determine a new camera position and orientation in the global reference frame from the estimated new position and orientation and the obtained position and orientation of the different imaged object in the global reference frame; and

use at least one of the determined new camera position and orientation in the global reference frame to estimate and correct errors in the position, velocity, and orientation of the inertial navigation system and the inertial sensors.

12. The mobile platform of claim 10, further comprising a Kalman filter, wherein the Kalman filter uses the at least one of the determined new camera position and orientation in the global reference frame to estimate and correct errors in the position, velocity and orientation of the inertial navigation system and the inertial sensors.

13. The mobile platform of claim 10, wherein the software further causes the processor to use a third transformation matrix between the camera reference frame and the inertial sensor reference frame to determine the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame.

14. The mobile platform of claim 13, wherein the software further causes the processor to use a fourth transformation matrix between the global reference frame and the navigation coordinate frame to determine the transformation matrix between the navigation coordinate frame and the inertial sensor reference frame.

15. The mobile platform of claim 10, further comprising a display, wherein the software held in the memory and run in the processor controls the processor to display the position on the mobile platform.

16. The mobile platform of claim 10, further comprising an object decoder to decode the object, the object decoder being coupled to the processor, wherein processor obtains position and orientation information about the object in a global reference frame using a Uniform Resource Identifier (URI) that is encoded in the object.

17. A system for accessing and updating positional information for a mobile platform comprising:

a means for imaging an object;

means for obtaining position and orientation information about the object in a global reference frame;

means for estimating a position and orientation of the means for imaging relative to the object from the image of the object;

means for determining a position and orientation of the means for imaging in the global reference frame from the estimated position and orientation and the obtained position and orientation of the object;

means for sensing movement of the mobile platform;

means for determining a transformation matrix between a navigation coordinate frame and a frame for the means for sensing movement using the determined position and orientation of the means for imaging and a known orientation between the frame for the means for sensing movement and the means for imaging, wherein the means for determining the transformation matrix uses a first transformation matrix between a frame for the means for imaging and an object reference frame and a second transformation matrix between the global reference frame and the object reference frame;

means for initializing an inertial navigation system with a velocity of zero, the determined position of the means for imaging and the determined transformation matrix between the navigation coordinate frame and the frame for the means for sensing movement;

means for determining a position, velocity, and orientation of the mobile platform using the data from the means for sensing movement, and the initialized velocity, position, and transformation matrix; and

means for providing the determined position of the mobile platform.

18. The system of claim 17, wherein the means for imaging images a different object;

the means for obtaining position and orientation information obtains position and orientation information about the different object in the global reference frame;

the means for estimating a position and orientation of the means for imaging estimates a new position and orientation of the means for imaging;

the means for determining a position and orientation of the means for imaging in the global reference frame determines a new position and orientation of the means for imaging in the global reference frame from the estimated new position and orientation and the obtained position and orientation of the different object, the system further comprising;

means for correcting errors in the position, velocity and orientation of the mobile platform using at least one of the determined new position and orientation of the means for imaging.

19. The system of claim 17, wherein the means for determining the transformation matrix further uses a third transformation matrix between the frame for the means for imaging and the frame for the means for sensing movement.

20. The system of claim 19, wherein the means for determining the transformation matrix further uses a fourth transformation matrix between the navigation coordinate frame and the global reference frame.

21. The system of claim 17, wherein means for providing the determined position of the mobile platform comprises a means for displaying the determined position on a digital map.

22. The system of claim 17, wherein the object is a data code label and the system further comprising a data code label reader.

23. The system of claim 17, wherein the means for obtaining position and orientation information about the object in a global reference frame is a web browser in the mobile platform.

24. A non-transitory computer-readable medium including program code stored thereon, comprising:

program code to decode and obtain position and orientation information in a global reference frame about an object that is imaged;

program code to estimate a position and orientation from an image of the object;

program code to determine a position and orientation in the global reference frame from the estimated position and orientation and the obtained position and orientation information in the global reference frame about the imaged object;

program code to determine a transformation matrix between a navigation coordinate frame and a frame for inertial sensors of an inertial navigation system using the determined position and orientation in the global reference frame and a known orientation between the frame for the inertial sensors and a frame for a camera that produced the image of the object, wherein the program code to determine the transformation matrix uses a first transformation matrix between a frame for the means for imaging and an object reference frame and a second transformation matrix between the global reference frame and the object reference frame;

program code to initialize the inertial navigation system using the determined position and orientation, the determined transformation matrix and a velocity of zero;

program code to determine an updated position using the initialized position, transformation matrix, velocity and data from the inertial sensors; and

program code to provide the updated position.

25. The non-transitory computer-readable medium including program code stored thereon of claim 24, further comprising:

program code to correct errors in position, velocity and orientation of the inertial navigation system and the inertial sensors using at least one of the determined position and a new orientation determined from an image of a different object.