Alignment of visual inertial odometry and satellite positioning system reference frames

• US 10,132,933 B2
• Filed: 02/02/2016
• Issued: 11/20/2018
• Est. Priority Date: 02/02/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Nallampatti Ekambaram, Venkatesan, Jose, Jubin, Niesen, Urs, Wu, Xinzhou, Garin, Lionel Jacques
• Primary Examiner(s): Issing, Gregory C.
• Application Number:
  US15/014,004
• Publication Number:
  US 20170219717A1
• Time in Patent Office: 1,022 Days
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 19/42 Determining position
• CPC Class Codes: G01S 19/47 the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
• CPC Class Codes: G01S 19/485 whereby the further system is an optical system or imaging system
• CPC Class Codes: G01S 19/52 Determining velocity
• CPC Class Codes: G01S 19/53 Determining attitude
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49

Abstract

A method for aligning visual-inertial odometry (VIO) and satellite positioning system (SPS) reference frames includes obtaining a plurality of range-rate measurements of a mobile platform from an SPS. The range-rate measurements are with respect to a global reference frame of the SPS. The method also includes obtaining a plurality of VIO velocity measurements of the mobile platform from a VIO system. The VIO velocity measurements are with respect to a local reference frame of the VIO system. At least one orientation parameter is then determined to align the local reference frame with the global reference frame based on the range-rate measurements and the VIO velocity measurements.

Claims

1. A method, comprising:

obtaining a plurality of satellite positioning system (SPS) measurements of a mobile platform from an SPS,

wherein the plurality of SPS measurements are with respect to a global reference frame of the SPS,

wherein the plurality of SPS measurements correspond to a plurality of time epochs, and

wherein the plurality of SPS measurements include one or both of one or more SPS velocity measurements of the mobile platform, and a plurality of range-rate measurements of the mobile platform;

obtaining a plurality of visual-inertial odometry (VIO) velocity measurements of the mobile platform from a VIO system,

wherein the plurality of VIO velocity measurements are with respect to a local reference frame of the VIO system, and

wherein the plurality of VIO velocity measurements correspond to the plurality of time epochs;

determining at least one orientation parameter for at least one time epoch to align the local reference frame with the global reference frame based on the plurality of SPS measurements and the plurality of VIO velocity measurements; and

determining a position of the mobile platform for the at least one time epoch based on a combination of the plurality of SPS measurements and the plurality of VIO velocity measurements.

2. The method of claim 1, further comprising:

determining the one or more SPS velocity measurements of the mobile platform based on the plurality of range-rate measurements,

wherein the at least one orientation parameter for the at least one time epoch is determined based on the one or more SPS velocity measurements and the plurality of VIO velocity measurements, and

wherein the position of the mobile platform for the at least one time epoch is determined based on a combination of the one or more SPS velocity measurements and the plurality of VIO velocity measurements.

3. The method of claim 1, wherein when the plurality of range-rate measurements comprises a number of range-rate measurements that is less than required to calculate an SPS velocity of the mobile platform for the at least one time epoch,

the at least one orientation parameter for the at least one time epoch is determined based on the plurality of range-rate measurements and the plurality of VIO velocity measurements, and

the position of the mobile platform for the at least one time epoch is determined based on a combination of the plurality of range-rate measurements and the plurality of VIO velocity measurements.

4. The method of claim 1, wherein,

obtaining the plurality of SPS measurements of the mobile platform from the SPS comprises obtaining the plurality of SPS measurements over a sliding window of time comprising a plurality of consecutive time epochs, and

obtaining the plurality of VIO velocity measurements from the VIO system comprises obtaining the plurality of VIO velocity measurements over the sliding window of time,

the method further comprising:

continuously determining the at least one orientation parameter based on the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time; and

continuously determining the position of the mobile platform based on the combination of the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time.

5. The method of claim 4, further comprising:

adjusting a size of the sliding window of time based on a quality of the plurality of range-rate measurements.

6. The method of claim 5, further comprising:

determining the quality of the plurality of range-rate measurements based on a magnitude of movement of the mobile platform.

7. The method of claim 1, further comprising:

filtering the plurality of range-rate measurements based on a quality of the plurality of range-rate measurements.

8. The method of claim 7, wherein filtering the plurality of range-rate measurements comprises discarding one or more range-rate measurements from the plurality of range-rate measurements based on a quality of the one or more range-rate measurements.

9. The method of claim 7, wherein filtering the plurality of range-rate measurements comprises weighting one or more range-rate measurements of the plurality of range-rate measurements based on a quality of the one or more range-rate measurements.

10. The method of claim 1, further comprising:

discarding one or more non-line-of-sight range-rate measurements from the plurality of range-rate measurements.

11. The method of claim 1, wherein determining the at least one orientation parameter to align the local reference frame with the global reference frame comprises:

determining a rotation matrix between the local reference frame and the global reference frame; and

translating the plurality of VIO velocity measurements into the global reference frame based on the rotation matrix.

31. The method of claim 1, wherein the position of the mobile platform is determined based on the combination of the plurality of SPS measurements and the plurality of VIO velocity measurements at each time epoch.

12. An apparatus, comprising:

means for obtaining a plurality of satellite positioning system (SPS) measurements of a mobile platform from an SPS,

wherein the plurality of SPS measurements are with respect to a global reference frame of the SPS,

wherein the plurality of SPS measurements correspond to a plurality of time epochs, and

wherein the plurality of SPS measurements include one or both of one or more SPS velocity measurements of the mobile platform, and a plurality of range-rate measurements of the mobile platform;

means for obtaining a plurality of visual-inertial odometry (VIO) velocity measurements of the mobile platform from a VIO system,

wherein the plurality of VIO velocity measurements are with respect to a local reference frame of the VIO system, and

wherein the plurality of VIO velocity measurements correspond to the plurality of time epochs;

means for determining at least one orientation parameter for at least one time epoch to align the local reference frame with the global reference frame based on the plurality of SPS measurements and the plurality of VIO velocity measurements; and

means for determining a position of the mobile platform for the at least one time epoch based on a combination of the plurality of SPS measurements and the plurality of VIO velocity measurements.

13. The apparatus of claim 12, further comprising:

means for determining the one or more SPS velocity measurements of the mobile platform based on the plurality of range-rate measurements,

wherein the means for determining the at least one orientation parameter determines the at least one orientation parameter for the at least one epoch based on the one or more SPS velocity measurements and the plurality of VIO velocity measurements, and

wherein the means for determining the position of the mobile platform determines the position of the mobile platform for the at least one time epoch based on a combination of the one or more SPS velocity measurements and the plurality of VIO velocity measurements.

14. The apparatus of claim 12, wherein when the plurality of range-rate measurements comprises a number of range-rate measurements that is less than required to calculate a SPS velocity of the mobile platform for the at least one time epoch,

the means for determining the at least one orientation parameter determines the at least one orientation parameter for the at least one time epoch based on the plurality of range-rate measurements and the plurality of VIO velocity measurements, and

the means for determining the position of the mobile platform determines the position of the mobile platform for the at least one time epoch based on a combination of the plurality of range-rate measurements and the plurality of VIO velocity measurements.

15. The apparatus of claim 12, wherein,

the means for obtaining the plurality of SPS measurements of the mobile platform from the SPS obtains the plurality of range-rate measurements over a sliding window of time comprising a plurality of consecutive time epochs, and

the means for obtaining the plurality of VIO velocity measurements from the VIO system obtains the plurality of VIO velocity measurements over the sliding window of time,

means for determining the at least one orientation parameter continuously determines the at least one orientation parameter based on the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time, and

the means for determining the position of the mobile platform continuously determines the position of the mobile platform based on the combination of the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time.

16. The apparatus of claim 15, further comprising means for adjusting a size of the sliding window of time based on a quality of the plurality of range-rate measurements.

17. The apparatus of claim 12, wherein the means for determining the at least one orientation parameter to align the local reference frame with the global reference frame comprises:

means for determining a rotation matrix between the local reference frame and the global reference frame; and

means for translating the plurality of VIO velocity measurements into the global reference frame based on the rotation matrix.

32. The apparatus of claim 12, wherein the means for determining the position of the mobile platform determines the position of the mobile platform based on the combination of the one or more SPS velocity measurements and the plurality of VIO velocity measurements at each time epoch.

18. An apparatus, comprising:

at least one processor; and

at least one memory coupled to the at least one processor, the at least one processor and the at least one memory being configured to direct the apparatus to;

obtain a plurality of satellite positioning system (SPS) measurements of a mobile platform from an SPS,

wherein the plurality of SPS measurements are with respect to a global reference frame of the SPS,

wherein the plurality of SPS measurements correspond to a plurality of time epochs, and

wherein the plurality of SPS measurements include one or both of one or more SPS velocity measurements of the mobile platform, and a plurality of range-rate measurements of the mobile platform;

obtain a plurality of visual-inertial odometry (VIO) velocity measurements of the mobile platform from a VIO system,

wherein the plurality of VIO velocity measurements are with respect to a local reference frame of the VIO system, and

wherein the plurality of VIO velocity measurements correspond to the plurality of time epochs;

determine at least one orientation parameter for at least one time epoch to align the local reference frame with the global reference frame based on the plurality of SPS measurements and the plurality of VIO velocity measurements; and

determine a position of the mobile platform for the at least one time epoch based on a combination of the plurality of SPS measurements and the plurality of VIO velocity measurements.

19. The apparatus of claim 18, wherein the at least one processor and the at least one memory are further configured to direct the apparatus to:

determine the one or more SPS velocity measurements of the mobile platform based on the plurality of range-rate measurements,

wherein determining the at least one orientation parameter for the at least one time epoch is based on the one or more SPS velocity measurements and the plurality of VIO velocity measurements, and

wherein determining the position of the mobile platform for the at least one time epoch is based on a combination of the one or more SPS measurements and the plurality of VIO velocity measurements.

20. The apparatus of claim 18, wherein when the plurality of range-rate measurements comprises a number of range-rate measurements that is less than required to calculate an SPS velocity of the mobile platform for the at least one time epoch,

determining the at least one orientation parameter for the at least one time epoch is based on the plurality of range-rate measurements and the plurality of VIO velocity measurements, and

determining the position of the mobile platform for the at least one time epoch is based on a combination of the plurality of range-rate measurements and the plurality of VIO velocity measurements.

21. The apparatus of claim 18, wherein the at least one processor and the at least one memory are further configured to direct the apparatus to:

obtain the plurality of SPS measurements over a sliding window of time comprising a plurality of consecutive time epochs;

obtain the plurality of VIO velocity measurements over the sliding window of time;

continuously determine the at least one orientation parameter based on the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time; and

continuously determine the position of the mobile platform based on the combination of the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time.

22. The apparatus of claim 21, wherein the at least one processor and the at least one memory are further configured to direct the apparatus to adjust a size of the sliding window of time based on a quality of the plurality of range-rate measurements.

23. The apparatus of claim 18, wherein the at least one processor and the at least one memory are further configured to direct the apparatus to filter the plurality of range-rate measurements based on a quality of the plurality of range-rate measurements.

24. The apparatus of claim 18, wherein the at least one processor and the at least one memory are further configured to direct the apparatus to:

determine a rotation matrix between the local reference frame and the global reference frame; and

translate the plurality of VIO velocity measurements into the global reference frame based on the rotation matrix.

33. The apparatus of claim 18, wherein the at least one processor and the at least one memory are further configured to direct the apparatus to determine the position of the mobile platform based on the combination of the one or more SPS measurements and the plurality of VIO velocity measurements at each time epoch.

25. A non-transitory computer-readable storage medium having computer-executable instructions recorded thereon, wherein executing the computer-executable instructions on one or more processors causes the one or more processors to:

obtain a plurality of satellite positioning system (SPS) measurements of a mobile platform from an SPS,

wherein the plurality of SPS measurements are with respect to a global reference frame of the SPS,

wherein the plurality of SPS measurements correspond to a plurality of time epochs, and

wherein the plurality of SPS measurements include one or both of one or more SPS velocity measurements of the mobile platform, and a plurality of range-rate measurements of the mobile platform;

obtain a plurality of visual-inertial odometry (VIO) velocity measurements of the mobile platform from a VIO system,

wherein the plurality of VIO velocity measurements are with respect to a local reference frame of the VIO system, and

wherein the plurality of VIO velocity measurements correspond to the plurality of time epochs;

determine at least one orientation parameter for at least one time epoch to align the local reference frame with the global reference frame based on the plurality of SPS measurements and the plurality of VIO velocity measurements; and

determine a position of the mobile platform for the at least one time epoch based on a combination of the plurality of SPS measurements and the plurality of VIO velocity measurements.

26. The non-transitory computer-readable storage medium of claim 25, wherein executing the computer-executable instructions on the one or more processors causes the one or more processors to:

determine one or more SPS velocity measurements of the mobile platform based on the plurality of range-rate measurements,

wherein determining the at least one orientation parameter for the at least one time epoch is based on the one or more SPS velocity measurements and the plurality of VIO velocity measurements, and

wherein determining the position of the mobile platform for the at least one time epoch is based on a combination of the one or more SPS velocity measurements and the plurality of VIO velocity measurements.

27. The non-transitory computer-readable storage medium of claim 25, wherein when the plurality of range-rate measurements comprises a number of range-rate measurements that is less than required to calculate an SPS velocity of the mobile platform for the at least one time epoch, executing the computer-executable instructions on the one or more processors causes the one or more processors to:

determine the at least one orientation parameter for the at least one epoch time based on the plurality of range-rate measurements and the plurality of VIO velocity measurements, and

determine the position of the mobile platform for the at least one time epoch based on a combination of the plurality of range-rate measurements and the plurality of VIO velocity measurements.

28. The non-transitory computer-readable storage medium of claim 25, wherein executing the computer-executable instructions on the one or more processors causes the one or more processors to:

obtain the plurality of SPS measurements over a sliding window of time comprising a plurality of consecutive time epochs;

obtain the plurality of VIO velocity measurements over the sliding window of time;

continuously determine the at least one orientation parameter based on the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time; and

continuously determine the position of the mobile platform based on the combination of the plurality of SPS measurements and the plurality of VIO velocity measurements obtained over the sliding window of time.

29. The non-transitory computer-readable storage medium of claim 28, wherein executing the computer-executable instructions on the one or more processors causes the one or more processors to adjust a size of the sliding window of time based on a quality of the plurality of range-rate measurements.

30. The non-transitory computer-readable storage medium of claim 25, wherein executing the computer-executable instructions on the one or more processors causes the one or more processors to:

determine a rotation matrix between the local reference frame and the global reference frame; and

translate the plurality of VIO velocity measurements into the global reference frame based on the rotation matrix.

34. The non-transitory computer-readable storage medium of claim 25, wherein executing the computer-executable instructions on the one or more processors causes the one or more processors to determine the position of the mobile platform based on the combination of the one or more SPS velocity measurements and the plurality of VIO velocity measurements at each time epoch.