Iterative Kalman Smoother for robust 3D localization for vision-aided inertial navigation
First Claim
Patent Images
1. A vision-aided inertial navigation system comprising:
- at least one image source to produce image data along a trajectory of the vision-aided inertial navigation system (VINS) within an environment, wherein the image data contains a plurality of features observed within the environment at a plurality of poses of the VINS along the trajectory;
an inertial measurement unit (IMU) to produce IMU data indicative of motion of the vision-aided inertial navigation system; and
a hardware-based processing unit comprising an estimator that computes, based on the image data and the IMU data, a sliding window of state estimates for at least a position and orientation of the vision-aided inertial navigation system for a plurality of poses of the VINS along the trajectory and respective covariances for each of the state estimates, each of the respective covariances representing an amount of uncertainty in the corresponding state estimate, andwherein the estimator computes the state estimates by;
classifying the visual features observed at the poses of the VINS within the sliding window into at least a first set of the features and a second set of features as a function of a position within the sliding window for the respective pose from which the respective feature was observed, the second set of features being associated with one or more older poses than the first set of features within the sliding window,applying an extended Kalman filter to update, within the sliding window, each of the state estimates for the VINS and the features using the IMU data and the image data obtained associated with both the first set of features and the second set of features observed from the plurality of poses within the sliding window, andupdating, for each of the state estimates, the respective covariance using the IMU data and the image data associated with the second set of features without using the image data associated with the first set of features.
2 Assignments
0 Petitions
Accused Products
Abstract
A vision-aided inertial navigation system (VINS) is described in which a filter-based sliding-window estimator implements an Iterative Kalman Smoother (IKS) to track the 3D motion of a VINS system, such as a mobile device, in real-time using visual and inertial measurements.
41 Citations
19 Claims
-
1. A vision-aided inertial navigation system comprising:
-
at least one image source to produce image data along a trajectory of the vision-aided inertial navigation system (VINS) within an environment, wherein the image data contains a plurality of features observed within the environment at a plurality of poses of the VINS along the trajectory; an inertial measurement unit (IMU) to produce IMU data indicative of motion of the vision-aided inertial navigation system; and a hardware-based processing unit comprising an estimator that computes, based on the image data and the IMU data, a sliding window of state estimates for at least a position and orientation of the vision-aided inertial navigation system for a plurality of poses of the VINS along the trajectory and respective covariances for each of the state estimates, each of the respective covariances representing an amount of uncertainty in the corresponding state estimate, and wherein the estimator computes the state estimates by; classifying the visual features observed at the poses of the VINS within the sliding window into at least a first set of the features and a second set of features as a function of a position within the sliding window for the respective pose from which the respective feature was observed, the second set of features being associated with one or more older poses than the first set of features within the sliding window, applying an extended Kalman filter to update, within the sliding window, each of the state estimates for the VINS and the features using the IMU data and the image data obtained associated with both the first set of features and the second set of features observed from the plurality of poses within the sliding window, and updating, for each of the state estimates, the respective covariance using the IMU data and the image data associated with the second set of features without using the image data associated with the first set of features. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method for computing state estimates for a vision-aided inertial navigation system (VINS) comprising:
-
receiving image data along a trajectory of the VINS within an environment, wherein the image data contains a plurality of features observed within the environment at a plurality of poses of the VINS along the trajectory; receiving inertial measurement data from an inertial measurement unit (IMU) indicative of motion of the vision-aided inertial navigation system; and computing a sliding window of state estimates for a position and orientation of the vision-aided inertial navigation system for a plurality of poses of the VINS along the trajectory with a processing unit comprising an estimator, wherein computing the state estimates comprises; classifying the visual features observed at the poses of the VINS within the sliding window into a first set of features and a second set of features as a function of a position within the sliding window for the respective pose from which the respective feature was observed, the second set of features being associated with one or more older poses within the sliding window than the first set of features; applying an extended Kalman filter to update, within the sliding window, each of the state estimates for the VINS and the features using the IMU data and the image data obtained associated with both the first set of features and the second set of features observed from the plurality of poses within the sliding window, and updating, within the sliding window and for each of the state estimates, the respective covariance using the IMU data and the image data associated with the second set of features without using the IMU data and the image data associated with the first set of features. - View Dependent Claims (11, 12, 13, 14, 15, 16)
-
-
17. A non-transitory computer-readable storage device comprising program code to cause a processor to perform the operations of:
-
receiving image data along a trajectory of the vision-aided inertial navigation system (VINS) within an environment, wherein the image data contains a plurality of features observed within the environment at a plurality of poses of the VINS along the trajectory; receiving inertial measurement data from an inertial measurement unit (IMU) indicative of motion of the vision-aided inertial navigation system; and computing a sliding window of state estimates for a position and orientation of the vision-aided inertial navigation system for a plurality of poses of the VINS along the trajectory with a processing unit comprising an estimator, wherein computing the state estimates comprises; classifying the visual features observed at the poses of the VINS within the sliding window into a first set of features and a second set of features as a function of a position within the sliding window for the respective pose from which the respective feature was observed, the second set of features being associated with one or more older poses within the sliding window than the first set of features; applying an extended Kalman filter to update, within the sliding window, each of the state estimates for the VINS and the features using the IMU data and the image data obtained associated with both the first set of features and the second set of features observed from the plurality of poses within the sliding window, and updating, within the sliding window and for each of the state estimates, the respective covariance using the IMU data and the image data associated with the second set of features without using the IMU data and the image data associated with the first set of features. - View Dependent Claims (18, 19)
-
Specification