Constrained key frame localization and mapping for vision-aided inertial navigation
First Claim
1. A vision-aided inertial navigation system comprising:
- an image source to produce image data for a first keyframe, one or more non-keyframes and a second keyframe along a trajectory of the vision-aided inertial navigation system (VINS), the one or more non-keyframes located between the first keyframe and second keyframe along the trajectory;
an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes; and
a processing unit comprising an estimator that processes the IMU data and the image data to compute respective state estimates for a position and orientation of the VINS for the first keyframe and for the second keyframe,wherein, when computing the state estimates, the estimator constrains the state estimates for the second keyframe relative to the state estimates for the first keyframe based on the IMU data and the image data from the one or more non-keyframes, andwherein, when constraining the state estimates, the estimator treats a landmark observed within the image data for the first keyframe or the second keyframe as different from the same landmark observed within the image data for the non-keyframes.
1 Assignment
0 Petitions
Accused Products
Abstract
Estimation techniques for vision-aided inertial navigation are described. In one example, a vision-aided inertial navigation system (VINS) comprises an image source to produce image data for a keyframe and one or more non-keyframes along a trajectory, the one or more non-keyframes preceding the keyframe along the trajectory. The VINS comprises an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes, and a processing unit comprising an estimator that processes the IMU data and the image data to compute state estimates of the VINS. The estimator computes the state estimates of the VINS for the keyframe by constraining the state estimates based on the IMU data and the image data for the one or more non-keyframes of the VINS without computing state estimates of the VINS for the one or more non-keyframes.
30 Citations
27 Claims
-
1. A vision-aided inertial navigation system comprising:
-
an image source to produce image data for a first keyframe, one or more non-keyframes and a second keyframe along a trajectory of the vision-aided inertial navigation system (VINS), the one or more non-keyframes located between the first keyframe and second keyframe along the trajectory; an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes; and a processing unit comprising an estimator that processes the IMU data and the image data to compute respective state estimates for a position and orientation of the VINS for the first keyframe and for the second keyframe, wherein, when computing the state estimates, the estimator constrains the state estimates for the second keyframe relative to the state estimates for the first keyframe based on the IMU data and the image data from the one or more non-keyframes, and wherein, when constraining the state estimates, the estimator treats a landmark observed within the image data for the first keyframe or the second keyframe as different from the same landmark observed within the image data for the non-keyframes. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for computing state estimates for a vision-aided inertial navigation system (VINS) comprising:
-
receiving image data produced by an image source of the vision-aided inertial navigation system for a first keyframe, one or more non-keyframes and a second keyframe along a trajectory of the vision-aided inertial navigation system (VINS), the one or more non-keyframes located between the first keyframe and second keyframe along the trajectory; receiving, from an inertial measurement unit (IMU), IMU data indicative of motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes; and processing the IMU data and the image data to compute respective state estimates for a position and orientation of the VINS for the first keyframe and for the second keyframe, wherein computing the state estimates comprises constraining the state estimates for the second keyframe relative to the state estimates for the first keyframe based on the IMU data and the image data from the one or more non-keyframes, and wherein constraining the state estimates comprises treating a landmark observed within the image data for the first keyframe and the second keyframe as different from the same landmark observed within the image data for the non-keyframes. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19)
-
-
20. A mobile device comprising:
-
an image source to produce image data for a first keyframe, one or more non-keyframes and a second keyframe along a trajectory of the vision-aided inertial navigation system (VINS), the one or more non-keyframes located between the first keyframe and second keyframe along the trajectory; an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes; and a processor configured to compute respective state estimates for a respective position and orientation of the VINS for the first keyframe and for the second keyframe, wherein, when computing the state estimates, the processor constrains the state estimates for the second keyframe relative to the state estimates for the first keyframe based on the IMU data and the image data from the one or more non-keyframes. - View Dependent Claims (21)
-
-
22. A non-transitory computer-readable storage device comprising program code to cause a processor to:
-
receive image data produced by an image source of the vision-aided inertial navigation system for a first keyframe, one or more non-keyframes and a second keyframe along a trajectory of the vision-aided inertial navigation system (VINS), the one or more non-keyframes located between the first keyframe and second keyframe along the trajectory; receive, from an inertial measurement unit (IMU), IMU data indicative of motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes; and process the IMU data and the image data to compute state estimates for a position and orientation of the VINS within the second keyframe by constraining the state estimates relative to state estimates for the position and orientation of the VINS within the first keyframe based on the IMU data and the image data of the one or more non-keyframes of the VINS without computing state estimates for the position and orientation of the VINS for the one or more non-keyframes. - View Dependent Claims (23)
-
-
24. A vision-aided inertial navigation system comprising:
-
an image source to produce image data for a first keyframe, one or more non-keyframes and a second keyframe along a trajectory of the vision-aided inertial navigation system (VINS), the one or more non-keyframes located between the first keyframe and second keyframe along the trajectory; an inertial measurement unit (IMU) to produce IMU data indicative of a motion of the VINS along the trajectory for the keyframe and the one or more non-keyframes; and a processing unit comprising an estimator that processes the IMU data and the image data to compute respective state estimates for a position and orientation of the VINS for the first keyframe and for the second keyframe, wherein, when computing the state estimates, the estimator constrains the state estimates for the second keyframe relative to the state estimates for the first keyframe based on the IMU data and the image data from the one or more non-keyframes. - View Dependent Claims (25, 26, 27)
-
Specification