Method and apparatus for generating three-dimensional pose using monocular visual sensor and inertial measurement unit
First Claim
1. An apparatus for providing a three-dimensional pose data-stream comprising:
- one or more monocular visual sensors for providing images of an environment surrounding the apparatus;
an inertial measurement unit (IMU) for providing gyroscope, acceleration and velocity information, collectively referred to as IMU information;
a feature tracking module for generating feature track information for the images;
one or more global tracking units for providing global location information; and
a landmark matcher, coupled to the feature tracking module, for generating an error state derived from comparing feature locations from at least one image to a visual landmark within the environment; and
an error-state filter, coupled to the feature tracking module, the IMU, the one or more global tracking units, the landmark matcher and the one or more visual sensors, for correcting the IMU information and the global location information and producing a pose estimation based on at least one error-state model chosen according to the images, the IMU information, the global location information and the feature tracking information.
1 Assignment
0 Petitions
Accused Products
Abstract
An apparatus for providing three-dimensional pose comprising monocular visual sensors for providing images of an environment surrounding the apparatus, an inertial measurement unit (IMU) for providing gyroscope, acceleration and velocity information, collectively IMU information, a feature tracking module for generating feature tracking information for the images, and an error-state filter, coupled to the feature track module, the IMU and the one or more visual sensors, for correcting IMU information and producing a pose estimation based on at least one error-state model chosen according to the sensed images, the IMU information and the feature tracking information.
-
Citations
23 Claims
-
1. An apparatus for providing a three-dimensional pose data-stream comprising:
-
one or more monocular visual sensors for providing images of an environment surrounding the apparatus; an inertial measurement unit (IMU) for providing gyroscope, acceleration and velocity information, collectively referred to as IMU information; a feature tracking module for generating feature track information for the images; one or more global tracking units for providing global location information; and a landmark matcher, coupled to the feature tracking module, for generating an error state derived from comparing feature locations from at least one image to a visual landmark within the environment; and an error-state filter, coupled to the feature tracking module, the IMU, the one or more global tracking units, the landmark matcher and the one or more visual sensors, for correcting the IMU information and the global location information and producing a pose estimation based on at least one error-state model chosen according to the images, the IMU information, the global location information and the feature tracking information. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A computer implemented method for generating a three-dimensional pose estimation comprising:
-
sensing images of a surrounding environment using a monocular visual sensor; providing gyroscope, acceleration and velocity information, collectively referred to as IMU information; providing location information for tracking global coordinates; matching the images to visual landmarks; and tracking features in the images for providing inliers of the images; and generating an error correction for the IMU information and the location information and generating a pose estimation based on the sensed images, the IMU information, the location information and the inliers of the images. - View Dependent Claims (17, 18, 19, 20)
-
-
21. An apparatus for providing a three-dimensional pose data-stream comprising:
-
one or more monocular visual sensors for providing images of an environment surrounding the apparatus; an inertial measurement unit (IMU) for providing gyroscope, acceleration and velocity information, collectively referred to as IMU information; a feature tracking module for generating feature track information for the images; an error-state smoother, coupled to the feature tracking module, the IMU and the one or more visual sensors, for correcting the IMU information and producing a pose estimation based on at least one error-state model chosen according to the images, the IMU information and the feature tracking information; and wherein the error-state model of the error-state smoother is selected as comprising at least one of an epipolar constraint model, a reprojection constraint model and a rotation-only model, based on parameters provided to the error-state smoother.
-
-
22. An apparatus for providing a three-dimensional pose data-stream comprising:
-
one or more monocular visual sensors for providing images of an environment surrounding the apparatus; an inertial measurement unit (IMU) for providing gyroscope, acceleration and velocity information, collectively referred to as IMU information; a feature tracking module for generating feature track information for the images; a magnetometer for providing magnetic field information; an error-state filter, coupled to the feature tracking module, the IMU, the magnetometer and the one or more visual sensors, for correcting the IMU information and the magnetic field information and producing a pose estimation based on at least one error-state model chosen according to the images, the IMU information, the magnetic field information and the feature tracking information; and wherein successive readings of the magnetometer are compared to the IMU information and to a nominal magnetic field providing error correction for the error-state filter.
-
-
23. A computer implemented method for generating a three-dimensional pose estimation comprising:
-
sensing images of a surrounding environment using a monocular visual sensor; providing gyroscope, acceleration and velocity information, collectively referred to as IMU information; tracking features in the images for providing inliers of the images; generating an error correction for the IMU information and a pose estimation based on the sensed images, the IMU information and the inliers of the images; and wherein the generating a pose estimation comprises selecting between a reprojection model, an epipolar model or a rotation-only model based on light levels and the IMU information.
-
Specification