Image-based system and methods for vehicle guidance and navigation

• US 8,320,616 B2
• Filed: 08/21/2007
• Issued: 11/27/2012
• Est. Priority Date: 08/21/2006
• Status: Active Grant

• Current Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Original Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Inventors: Gans, Nicholas Raphael, Dixon, Warren Everett, Kaiser, Michael Kent, Mehta, Siddhartha Satish
• Primary Examiner(s): LEE, JOHN W
• Application Number:
  US12/376,709
• Publication Number:
  US 20090285450A1
• Time in Patent Office: 1,925 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method of estimating position and orientation of a vehicle using image data is provided. The method includes capturing an image of a region external to the vehicle using a camera mounted to the vehicle, and identifying in the image a set of feature points of the region. The method further includes subsequently capturing another image of the region from a different orientation of the camera, and identifying in the image the same set of feature points. A pose estimation of the vehicle is generated based upon the identified set of feature points and corresponding to the region. Each of the steps are repeated at with respect to a different region at least once so as to generate at least one succeeding pose estimation of the vehicle. The pose estimations are then propagated over a time interval by chaining the pose estimation and each succeeding pose estimation one with another according to a sequence in which each was generated.

Claims

1. A method of estimating position and orientation of a vehicle using image data, the method comprising:

(a) capturing an image of a region external to the vehicle using a camera mounted to the vehicle, and identifying in the image a set of feature points of the region;

(b) subsequently capturing another image of the region from a different orientation of the camera, and identifying in the image the same set of feature points;

(c) generating a pose estimation of the vehicle by determining a Euclidean homography matrix based upon the identified feature points and decomposing the Euclidean homography matrix to determine a scaling factor, a unit normal vector perpendicular to the region, a rotation, and a ratio of a translation to a distance from the camera to the region, wherein the distance is measured by a vector parallel to the unit normal vector;

(d) generating at least one succeeding pose estimation of the vehicle by repeating steps (a)-(c) with respect to a different region; and

(e) propagating the pose estimations over a time interval by chaining the pose estimation and each succeeding pose estimation one with another according to a sequence in which each was generated.

2. The method of claim 1, further comprising determining the distance from the camera to the region and, based upon the determined distance, computing the translation from the ratio of the translation to the distance from the camera to the region.

3. The method of claim 2, wherein the step of determining the distance comprises projecting another vector measuring the distance from the camera to the region onto the unit normal vector according to the equation d=n·

a, where d measures the distance, n is the unit normal vector, and a is the other vector measuring the distance from the camera to the region, wherein the other vector is not parallel to the unit normal vector.

4. The method of claim 2, wherein the rotation measured at a time t₁ is R₀₁(t₁), and the translation measured at time t₁ is x₀₁(t₁) and further comprising determining, with respect to a constant world frame, a corresponding rotation R₁(t₁) and a corresponding translation x₁(t₁) based upon the equations:

R₁=R₀R₀₁(t₁),

x₁=R₀₁x₀₁(t₁)+x₀

where R₀ is a previously determined rotation with respect to the constant world frame, x₀ is a previously determined translation with respect to the constant world frame.

5. The method of claim 4, wherein the previously determined rotation and translation are determined based upon data generated by an inertial measurement system.

6. The method of claim 2, further comprising determining another distance parallel to another unit normal vector perpendicular to another region based upon the following equation if the two regions are coplanar:

d_b(t₁)=d(t₀)+x₀₁(t₁)·

n(t₀)

where d_b(t₁) is the other distance measure at time t₁, d_a(t₀) is the distance measured at time t₀, t₁>

t₀, x₀₁(t₁) is the translation measured at time t₁, and n(t₀) is the unit normal vector determined at time t₀.

7. The method of claim 3, further comprising determining another distance parallel to another unit normal vector perpendicular to another region according to the following steps if the two regions are not coplanar:

solve two homography equations, given as

8. A system for estimating position and orientation of a vehicle using image data, the system comprising:

a camera mounted to the vehicle; and

a pose estimator configured to

(a) cause said camera to capture an image of a region external to the vehicle using a camera mounted to the vehicle, and identify in the image a set of feature points of the region;

(b) subsequently cause said camera to capture another image of the region from a different orientation relative to the region, and identify in the image the set of feature points;

(c) generate a pose estimation of the vehicle by determining a Euclidean homography matrix based upon the identified feature points and decomposing the Euclidean homography matrix to determine a scaling factor, a unit normal vector perpendicular to the region, a rotation, and a ratio of a translation to a distance from the camera to the region, wherein the distance is measured by a vector parallel to the unit normal vector;

(d) generate at least one succeeding pose estimation of the vehicle by repeating steps (a)-(c) with respect to a different region; and

(e) propagate the pose estimations over a time interval by chaining the pose estimation and each succeeding pose estimation one with another according to a sequence in which each was generated.

9. The system of claim 8, wherein said pose estimator is further configured to determine the distance from the camera to the region and, based upon the determined distance, compute the translation from the ratio of the translation to the distance from the camera to the region.

10. The system of claim 9, wherein said pose estimator is configured to determine the distance by performing a projection of another vector measuring the distance from the camera to the region onto the unit normal vector according to the equation d=n·

a, where d measures the distance, n is the unit normal vector, and a is the other vector measuring the distance from the camera to the region, wherein the other vector is not parallel to the unit normal vector.

11. The system of claim 9, wherein the rotation measured at a time t₁ is R₀₁(t₁), and the translation measured at time t₁ is x₀₁(t₁), and wherein the pose estimator is further configured to determine, with respect to a constant world frame, a corresponding rotation R₁(t₁) and a corresponding translation x₁(t₁) based upon the equations:

R₁=R₀R₀₁(t₁),

x₁=R₀₁x₀₁(t₁)+x₀

where R₀ is a previously determined rotation with respect to the constant world frame, x₀ is a previously determined translation with respect to the constant world frame.

12. The system of claim 11, further comprising an inertial measurement system, the previously determined rotation and translation being determined based upon data generated by said inertial measurement system.

13. The system of claim 9, wherein said pose estimator is further configured to determine another distance parallel to another unit normal vector perpendicular to another region based upon the following equation if the two regions are coplanar:

d_b(t₁)=d_a(t₀)+x₀₁(t₁)·

n(t₀)

where d_b(t₁) is the other distance measure at time t₁, d_a(t₀) is the distance measured at time t₀, t₁>

t₁, x₀₁(t₁) is the translation measured at time t₁, and n(t₀) is the unit normal vector determined at time t₀.

14. The system of claim 9, wherein said pose estimator is further configured to determine another distance parallel to another unit normal vector perpendicular to another region according to the following procedure if the two regions are not coplanar;

solve two homography equations, given as

15. A non-transitory computer-readable storage medium having embedded therein instruction code for causing the computer to:

(a) capture an image of a region external to a vehicle, the image captured using a camera mounted to a vehicle, and identify in the image a set of feature points of the region;

(b) subsequently capture another image of the region, the other image captured using the camera at a different orientation relative to the region, and identify in the other image the set of feature points;

(c) generate a pose estimation of the vehicle by determining a Euclidean homography matrix based upon the identified feature points and decomposing the Euclidean homography matrix to determine a scaling factor, a unit normal vector perpendicular to the region, a rotation, and a ratio of a translation to a distance from the camera to the region, wherein the distance is measured by a vector parallel to the unit normal vector;

(d) generate at least one succeeding pose estimation of the vehicle by repeating steps (a)-(c) with respect to a different region; and

(e) propagate the pose estimations over a time interval by chaining the pose estimation and each succeeding pose estimations one with another according to a sequence in which each was generated.

16. The non-transitory computer-readable storage medium of claim 15, further comprising instruction code for causing the computer to determine the distance from the camera to the region and, based upon the determined distance, compute the translation from the ratio of the translation to the distance from the camera to the region.

17. The non-transitory computer-readable storage medium of claim 16, wherein the procedure for determining the distance comprises projecting another vector measuring the distance from the camera to the region onto the unit normal vector according to the equation d=n·

a, where d measures the distance, n is the unit normal vector, and a is the other vector measuring the distance from the camera to the region, wherein the other vector is not parallel to the unit normal vector.