SYSTEMS AND METHODS FOR ESTIMATING THE STRUCTURE AND MOTION OF AN OBJECT

• US 20140168461A1
• Filed: 06/13/2012
• Published: 06/19/2014
• Est. Priority Date: 06/13/2011
• Status: Active Grant

• Current Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Original Assignee: University of Florida Research Foundation, Incorporated (State University System of Florida)
• Inventors: Dixon, Warren, Dani, Ashwin P.
• Granted Patent: US 9,179,047 B2
• Time in Patent Office: Days
• US Class Current: 348/222.1
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

In one embodiment, the structure and motion of a stationary object are determined using two images and a linear velocity and linear acceleration of a camera. In another embodiment, the structure and motion of a stationary or moving object are determined using an image and linear and angular velocities of a camera.

Claims

1. A method for estimating the structure and motion of a stationary object, the method comprising:

capturing first and second images of the object with a moving camera;

recording a linear velocity and a linear acceleration of the camera the instants at which each image was captured;

extracting matching feature points of the object from the images;

computing Euclidean coordinates of the feature points for each image;

estimating a rotation matrix based upon the Euclidean coordinates, the rotation matrix comprising a representation of rotation between the two images;

computing an estimate of a state vector, the state vector being based upon the depths of the feature points relative to the camera and linear velocities of the camera; and

computing unknown camera velocities and coordinates of each feature point based upon the state vector.

2. The method of claim 1, wherein recording a linear velocity and a linear acceleration comprises recording only a single linear velocity and a single linear velocity in a single direction.

3. The method of claim 1, wherein computing Euclidean coordinates comprises computing the coordinates using pixel coordinates of the feature points and a camera calibration matrix.

4. The method of claim 1, wherein estimating a rotation matrix comprises first estimating a homography matrix that is a geometry transformation between the first and second images that encodes rotation and scaled translation information.

5. The method of claim 4, wherein estimating a rotation matrix further comprises decomposing the homography matrix to obtain the rotation matrix.

6. The method of claim 1, wherein computing an estimate of a state vector comprises first selecting an initial state vector and updating it based upon the depths.

7. The method of claim 1, further comprising selecting an initial rotation error vector.

8. The method of claim 7, further comprising computing the angular velocity of the camera based upon the rotation matrix.

9. The method of claim 8, wherein computing the angular velocity of the camera further comprises computing an estimate of a rotation error vector and an estimate of the derivative of the rotation error vector.

10. A non-transitory computer-readable medium that stores a structure and motion estimator, the medium comprising:

logic configured to capture first and second images of the object with a moving camera;

logic configured to record a linear velocity and a linear acceleration of the camera the instants at which each image was captured;

logic configured to extract matching feature points of the object from the images;

logic configured to compute Euclidean coordinates of the feature points for each image;

logic configured to estimate a rotation matrix based upon the Euclidean coordinates, the rotation matrix comprising a representation of rotation between the two images;

logic configured to compute an estimate of a state vector, the state vector being based upon the depths of the feature points relative to the camera and linear velocities of the camera; and

logic configured to compute unknown camera velocities and coordinates of each feature point based upon the state vector.

11. The computer-readable medium of claim 10, wherein the logic configured to compute Euclidean coordinates comprises logic configured to compute the coordinates using pixel coordinates of the feature points and a camera calibration matrix.

12. The computer-readable medium of claim 10, wherein the logic configured to estimate a rotation matrix comprises logic configured to estimate a homography matrix that is a geometry transformation between the first and second images that encodes rotation and scaled translation information.

13. The computer-readable medium of claim 12, wherein the logic configured to estimate a rotation matrix further comprises logic configured to decompose the homography matrix to obtain the rotation matrix.

14. The computer-readable medium of claim 10, wherein the logic configured to compute an estimate of a state vector comprises logic configured to select an initial state vector and update it based upon the depths.

15. The computer-readable medium of claim 10, further comprising logic configured to select an initial rotation error vector.

16. The computer-readable medium of claim 15, further comprising logic configured to compute the angular velocity of the camera based upon the rotation matrix.

17. The computer-readable medium of claim 16, wherein the logic configured to compute the angular velocity of the camera further comprises logic configured to compute an estimate of a rotation error vector and an estimate of the derivative of the rotation error vector.

18. A method for estimating the structure and motion of a stationary object, the method comprising, the method comprising:

capturing an image of the object with a moving camera;

recording linear and angular velocities of the camera the instant at which the image was captured;

extracting feature points of the object from the image;

computing Euclidean coordinates of the feature points;

computing a state vector [{circumflex over (x)}₁,{circumflex over (x)}₂,{circumflex over (x)}₃], wherein x₁ is the inverse depths of the feature points, x₂ is the ratios of x and z coordinates of the feature points, and {circumflex over (x)}₃ is the ratios of y and z coordinates of the feature points;

determining z coordinates of each feature point in the image based upon the state vector; and

computing x and y coordinates of each feature point in the image based upon the state vector.

19. The method of claim 18, wherein computing Euclidean coordinates comprises computing the coordinates using pixel coordinates of the feature points and a camera calibration matrix.

20. The method of claim 18, wherein computing a state vector [{circumflex over (x)}₁,{circumflex over (x)}₂,{circumflex over (x)}₃] comprises computing the state vector using pixel coordinates of the feature points, an auxiliary vector, and an observer parameter.

21. The method of claim 18, wherein determining z coordinates comprises inverting {circumflex over (x)}₃.

22. The method of claim 18, wherein computing x and y coordinates comprises computing the coordinates using the following relations:

X=x₁/{circumflex over (x)}₃,

and

Y=x₂/{circumflex over (x)}₃

wherein X is the x coordinate and Y is the y coordinate.

23. The method of claim 18, further comprising selecting an initial auxiliary vector prior to computing the state [{circumflex over (x)}₁,{circumflex over (x)}₂,{circumflex over (x)}₃].

24. The method of claim 18, further comprising, prior to computing the state vector, choosing a matrix D, which is a representation of the rotation of the object relative to the camera.

25. The method of claim 24, further comprising choosing a matrix A, which is a representation of the linear and angular velocities of the camera.

26. The method of claim 25, further comprising computing observer parameter and gain matrices.

27. A non-transitory computer-readable medium that stores a structure and motion estimator, the medium comprising:

logic configured to capture an image of the object with a moving camera;

logic configured to record linear and angular velocities of the camera the instant at which the image was captured;

logic configured to extract feature points of the object from the image;

logic configured to compute Euclidean coordinates of the feature points;

logic configured to compute a state vector [{circumflex over (x)}₁,{circumflex over (x)}₂,{circumflex over (x)}₃], wherein x₁ is the inverse depths of the feature points, x₂ is the ratios of x and z coordinates of the feature points, and {circumflex over (x)}₃ is the ratios of y and z coordinates of the feature points;

logic configured to determine z coordinates of each feature point in the image based upon the state vector; and

logic configured to compute x and y coordinates of each feature point in the image based upon the state vector.

28. The computer-readable medium of claim 27, wherein the logic configured to compute Euclidean coordinates comprises logic configured to compute the coordinates using pixel coordinates of the feature points and a camera calibration matrix.

29. The computer-readable medium of claim 27, wherein the logic configured to compute a state vector [{circumflex over (x)}₁,{circumflex over (x)}₂,{circumflex over (x)}₃] comprises logic configured to compute the state vector using pixel coordinates of the feature points, an auxiliary vector, and an observer parameter.

30. The computer-readable medium of claim 27, wherein the logic configured to determine z coordinates comprise logic configured to invert {circumflex over (x)}₃.

31. The computer-readable medium of claim 27, wherein logic configured to computing x and y coordinates comprises logic configured to computing the coordinates using the following relations:

X=x₁/{circumflex over (x)}₃,

and

Y=x₂/{circumflex over (x)}₃

wherein X is the x coordinate and Y is the y coordinate.

32. The computer-readable medium of claim 27, further comprising logic configured to select an initial auxiliary vector prior to computing the state vector [{circumflex over (x)}₁,{circumflex over (x)}₂,{circumflex over (x)}₃].

33. The computer-readable medium of claim 27, further comprising logic configured to, prior to computing the state vector, choose a matrix D, which is a representation of the rotation of the object relative to the camera.

34. The computer-readable medium of claim 33, further comprising logic configured to choose a matrix A, which is a representation of the linear and angular velocities of the camera.

35. The computer-readable medium of claim 34, further comprising logic configured to compute observer parameter and gain matrices.