Methods and Systems for Vision-Based Motion Estimation

• US 20160063330A1
• Filed: 09/03/2014
• Published: 03/03/2016
• Est. Priority Date: 09/03/2014
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Xu, Xinyu, Liao, Miao, van Beek, Petrus J. L.
• Granted Patent: US 10,339,389 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

Aspects of the present invention are related to methods and systems for vision-based computation of ego-motion.

Claims

1. A method for estimating motion, said method comprising:

receiving an incoming image;

performing feature detection on said received, incoming image to identify a plurality of regions in said received, incoming image, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;

computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said incoming image;

performing feature matching between said plurality of feature descriptors for said received, incoming image and a plurality of feature descriptors computed for a previous image, thereby producing a plurality of feature correspondences;

for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;

computing a motion estimate from said plurality of pairs of world coordinates;

selecting a key pose;

generating a current camera pose in a global coordinate frame;

determining a motion trajectory from said current camera pose; and

updating said plurality of feature descriptors computed for a previous frame to said plurality of feature descriptors for said received, incoming frame.

2. A method as described in claim 1, wherein said motion estimate comprises a rotation matrix and a translation vector.

3. A method as described in claim 2, wherein said rotation matrix is associated with an in-plane rotation.

4. A method as described in claim 1, wherein said received, incoming image is received from a camera rigidly mounted on a mobile agent.

5. A method as described in claim 1, wherein said performing feature detection comprises using a feature detector selected from the group consisting of an edge detector, a corner detector, a blob detector, a peak detector, a SIFT key-point detector, a SURF key-point detector, a FAST key-point detector, a grid-based FAST key-point detector, an ORB key-point detector, a MSER key-point detector, a BRIEF key-point detector, a BRISK key-point detector, a FREAK key-point detector and a STAR key-point detector.

6. A method as described in claim 1, wherein said feature descriptor is a feature descriptor selected from the group consisting of a block of pixel values, a block of normalized pixel values, a block of gradient values, a block of adjusted pixel values, a SIFT feature descriptor, a SURF feature descriptor, an ORGB feature descriptor, a BRIEF feature descriptor, a BRISK feature descriptor and a FREAK feature descriptor.

7. A method as described in claim 1, wherein said performing feature matching comprises a fast, nearest-neighbor search with a k-dimensional tree.

8. A method as described in claim 1, wherein said performing feature matching comprises pruning a plurality of candidate matches.

9. A method as described in claim 1, wherein said computing a motion estimate comprises using a motion estimator selected from the group consisting of an Orthogonal-Procrustes-Analysis-based motion estimator and an Absolute-Orientation-based motion estimator.

10. A method as described in claim 1, wherein said computing a motion estimate comprises rejecting an outlier in said plurality of feature correspondences.

11. A method as described in claim 10, wherein said rejecting comprises performing RANSAC motion estimation.

12. A method as described in claim 1, wherein said selecting a key pose comprises:

comparing a rotation angle of said motion estimate to an angle threshold and a distance traveled by a mobile agent to a distance threshold; and

when said angle comparison meets a first criterion or said distance comparison meets a second criterion, detecting a key pose.

13. A method as described in claim 12, wherein:

said first criterion is said rotation angle is greater than said angle threshold; and

said second criterion is said distance traveled is greater than said distance threshold.

14. A method as described in claim 1, wherein said generating a current camera pose comprises concatenating said estimated motion to a previous camera pose.

15. A mobile agent comprising:

a rigidly mounted camera;

a processor component; and

a non-transitory computer-readable medium encoded with a computer program code for causing said processor component to execute a method for estimating motion, said method comprising;

receiving an incoming image;

performing feature detection on said received, incoming image to identify a plurality of regions in said received, incoming image, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;

computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said incoming image;

performing feature matching between said plurality of feature descriptors for said received, incoming image and a plurality of feature descriptors computed for a previous image, thereby producing a plurality of feature correspondences;

for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;

computing a motion estimate from said plurality of pairs of world coordinates;

selecting a key pose;

generating a current camera pose in a global coordinate frame;

determining a motion trajectory from said current camera pose; and

updating said plurality of feature descriptors computed for a previous frame to said plurality of feature descriptors for said received, incoming frame.

16. A mobile agent as described in claim 15, wherein said camera is rigidly mounted with respect to a ground plane.

17. A mobile agent as described in claim 15, wherein

said camera is mounted on a front end or a back end of said mobile agent and above a body chassis of said mobile agent;

and

said when said camera is mounted on said front end of said mobile agent, said camera is tilted to have a field of view of a ground path directly in front of said mobile agent and when said camera is mounted on said back end of said mobile agent, said camera is tilted to have a field of view of said ground path direct behind said mobile agent.

18. A non-transitory computer-readable medium encoded with a computer program code for causing a processor to execute a method for estimating motion, said method comprising:

receiving an incoming image;

performing feature detection on said received, incoming image to identify a plurality of regions in said received, incoming image, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;

computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said incoming image;

performing feature matching between said plurality of feature descriptors for said received, incoming image and a plurality of feature descriptors computed for a previous image, thereby producing a plurality of feature correspondences;

for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;

computing a motion estimate from said plurality of pairs of world coordinates;

selecting a key pose;

generating a current camera pose in a global coordinate frame;

determining a motion trajectory from said current camera pose; and

updating said plurality of feature descriptors computed for a previous frame to said plurality of feature descriptors for said received, incoming frame.

19. A non-transitory computer-readable medium as described in claim 18, wherein said processor is integrated on a mobile agent.

20. A non-transitory computer-readable medium as described in claim 18, wherein said incoming image is generated by a camera attached to a mobile agent.