Methods and systems for mobile-agent navigation

• US 9,969,337 B2
• Filed: 11/16/2015
• Issued: 05/15/2018
• Est. Priority Date: 09/03/2014
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Liao, Miao
• Primary Examiner(s): Dager, Jonathan M
• Application Number:
  US14/941,691
• Publication Number:
  US 20160068114A1
• Time in Patent Office: 911 Days
• Field of Search: None
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G05D 1/0253 extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow determining position or orientation from images G06T7/70
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

Feature detection may be performed on an image. After feature descriptors for each detected feature are computed, feature matching between feature descriptors for the current image and for a key image frame is performed. If a sufficient number of good matches are identified, key points associated with the feature correspondences may be projected from image coordinates to world coordinates. A distance, in the world coordinate frame, between each feature correspondence may he computed. When the computed distances indicate sufficient movement of the mobile agent to ensure accurate motion estimation, a motion estimate may he computed from the pairs of world coordinates associated with the feature correspondences. A current camera pose in a global coordinate frame may be generated. A motion trajectory may then be determined and feature descriptors for the key image may be updated to the feature descriptors for the current image frame.

Claims

1. A method for estimating motion, said method comprising:

performing feature detection on a current image frame to identify a plurality of regions in said current image frame, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;

computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said current image frame;

performing feature matching between said plurality of feature descriptors for said current image frame and a plurality of feature descriptors computed for a key image frame, thereby producing a plurality of feature correspondences; and

when a number of feature correspondences in said plurality of feature correspondences meets a sufficiency criterion;

for each feature correspondence in said plurality of feature correspondences, computing a distance between said corresponding features; and

when said plurality of distances meet a first criterion;

for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;

computing a motion estimate from said plurality of pairs of world coordinates;

generating a current camera pose in a global coordinate frame;

determining a motion trajectory from said current camera pose; and

updating said plurality of feature descriptors computed for said key image frame to said plurality of feature descriptors for said current image frame.

2. A method as described in claim 1, wherein said motion estimate comprises a rotation matrix and a translation vector.

3. A method as described in claim 2, wherein said rotation matrix is associated with an in-plane rotation.

4. A method as described in claim 1, wherein said current image frame is received from a camera rigidly mounted on a mobile agent.

5. A method as described in claim 1, wherein said performing feature detection comprises using a feature detector selected from the group consisting of an edge detector, a corner detector, a blob detector, a peak detector, a SIFT key-point detector, a SURF key-point detector, a FAST key-point detector, a grid-based FAST key-point detector, an ORB key-point detector, a MSER key-point detector, a BRIEF key-point detector, a BRISK key-point detector, a FREAK key-point detector and a STAR key-point detector.

6. A method as described in claim 1, wherein said feature descriptor is a feature descriptor selected from the group consisting of a block of pixel values, a block of normalized pixel values, a block of gradient values, a block of adjusted pixel values, a SIFT feature descriptor, a SURF feature descriptor, an ORB feature descriptor, a BRIEF feature descriptor, a BRISK feature descriptor and a FREAK feature descriptor.

7. A method as described in claim 1, wherein said performing feature matching comprises a bitwise comparison of a first feature descriptor and a second feature descriptor.

8. A method as described in claim 1, wherein said performing feature matching comprises pruning a plurality of candidate matches.

9. A method as described in claim 1, wherein said computing a motion estimate comprises using a motion estimator selected from the group consisting of an Orthogonal-Procrustes-Analysis-based motion estimator and an Absolute-Orientation-based motion estimator.

10. A method as described in claim 1, wherein said computing a motion estimate comprises rejecting an outlier in said plurality of feature correspondences.

11. A method as described in claim 10, wherein said rejecting comprises performing RANSAC motion estimation.

12. A method as described in claim 1, wherein said generating a current camera pose comprises concatenating said estimated motion to a previous camera pose.

13. A mobile agent comprising:

a rigidly mounted camera;

a processor component; and

a non-transitory computer-readable medium encoded with a computer program code for causing said processor component to execute a method for estimating motion, said method comprising;

performing feature detection on a current image frame to identify a plurality of regions in said current image frame, wherein each region in said plurality of regions is associated with a key point in an image coordinate frame;

computing a feature descriptor for each region in said plurality of regions, thereby producing a plurality of feature descriptors for said current image frame;

performing feature matching between said plurality of feature descriptors for said current image frame and a plurality of feature descriptors computed for a key image frame, thereby producing a plurality of feature correspondences; and

when there are a sufficient number of feature correspondences in said plurality of feature correspondences;

for each feature correspondence in said plurality of feature correspondences, computing a distancing between said corresponding features; and

when said plurality of distances meet a first criterion;

for each feature correspondence in said plurality of feature correspondences, projecting said associated key points from said image coordinate frames to a world coordinate frame, thereby producing a plurality of pairs of world coordinates;

computing a motion estimate from said plurality of pairs of world coordinates;

generating a current camera pose in a global coordinate frame;

determining a motion trajectory from said current camera pose; and

updating said plurality of feature descriptors computed for said key image frame to said plurality of feature descriptors for said current image frame.

14. A mobile agent as described in claim 13, wherein said camera is rigidly mounted with respect to a ground plane.

15. A mobile agent as described in claim 13, wherein

said camera is mounted on a front end or a back end of said mobile agent and above a body chassis of said mobile agent;

and

said when said camera is mounted on said front end of said mobile agent, said camera is tilted to have a field of view of a ground path directly in front of said mobile agent and when said camera is mounted on said back end of said mobile agent, said camera is tilted to have a field of view of said ground path direct behind said mobile agent.

16. A non-transitory computer-readable medium encoded with a computer program code for causing a processor to execute a method for adding a waypoint to an ordered plurality of waypoints, said method comprising:

receiving a new waypoint;

adding said new waypoint to a terminal end of an ordered plurality of waypoints associated with a mobile-agent path;

determining if a mobile agent is running; and

when said mobile agent is not running;

updating a target waypoint to said new waypoint; and

driving toward said target waypoint.

17. A non-transitory computer-readable medium encoded with a computer program code for causing a processor to correct for drift, said method comprising:

computing a current position and a current heading of a mobile agent;

determining if said mobile agent is close to a target waypoint; and

when said mobile agent is determined to be close to said target waypoint;

determining if said target waypoint is associated with a machine-readable code; and

when said target waypoint is determined to be associated with a machine-readable code;

identifying said machine-readable code;

computing a position of said mobile agent relative to said machine-readable code;

determining an actual position and an actual heading of said mobile agent based on said position of said mobile agent relative to said two-dimensional machine-readable code;

updating said current position to said actual position; and

updating said current heading to said actual heading.

18. A non-transitory computer-readable medium as described in claim 17, wherein the machine-readable code is a code selected from the group consisting of a barcode, a QR code, an Aztec code and a Data Matrix code.

19. A non-transitory computer-readable medium encoded with a computer program code for causing a processor to optimize a camera parameter, said method comprising:

receiving a plurality of camera parameter values;

causing a mobile agent to traverse a known path from a known starting point to a known ending point;

recording a plurality of image frames while said mobile agent traverses said known path;

estimating an endpoint of said mobile agent using said plurality of image frames and said plurality of camera parameter values;

computing a difference between said known ending point and said endpoint; and

when said difference meets a first criterion;

optimizing said camera parameters thereby producing a plurality of optimized camera parameter values; and

replacing said plurality of camera parameter values with said plurality of optimized camera parameter values.

20. A non-transitory computer-readable medium as described in claim 19, wherein said known path is a straight line.