System and method for drift-free global trajectory estimation of a mobile platform

• US 10,518,879 B1
• Filed: 10/07/2016
• Issued: 12/31/2019
• Est. Priority Date: 07/10/2014
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Zhang, Lei, Khosla, Deepak, Kim, Kyungnam, Xu, Jiejun, Jeong, Changsoo S.
• Primary Examiner(s): Rush, Eric
• Application Number:
  US15/288,622
• Time in Patent Office: 1,180 Days
• Field of Search: 382100, 382103, 382104, 382153, 382181, 382190, 382209, 382210, 382217, 382218, 382224, 382225, 382291, 382305, 382325, 348113, 348114, 348116-118, 701 1- 3, 701 28, 701400, 701408, 701409, 701450, 701461, 701519, 701521-524, 701526
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft

Abstract

Described is a system for trajectory estimation of a mobile platform, such as a UAV. In operation, the system generates an initial trajectory estimate for the mobile platform which is stored in a trajectory buffer as a buffered trajectory. Images captured at a location are compared with a location recognition database to generate a location label for a current location to designate the current location as a new location or a revisited location. If the location is a revisited location, the system determines if trajectory correction is required. If so, the buffered trajectory is corrected to generate a corrected trajectory as the drift-free trajectory. Finally, the drift-free trajectory can be used in a variety of applications. For example, the drift-free trajectory can be used to cause the mobile platform to traverse a path that coincides with the drift-free trajectory.

Claims

1. A system for trajectory estimation of a mobile platform, the system comprising:

one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of;

generating an initial trajectory estimate for the mobile platform based on sensor data regarding a scene;

determining if a location is a new location or a revisited location using real-time feature point detection;

if the location is determined to be a revisited location, determining if trajectory correction is required; and

if trajectory correction is required, generating a corrected trajectory as a drift-free trajectory, the corrected trajectory being generated based on a previously recorded position at the location;

wherein in generating the corrected trajectory, the corrected trajectory is generated by performing operations of;

sampling equally-spaced points from a latest trajectory segment in a buffered trajectory;

determining corrected positions for each point in the latest trajectory segment, such that when the latest trajectory segment is a straight line, the corrected position for each point is determined based on a geometric distance, the geometric distance being a distance from each point to a starting point of the latest trajectory segment; and

connecting the corrected positions to form the corrected trajectory.

2. The system as set forth in claim 1, further comprising an operation of causing a mobile platform to traverse a path that coincides with the drift-free trajectory.

3. The system as set forth in claim 1, wherein if the location is determined to be a new location, storing coordinates for the new location in a trajectory buffer and designating the initial trajectory estimate as the drift-free trajectory.

4. The system as set forth in claim 1, wherein if trajectory correction is not required, then designating the initial trajectory estimate as the drift-free trajectory.

5. The system as set forth in claim 1, wherein generating the initial trajectory estimate for the mobile platform is performed using a simultaneous localization and mapping (SLAM) module.

6. The system as set forth in claim 1, wherein the sensor data is generated using a sensor system selected from a group consisting of only a monocular camera, only a stereo camera, or a combination of a monocular camera with a depth image sensor.

7. The system as set forth in claim 1, wherein in determining if trajectory correction is required, trajectory correction is required if a current position of the mobile platform at the location is greater than a predetermined distance threshold from a previously recorded position at the location.

8. A system for trajectory estimation of a mobile platform, the system comprising:

one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of;

generating an initial trajectory estimate for the mobile platform based on sensor data regarding a scene;

determining if a location is a new location or a revisited location using real-time feature point detection;

if the location is determined to be a revisited location, determining if trajectory correction is required; and

if trajectory correction is required, generating a corrected trajectory as a drift-free trajectory, the corrected trajectory being generated based on a previously recorded position at the location;

wherein in generating the corrected trajectory, the corrected trajectory is generated by performing operations of;

sampling equally-spaced points from a latest trajectory segment in a buffered trajectory;

determining corrected positions for each point in the latest trajectory segment, such that when the latest trajectory segment is a curved line, the corrected position for each point is determined based on a curve distance, the curve distance being a distance from each point to a starting point of the latest trajectory segment along the curved line; and

connecting the corrected positions to form the corrected trajectory.

9. A computer program product for trajectory estimation of a mobile platform, the computer program product comprising:

a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of;

generating an initial trajectory estimate for the mobile platform based on sensor data regarding a scene;

determining if a location is a new location or a revisited location using real-time feature point detection;

if the location is determined to be a revisited location, determining if trajectory correction is required; and

if trajectory correction is required, generating a corrected trajectory as a drift-free trajectory, the corrected trajectory being generated based on a previously recorded position at the location;

wherein in generating the corrected trajectory, the corrected trajectory is generated by performing operations of;

sampling equally-spaced points from a latest trajectory segment in a buffered trajectory;

determining corrected positions for each point in the latest trajectory segment, such that when the latest trajectory segment is a straight line, the corrected position for each point is determined based on a geometric distance, the geometric distance being a distance from each point to a starting point of the latest trajectory segment; and

connecting the corrected positions to form the corrected trajectory.

10. The computer program product as set forth in claim 9, further comprising instructions for causing a mobile platform to traverse a path that coincides with the drift-free trajectory.

11. The computer program product as set forth in claim 9, wherein if the location is determined to be a new location, storing coordinates for the new location in a trajectory buffer and designating the initial trajectory estimate as the drift-free trajectory.

12. The computer program product as set forth in claim 9, wherein if trajectory correction is not required, then designating the initial trajectory estimate as the drift-free trajectory.

13. The computer program product as set forth in claim 9, wherein generating the initial trajectory estimate for the mobile platform is performed using a simultaneous localization and mapping (SLAM) module.

14. The computer program product as set forth in claim 9, wherein the sensor data is generated using a sensor system selected from a group consisting of only a monocular camera, only a stereo camera, or a combination of a monocular camera with a depth image sensor.

15. The computer program product as set forth in claim 9, wherein in determining if trajectory correction is required, trajectory correction is required if a current position of the mobile platform at the location is greater than a predetermined distance threshold from a previously recorded position at the location.

16. A computer program product for trajectory estimation of a mobile platform, the computer program product comprising:

a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of;

generating an initial trajectory estimate for the mobile platform based on sensor data regarding a scene;

determining if a location is a new location or a revisited location using real-time feature point detection;

if the location is determined to be a revisited location, determining if trajectory correction is required; and

if trajectory correction is required, generating a corrected trajectory as a drift-free trajectory, the corrected trajectory being generated based on a previously recorded position at the location;

wherein in generating the corrected trajectory, the corrected trajectory is generated by performing operations of;

sampling equally-spaced points from a latest trajectory segment in a buffered trajectory;

determining corrected positions for each point in the latest trajectory segment, such that when the latest trajectory segment is a curved line, the corrected position for each point is determined based on a curve distance, the curve distance being a distance from each point to a starting point of the latest trajectory segment along the curved line; and

connecting the corrected positions to form the corrected trajectory.

17. A computer implemented method for trajectory estimation of a mobile platform, the method comprising an act of:

executing instructions encoded on a non-transitory computer-readable medium, such that upon execution, one or more processors perform operations of;

generating an initial trajectory estimate for the mobile platform based on sensor data regarding a scene;

determining if a location is a new location or a revisited location using real-time feature point detection;

if the location is determined to be a revisited location, determining if trajectory correction is required; and

if trajectory correction is required, generating a corrected trajectory as a drift-free trajectory, the corrected trajectory being generated based on a previously recorded position at the location;

wherein in generating the corrected trajectory, the corrected trajectory is generated by performing operations of;

sampling equally-spaced points from a latest trajectory segment in a buffered trajectory;

determining corrected positions for each point in the latest trajectory segment, such that when the latest trajectory segment is a straight line, the corrected position for each point is determined based on a geometric distance, the geometric distance being a distance from each point to a starting point of the latest trajectory segment; and

connecting the corrected positions to form the corrected trajectory.

18. The method as set forth in claim 17, further comprising an operation of causing a mobile platform to traverse a path that coincides with the drift-free trajectory.

19. The method as set forth in claim 17, wherein if the location is determined to be a new location, storing coordinates for the new location in a trajectory buffer and designating the initial trajectory estimate as the drift-free trajectory.

20. The method as set forth in claim 17, wherein if trajectory correction is not required, then designating the initial trajectory estimate as the drift-free trajectory.

21. The method as set forth in claim 17, wherein generating the initial trajectory estimate for the mobile platform is performed using a simultaneous localization and mapping (SLAM) module.

22. The method as set forth in claim 17, wherein the sensor data is generated using a sensor system selected from a group consisting of only a monocular camera, only a stereo camera, or a combination of a monocular camera with a depth image sensor.

23. The method as set forth in claim 17, wherein in determining if trajectory correction is required, trajectory correction is required if a current position of the mobile platform at the location is greater than a predetermined distance threshold from a previously recorded position at the location.

24. A computer implemented method for trajectory estimation of a mobile platform, the method comprising an act of:

executing instructions encoded on a non-transitory computer-readable medium, such that upon execution, one or more processors perform operations of;

generating an initial trajectory estimate for the mobile platform based on sensor data regarding a scene;

determining if a location is a new location or a revisited location using real-time feature point detection;

if the location is determined to be a revisited location, determining if trajectory correction is required; and

if trajectory correction is required, generating a corrected trajectory as a drift-free trajectory, the corrected trajectory being generated based on a previously recorded position at the location;

wherein in generating the corrected trajectory, the corrected trajectory is generated by performing operations of;

sampling equally-spaced points from a latest trajectory segment in a buffered trajectory;

determining corrected positions for each point in the latest trajectory segment, such that when the latest trajectory segment is a curved line, the corrected position for each point is determined based on a curve distance, the curve distance being a distance from each point to a starting point of the latest trajectory segment along the curved line; and

connecting the corrected positions to form the corrected trajectory.