VERIFICATION MODULE SYSTEM AND METHOD FOR MOTION-BASED LANE DETECTION WITH MULTIPLE SENSORS

• US 20190063945A1
• Filed: 08/22/2017
• Published: 02/28/2019
• Est. Priority Date: 08/22/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: Tusimple
• Inventors: LIU, SIYUAN, WANG, MINGDONG, HOU, XIAODI
• Granted Patent: US 10,816,354 B2
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/3658 Lane guidance
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

A method of lane detection for a non-transitory computer readable storage medium storing one or more programs is disclosed. The one or more programs include instructions, which when executed by a computing device, cause the computing device to perform the following steps comprising: generating a ground truth associated with lane markings expressed in god'"'"'s view; receiving features from at least one of a hit-map image and a fitted lane marking, wherein the hit-map image includes a classification of pixels that hit a lane marking, and the fitted lane marking includes pixels optimized based on the hit-map image; and training a confidence module based on the features and the ground truth, the confidence module configured to determine on-line whether a fitted lane marking is reasonable, using parameters that express a lane marking in an arc.

Claims

1. A method of lane detection for a non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by a computing device, cause the computing device to perform the following steps comprising:

generating a ground truth associated with lane markings expressed in god'"'"'s view;

receiving features from at least one of a hit-map image and a fitted lane marking, wherein the hit-map image includes a classification of pixels that hit a lane marking, and the fitted lane marking includes pixels optimized based on the hit-map image; and

training a confidence module based on the features and the ground truth, the confidence module configured to determine on-line whether a fitted lane marking is reasonable, using parameters that express a lane marking in an arc.

2. The method according to claim 1, wherein generating a ground truth includes:

generating a labeled lane marking by annotating a lane marking, using the parameters.

3. The method according to claim 1 further comprising:

training a lane detection algorithm by using the ground truth; and

generating a hit-map image for a current view based on the lane detection algorithm.

4. The method according to claim 3 further comprising:

generating a fitted lane marking based on the hit-map image and a lane template that includes features of a view immediately previous to the current view.

5. The method according to claim 4, wherein generating a fitted lane marking includes:

optimizing, based on priors or constraints, the lane template to obtain a local optimal.

6. The method according to claim 1 further comprising:

determining that a confidence level of the fitted lane marking is reasonable, using the parameters; and

outputting the fitted lane marking as a predicted lane marking.

7. The method according to claim 6 further comprising:

comparing the fitted lane marking having a reasonable confidence level against the ground truth; and

determining a failure pattern associated with the fitted lane marking when a comparison result indicates that the fitted lane marking fails to match the ground truth.

8. The method according to claim 1 further comprising:

determining that a confidence level of the fitted lane is unreasonable, using the parameters; and

rejecting the fitted lane marking.

9. The method according to claim 8 further comprising:

comparing the fitted lane marking having an unreasonable confidence level against the ground truth; and

determining a failure pattern associated with the fitted lane marking when a comparison result indicates that the fitted lane marking matches the ground truth.

10. The method according to claim 1 further comprising:

refining the confidence module by adding additional labeled data in training the confidence module.

11. A system for lane detection, the system comprising:

an internet server, comprising;

an I/O port, configured to transmit and receive electrical signals to and from a client device;

a memory;

one or more processing units; and

one or more programs stored in the memory and configured for execution by the one or more processing units, the one or more programs including instructions for;

generating a ground truth associated with lane markings expressed in god'"'"'s view;

receiving features from at least one of a hit-map image and a fitted lane marking, wherein the hit-map image includes a classification of pixels that hit a lane marking, and the fitted lane marking includes pixels optimized based on the hit-map image; and

training a confidence module based on the features and the ground truth, the confidence module configured to determine on-line whether a fitted lane marking is reasonable, using parameters that express a lane marking in an arc.

12. The system according to claim 11, wherein generating a ground truth includes:

generating a labeled lane marking by annotating a lane marking, using the parameters.

13. The system according to claim 11 further comprising:

training a lane detection algorithm by using the ground truth; and

generating a hit-map image for a current view based on the lane detection algorithm.

14. The system according to claim 13 further comprising:

generating a fitted lane marking based on the hit-map image and a lane template that includes features of a view immediately previous to the current view.

15. The system according to claim 14, wherein generating a fitted lane marking includes:

optimizing, based on priors or constraints, the lane template to obtain a local optimal.

16. The system according to claim 11 further comprising:

determining that a confidence level of the fitted lane marking is reasonable, using the parameters; and

outputting the fitted lane marking as a predicted lane marking.

17. The system according to claim 16 further comprising:

comparing the fitted lane marking having a reasonable confidence level against the ground truth; and

determining a failure pattern associated with the fitted lane marking when a comparison result indicates that the fitted lane marking fails to match the ground truth.

18. The system according to claim 11 further comprising:

determining that a confidence level of the fitted lane is unreasonable, using the parameters; and

rejecting the fitted lane marking.

19. The system according to claim 18 further comprising:

comparing the fitted lane marking having an unreasonable confidence level against the ground truth; and

determining a failure pattern associated with the fitted lane marking when a comparison result indicates that the fitted lane marking matches the ground truth.

20. The system according to claim 11 further comprising:

refining the confidence module by adding additional labeled data in training the confidence module.