LOCALIZATION USING ROAD MARKINGS

• US 20140343842A1
• Filed: 05/17/2013
• Published: 11/20/2014
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Ranganathan, Ananth, Wu, Tao
• Granted Patent: US 9,488,483 B2
• Time in Patent Office: Days
• US Class Current: 701/472
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/30 Map- or contour-matching

Abstract

One or more embodiments of techniques or systems for creating a road marking classification template and vehicle localization using road markings are provided herein. A road marking classification template database includes templates of training images taken from different navigation environments. A training image with a road marking can be rectified and enhanced. Corners can be calculated for the road marking using the rectified or enhanced image. Locations can be determined for the corners and stored as part of a template. Similarly, a runtime image can also be rectified, enhanced, boosted, etc. Additionally, corners can be calculated for the runtime image and matched against corners of templates from the template database. A location or a pose of a vehicle can be determined using the match. In this way, vehicle localization is provided such that drift or other issues associated with GPS, such as occlusion, are mitigated, for example.

Claims

1. A method for vehicle localization, comprising:

receiving a runtime image of a navigation environment associated with a perspective from a vehicle, the runtime image comprising one or more runtime road markings;

transforming the runtime image into a rectified runtime image by reducing perspective distortion associated with the runtime image, the rectified runtime image comprising a rectified view of the one or more runtime road markings;

enhancing the rectified runtime image by extracting one or more enhanced runtime road marking regions from the rectified runtime image;

calculating one or more runtime corners for the one or more runtime road markings based on the one or more enhanced runtime road marking regions;

matching the one or more runtime corners with one or more template corners from one or more templates; and

determining a location of the vehicle or a pose of the vehicle based on the matching,

wherein the receiving, transforming, enhancing, calculating, matching, and determining is implemented via a processing unit.

2. The method of claim 1, wherein transforming the runtime image comprises performing an inverse perspective transform on the runtime image.

3. The method of claim 1, comprising performing shadow boosting on at least a portion of the rectified runtime image by equalizing a blue spectrum across the rectified view of the one or more runtime road markings within the rectified runtime image.

4. The method of claim 1, wherein enhancing the rectified runtime image comprises extracting the one or more enhanced runtime road marking regions based on a maximally stable external region (MSER) algorithm.

5. The method of claim 1, comprising enhancing the rectified runtime image based on an intensity of the rectified runtime image.

6. The method of claim 1, comprising calculating the one or more runtime corners based on features from accelerated segment test (FAST) corner detection.

7. The method of claim 1, comprising:

determining a supplemental location of the vehicle; and

matching the one or more runtime corners based on the supplemental location of the vehicle.

8. The method of claim 1, further comprising:

classifying the one or more runtime corners based on the one or more templates; and

matching the one or more runtime corners based on the classification.

9. The method of claim 1, wherein matching the one or more runtime corners comprises putative matching or structural matching.

10. The method of claim 1, comprising:

determining one or more additional runtime corners based on the one or more template corners; and

determining the location of the vehicle or the pose of the vehicle based on the one or more additional runtime corners.

11. The method of claim 10, comprising determining the one or more additional runtime corners based on a snake algorithm.

12. The method of claim 1, comprising:

receiving a tilt angle of a ground plane associated with the vehicle; and

transforming the runtime image into the rectified runtime image based on the tilt angle of the ground plane.

13. A system for vehicle localization, comprising:

a road marking detection component configured to receive a runtime image of a navigation environment associated with a perspective from a vehicle, the runtime image comprising one or more runtime road markings;

an image processing component configured to;

transform the runtime image into a rectified runtime image by reducing perspective distortion associated with the runtime image, the rectified runtime image comprising a rectified view of the one or more runtime road markings; and

enhance the rectified runtime image by extracting one or more enhanced runtime road marking regions from the rectified runtime image;

a corner component configured to calculate one or more runtime corners for the one or more runtime road markings based on the one or more enhanced runtime road marking regions;

a matching component configured to match the one or more runtime corners with one or more template corners from one or more templates; and

a localization component configured to determine a location of the vehicle or a pose of the vehicle based on the match.

14. The system of claim 13, comprising a shadow component configured to equalize a blue spectrum across the rectified view of the one or more runtime road markings within the rectified runtime image.

15. The system of claim 13, comprising a global positioning system (GPS) component configured to receive a supplemental location of the vehicle, the matching component configured to match based on the supplemental location of the vehicle.

16. The system of claim 13, comprising a gyroscope component configured to measure a tilt angle of a ground plane associated with the vehicle, the image processing component configured to transform the runtime image into the rectified runtime image based on the tilt angle of the ground plane.

17. A computer-readable storage medium comprising computer-executable instructions, which when executed via a processing unit on a computer performs acts, comprising:

transforming a runtime image of a navigation environment into a rectified runtime image, the rectified runtime image comprising a rectified view of one or more runtime road markings, the runtime image associated with a perspective from a vehicle;

calculating one or more runtime corners for the one or more runtime road markings based on one or more enhanced runtime road marking regions derived from the rectified runtime image; and

determining at least one of a location of the vehicle or a pose of the vehicle based on matching the one or more runtime corners with one or more template corners from one or more templates.

18. The computer-readable storage medium of claim 17, wherein transforming the runtime image comprises performing an inverse perspective transform on the runtime image.

19. The computer-readable storage medium of claim 17, comprising performing shadow boosting on the rectified runtime image by equalizing a blue spectrum across the rectified view of the one or more runtime road markings within the rectified runtime image.

20. The computer-readable storage medium of claim 17, comprising calculating one or more runtime corners based on features from accelerated segment test (FAST) corner detection.