Localization using road markings

• US 9,488,483 B2
• Filed: 05/17/2013
• Issued: 11/08/2016
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Ranganathan, Ananth, Wu, Tao
• Primary Examiner(s): Nguyen, John Q
• Assistant Examiner(s): Odeh, Nadeem
• Application Number:
  US13/896,827
• Publication Number:
  US 20140343842A1
• Time in Patent Office: 1,271 Days
• Field of Search: 701/472
• US Class Current: 1/1
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/30 Map- or contour-matching

Abstract

One or more embodiments of techniques or systems for creating a road marking classification template and vehicle localization using road markings are provided herein. A road marking classification template database includes templates of training images taken from different navigation environments. A training image with a road marking can be rectified and enhanced. Corners can be calculated for the road marking using the rectified or enhanced image. Locations can be determined for the corners and stored as part of a template. Similarly, a runtime image can also be rectified, enhanced, boosted, etc. Additionally, corners can be calculated for the runtime image and matched against corners of templates from the template database. A location or a pose of a vehicle can be determined using the match. In this way, vehicle localization is provided such that drift or other issues associated with GPS, such as occlusion, are mitigated, for example.

Claims

1. A method for vehicle localization, comprising:

receiving a runtime image of a navigation environment associated with a perspective from a vehicle, wherein the runtime image comprises one or more runtime road markings;

transforming the runtime image into a rectified runtime image by reducing perspective distortion associated with the runtime image, wherein the rectified runtime image comprises a rectified view of the one or more runtime road markings;

enhancing the rectified runtime image by extracting one or more enhanced runtime road marking regions from the rectified runtime image;

calculating one or more runtime corners for the one or more runtime road markings based on the one or more enhanced runtime road marking regions;

matching the one or more runtime corners with one or more template corners from one or more templates, wherein each template corner has at least a Global Positioning System (GPS) location corresponding to a GPS coordinate system and a pixel location corresponding to a pixel coordinate system within at least one rectified training image associated with a template, and the matching is performed to determine a GPS coordinate for a camera capturing the runtime image based at least upon pixel locations and GPS locations of at least one matched template corner; and

determining a location of the vehicle or a pose of the vehicle based on the matching,

wherein the receiving, transforming, enhancing, calculating, matching, and determining is implemented via a processing unit.

2. The method of claim 1, wherein the transforming the runtime image comprises performing an inverse perspective transform on the runtime image.

3. The method of claim 1, further comprising performing shadow boosting on at least a portion of the rectified runtime image by at least equalizing a blue spectrum across the rectified view of the one or more runtime road markings within the rectified runtime image.

4. The method of claim 1, further comprising enhancing the rectified runtime image based on an intensity of the rectified runtime image.

5. The method of claim 1, further comprising calculating the one or more runtime corners based on features from accelerated segment test (FAST) corner detection.

6. The method of claim 1, further comprising:

determining a supplemental location of the vehicle; and

matching the one or more runtime corners based on the supplemental location of the vehicle.

7. The method of claim 1, further comprising:

classifying the one or more runtime corners based on the one or more templates; and

matching the one or more runtime corners to the one or more template corners based on the classification.

8. The method of claim 1, wherein the matching the one or more runtime corners to the one or more template corners comprises putative matching or structural matching.

9. The method of claim 1, further comprising:

determining the location of the vehicle or the pose of the vehicle based on the one or more additional runtime corners.

10. The method of claim 9, further comprising determining the one or more additional runtime corners based on a snake algorithm.

11. The method of claim 1, further comprising:

receiving a tilt angle of a ground plane associated with the vehicle; and

transforming the runtime image into the rectified runtime image based on the tilt angle of the ground plane.

12. A system for vehicle localization, comprising:

a road marking detection component configured to receive a runtime image of a navigation environment associated with a perspective from a vehicle, the runtime image comprises one or more runtime road markings;

an image processing component configured to;

transform the runtime image into a rectified runtime image by reducing perspective distortion associated with the runtime image, wherein the rectified runtime image comprises a rectified view of the one or more runtime road markings; and

enhance the rectified runtime image by extracting one or more enhanced runtime road marking regions from the rectified runtime image;

a corner component configured to calculate one or more runtime corners for the one or more runtime road markings based on the one or more enhanced runtime road marking regions;

a matching component configured to match the one or more runtime corners with one or more template corners from one or more templates to determine a GPS coordinate for a camera capturing the runtime image based at least upon pixel locations and GPS locations of at least one matched template corner, wherein each template corner has at least a Global Positioning System (GPS) location corresponding to a GPS coordinate system and a pixel location corresponding to a pixel coordinate system within at least one rectified training image associated with a template; and

a localization component configured to determine a location of the vehicle or a pose of the vehicle based on the match.

13. The system of claim 12, further comprising a shadow component configured to even out shadow effects on the one or more runtime road markings by equalizing a blue spectrum across the rectified view of the one or more runtime road markings within the rectified runtime image.

14. The system of claim 12, further comprising a global positioning system (GPS) component configured to receive a supplemental location of the vehicle, the matching component configured to match the one or more runtime corners with one or more template corners based on the supplemental location of the vehicle.

15. The system of claim 12, further comprising a gyroscope component configured to measure a tilt angle of a ground plane associated with the vehicle, the image processing component configured to transform the runtime image into the rectified runtime image based on the tilt angle of the ground plane.

16. A non-transitory computer-readable storage medium comprising computer executable instructions, which when executed via a processing unit on a computer performs acts, comprising:

transforming a runtime image of a navigation environment into a rectified runtime image, the rectified runtime image comprising a rectified view of one or more runtime road markings, the runtime image associated with a perspective from a vehicle;

mitigating an effect of a shadow on at least a portion of a runtime road marking of the one or more runtime road markings;

enhancing the rectified runtime image by extracting one or more enhanced runtime marking regions from the rectified runtime image;

calculating one or more runtime corners for the one or more runtime road markings based on the one or more enhanced runtime road marking regions derived from the rectified runtime image; and

determining at least one of a location of the vehicle or a pose of the vehicle based on matching the one or more runtime corners with one or more template corners from one or more templates, wherein each template corner has at least a Global Positioning System (GPS) location corresponding to a GPS coordinate system and a pixel location corresponding to a pixel coordinate system within at least one rectified training image associated with a template, and the matching is performed to determine a GPS coordinate for a camera capturing the runtime image based at least upon pixel locations and GPS locations of at least one matched template corner.

17. The non-transitory computer-readable storage medium of claim 16, wherein the transforming the runtime image comprises performing an inverse perspective transform on the runtime image.

18. The non-transitory computer-readable storage medium of claim 16, wherein the mitigating comprises performing shadow boosting on the rectified runtime image by equalizing a blue spectrum across the rectified view of the one or more runtime road markings within the rectified runtime image.

19. The non-transitory computer-readable storage medium of claim 16, further comprising calculating one or more runtime corners based on features from accelerated segment test (FAST) corner detection.