CLEAR PATH DETECTION USING A VANISHING POINT

• US 20100097455A1
• Filed: 10/19/2009
• Published: 04/22/2010
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Carnegie Mellon University, GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: Carnegie Mellon University, GM Global Technology Operations Incorporated (Motors Liquidation Co. GUC Trust)
• Inventors: Wu, Qi, Zhang, Wende, Chen, Tsuhan
• Granted Patent: US 8,487,991 B2
• Time in Patent Office: Days
• US Class Current: 348/119
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method for estimating a vanishing point in a roadway using a current image generated by a camera on a vehicle includes defining an exemplary vanishing point for each of a plurality of sample images, identifying features within each of the plurality of sample images, monitoring the current image generated by the camera, identifying features within the current image, matching the current image to at least one of the sample images based upon the identified features within the current image and the identified features within the plurality of sample images, determining a vanishing point based upon the matching and the exemplary vanishing points for each of the matched sample images, and utilizing the vanishing point to navigate the vehicle.

Claims

1. A method for estimating a vanishing point in a roadway using a current image generated by a camera on a vehicle, the method comprising:

defining an exemplary vanishing point for each of a plurality of sample images;

identifying features within each of the plurality of sample images;

monitoring the current image generated by the camera;

identifying features within the current image;

matching the current image to at least one of the sample images based upon the identified features within the current image and the identified features within the plurality of sample images;

determining a vanishing point based upon the matching and the exemplary vanishing points for each of the matched sample images; and

utilizing the vanishing point to navigate the vehicle.

2. The method of claim 1, further comprising:

utilizing the vanishing point to determine a clear path of travel for the vehicle; and

wherein utilizing the vanishing point to navigate the vehicle comprises utilizing the clear path of travel to navigate the vehicle.

3. The method of claim 2, further comprising determining a horizon line based upon the exemplary vanishing points;

and

wherein utilizing the vanishing point to determine the clear path of travel for the vehicle comprises determining the clear path of travel based upon an image area below the horizon line.

4. The method of claim 2, further comprising:

filtering a plurality of image areas on the current image using a plurality of linear filters to obtain a filter response;

determining a dominant orientation of the filter response for each of the image areas;

determining an orientation ray based upon the dominant orientation for each of the image areas; and

determining a quantity of intersections among image areas and the orientation rays for each of the image areas.

5. The method of claim 4, further comprising:

determining clear path image areas based upon the quantity of intersections among image areas and the orientation rays.

6. The method of claim 5, further comprising:

utilizing a clear path detection analysis to identify image areas on the current image corresponding to clear path based upon the quantity of intersections among image areas and the orientation rays, wherein image areas surrounded by clear path image areas correspond to lane markers, and wherein image areas neighboring clear path image areas on a first side and not clear path image areas on an opposite side correspond to lane boundaries.

7. The method of claim 6, wherein image areas intersected by less than a predetermined threshold of orientation rays correspond to a clear path of travel for the vehicle.

8. The method of claim 6, wherein image areas intersected by greater than a predetermined threshold of orientation rays correspond to features in the current image.

9. The method of claim 1, wherein determining the vanishing point based upon the matching comprises:

initializing positions of the exemplary vanishing points in a feature space;

for each of the exemplary vanishing points, estimating a probability that the exemplary vanishing point corresponds to the vanishing point in the current image;

converging the exemplary vanishing points based upon the estimated probability; and

merging vanishing points based upon distance between exemplary vanishing points.

10. The method of claim 9, further comprising:

removing exemplary vanishing points residing a predetermined threshold distance from all the other exemplary vanishing points.

11. A method for detecting a clear path of travel for a vehicle using an image generated by a camera, the method comprising:

analyzing the image through clear path detection analysis to determine a clear path of travel within the image;

estimating a vanishing point in the image;

combining the clear path of travel and the vanishing point to describe an enhanced clear path of travel; and

utilizing the enhanced clear path of travel to navigate the vehicle.

12. The method of claim 11, wherein estimating a vanishing point in the image comprises:

defining an exemplary vanishing point for each of a plurality of sample images generated by the camera;

identifying features within each of the plurality of sample images;

monitoring a current image generated by the camera;

identifying features within the current image;

matching the current image to at least one of the sample images based upon the identified features within the current image and the identified features within the plurality of sample images; and

determining a vanishing point based upon the matching and the exemplary vanishing points for each of the matched sample images.

13. The method of claim 12, wherein determining the vanishing point based upon the matching comprises:

initializing positions of the exemplary vanishing points in a feature space;

for each of the exemplary vanishing points, estimating a probability that the exemplary vanishing point corresponds to the vanishing point in the current image;

converging the exemplary vanishing points based upon the estimated probability; and

merging vanishing points based upon distance between exemplary vanishing points.

14. The method of claim 13, further comprising:

removing exemplary vanishing points residing a predetermined threshold distance from all the other exemplary vanishing points.

15. The method of claim 12, wherein matching the current image to at least one of the sample images based upon the identified features within the current image and the identified features within the plurality of sample images comprises:

comparing a feature value for the current image to feature values of the plurality of sample images; and

selecting a predetermined number of matched sample images corresponding to feature values in nearest spatial proximity to the feature value of the current image.

16. The method of claim 12, wherein determining the vanishing point based upon the matching and the exemplary vanishing points for each of the matched sample images comprises:

inferring the vanishing point based upon the defined exemplary vanishing points for each of the matched sample images.

17. The method of claim 12, wherein identifying features within each of the plurality of sample images is executed offline.

18. The method of claim 12, wherein each of the plurality of sample images comprises a different combination of geometric layouts, lighting conditions, and camera perspectives.

19. The method of claim 11, wherein the analyzing the image through clear path detection analysis comprises utilizing an analysis selected from the group consisting of a patch-based analysis, a pixel-based analysis, an example-based analysis, a segmentation-based analysis, and a hierarchical-based analysis.

20. The method of claim 11, further comprising:

utilizing the clear path detection analysis to determine an enhanced vanishing point estimate.

21. The method of claim 20, wherein utilizing clear path detection analysis to determine the enhanced vanishing point estimate comprises:

defining texture-rich portions of the image as support patches useful to defining the enhanced vanishing point estimate; and

utilizing the support patches to determine the enhanced vanishing point estimate.

22. The method of claim 20, wherein utilizing clear path detection analysis to determine the enhanced vanishing point estimate comprises:

defining texture-rich portions of the image as support patches useful to defining the enhanced vanishing point estimate; and

utilizing the support patches to determine the enhanced vanishing point estimate;

wherein combining the clear path of travel and the vanishing point to describe the enhanced clear path of travel comprises utilizing the enhanced vanishing point to determine the enhanced clear path of travel; and

wherein the method further comprises performing a plurality of iterations of determining the enhanced vanishing point estimate and determining the enhanced clear path of travel.