Clear path detection using an example-based approach

• US 9,852,357 B2
• Filed: 06/30/2014
• Issued: 12/26/2017
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Carnegie Mellon University, GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: Carnegie Mellon University, GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Wu, Qi, Chen, Tsuhan
• Primary Examiner(s): Hung, Yubin
• Application Number:
  US14/319,841
• Publication Number:
  US 20140314279A1
• Time in Patent Office: 1,275 Days
• Field of Search: 382104, 382199, 382202, 701300-302
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method for detecting a clear path of travel for a vehicle using a current image generated by a camera includes defining an exemplary vanishing point for each of a plurality of sample images. Features within each of the plurality of sample images are identified, the current image generated by the camera is monitored, and features within the current image are identified. The current image is matched to at least one of the sample images based upon the identified features within the current image and the identified features within the plurality of sample images. A vanishing point for the current image is determined based upon the matching and the exemplary vanishing point for each of the matched sample images. The vanishing point for the current image is utilized to define a clear path of travel, and the clear path of travel is utilized to navigate the vehicle.

Claims

1. A method for detecting a clear path of travel for a vehicle using a current image generated by a camera, the method comprising:

during an off-line training process;

defining an exemplary vanishing point for each of a plurality of sample images, each of the plurality of sample images comprising a different combination of geometric layout, lighting conditions, camera perspectives, traffic conditions and camera lens radial distortion;

identifying features within each of the plurality of sample images, the features including a ground portion, a horizon, and any objects within each image;

determining a feature value for each of the identified features within each of the plurality of sample images;

classifying the identified features within each of the plurality of sample images as a clear path or an obstacle; and

storing the identified features within each of the plurality of sample images in a refined database; and

within at least one on-vehicle processing module including memory;

monitoring the current image generated by the camera;

identifying features within the current image, the features including a ground portion, a horizon, and any objects within the image;

determining a feature value for each of the identified features within the current image;

selecting candidate sample images from the plurality of sample images that most closely resemble the current image, comprising comparing the feature values of the sample images to the feature values of the current image and selecting as the candidate sample images a predetermined number of the plurality of sample images whose feature values are in nearest spatial proximity to the feature values of the current image;

determining a vanishing point for the current image based upon the exemplary vanishing points within said candidate sample images;

utilizing the vanishing point for the current image to define a clear path of travel, comprising;

filtering out features in the current image above the vanishing point;

identifying a clear path of travel below the vanishing point of the current image based upon the identified features classified as corresponding to clear path within at least one of said candidate sample images; and

utilizing the clear path of travel to navigate the vehicle.

2. The method of claim 1, wherein determining the vanishing point for the current image based upon the exemplary vanishing points within said candidate sample images comprises averaging the exemplary vanishing points within said candidate sample images.

3. The method of claim 2, wherein identifying a clear path of travel below the vanishing point of the current image based upon the identified features classified as corresponding to clear path within at least one of said candidate sample images comprises inferring the clear path of travel by fusing the identified features classified as corresponding to clear path from all candidate sample images.

4. The method of claim 3, wherein spatial proximity between the feature values of the sample images and the feature value of the current image is given by:

5. The method of claim 2, wherein spatial proximity between the feature values of the sample images and the feature value of the current image is given by:

6. The method of claim 1, wherein determining the vanishing point for the current image based upon the exemplary vanishing points within said candidate sample images comprises determining the vanishing point for the current image as the exemplary vanishing point within the candidate sample image whose feature values are in nearest spatial proximity to the feature values of the current image.

7. The method of claim 6, wherein identifying a clear path of travel below the vanishing point of the current image based upon the identified features classified as corresponding to clear path within at least one of said candidate sample images comprises inferring the clear path of travel by fusing the identified features classified as corresponding to clear path from all candidate sample images.

8. The method of claim 7, wherein spatial proximity between the feature values of the sample images and the feature value of the current image is given by:

9. The method of claim 6, wherein spatial proximity between the feature values of the sample images and the feature value of the current image is given by:

10. The method of claim 1, wherein identifying a clear path of travel below the vanishing point of the current image based upon the identified features classified as corresponding to clear path within at least one of said candidate sample images comprises inferring the clear path of travel by fusing the identified features classified as corresponding to clear path from all candidate sample images.

11. The method of claim 10, wherein spatial proximity between the feature values of the sample images and the feature value of the current image is given by:

12. The method of claim 1, wherein spatial proximity between the feature values of the sample images and the feature value of the current image is given by: