Clear path detection with patch smoothing approach

• US 8,890,951 B2
• Filed: 10/19/2009
• Issued: 11/18/2014
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Carnegie Mellon University, GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: Carnegie Mellon University, GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zhang, Wende, Wu, Qi, Chen, Tsuhan
• Primary Examiner(s): Winder, Patrice
• Assistant Examiner(s): WIDHALM, ANGELA M
• Application Number:
  US12/581,781
• Publication Number:
  US 20100097457A1
• Time in Patent Office: 1,856 Days
• Field of Search: 348/113, 348118-119, 348/135
• US Class Current: 348/119
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method for detecting a clear path of travel for a vehicle utilizing analysis of an image generated by a camera device located upon the vehicle includes monitoring the image, identifying through patch-based clear path detection analysis of the image a first patch within the image that indicates a not clear path, analyzing the first patch through patch smoothing, invalidating the first patch based upon the analyzing the first patch through patch smoothing, utilizing the invalidated first patch to define a clear path of travel for the vehicle, and utilizing the clear path of travel to navigate the vehicle.

Claims

1. Method for detecting a clear path of travel for a vehicle utilizing analysis of an image generated by a camera device located upon the vehicle, the method comprising:

monitoring the image;

identifying through patch-based clear path detection analysis of the image from a plurality of candidate patches a first patch within the image that indicates a not clear path, wherein the analysis includes a non-pixelated analysis comprising;

applying a feature detection filter to each candidate patch for generating a feature-based response,

comparing the feature-based response generated for each candidate patch to a predetermined threshold response, and

classifying each candidate patch as corresponding to one of a clear path and an interesting feature indicating an object upon the roadway based on the comparing;

determining whether one of the candidate patches is classified as corresponding to the interesting feature, and if patch is classified as corresponding to the interesting feature, identifying the candidate patch as the first patch;

analyzing the first patch through patch smoothing;

invalidating the first patch based upon the analyzing the first patch through patch smoothing;

utilizing the invalidated first patch to define the clear path of travel for the vehicle; and

utilizing the clear path of travel to navigate the vehicle.

2. The method of claim 1, wherein the patch smoothing comprises temporal smoothing.

3. The method of claim 2, wherein the temporal smoothing comprises comparing previous analyses of previous images to determine invalidity of the first patch.

4. The method of claim 3, wherein comparing the previous analyses comprises utilizing past indications of a clear path in corresponding areas of past images to determine the invalidity of the first patch.

5. The method of claim 2, wherein the temporal smoothing comprises comparing previous analyses of previous images to determine invalidity of the interesting feature.

6. The method of claim 5, wherein comparing the previous analyses comprises utilizing past indications of a clear path in corresponding areas of past images to determine the invalidity of the interesting feature.

7. The method of claim 1, wherein the patch smoothing comprises spatial smoothing.

8. The method of claim 7, wherein the spatial smoothing comprises comparing analysis of patches adjacent to the first patch to determine invalidity of the first patch.

9. The method of claim 8, wherein the comparing comprises utilizing indications of a clear path in the patches adjacent to the first patch to determine invalidity of the interesting feature.

10. The method of claim 9, wherein the utilizing indications of the clear path in the patches adjacent to the first patch comprises identifying similar characteristics in the adjacent patches to the first patch.

11. The method of claim 9, wherein the utilizing indications of the clear path in the patches adjacent to the first patch comprises identifying characteristics in the adjacent patches to the first patch leading to a clear path determination in the first patch.

12. The method of claim 7, wherein the patch smoothing further comprises temporal smoothing.

13. The method of claim 1, further comprising:

monitoring a subsequent image;

identifying through patch-based clear path detection analysis of the subsequent image a subsequent patch within the subsequent image indicating a not clear path corresponding to the first patch;

revalidating the first patch based upon the subsequent patch; and

utilizing the first patch and the subsequent patch to indicate a not clear path for use in navigating the vehicle.

14. The method of claim 1, further comprising:

monitoring a plurality of subsequent images;

identifying through patch-based clear path detection analysis of the subsequent images a plurality of subsequent patches within the subsequent images each indicating a not clear path corresponding to the first patch;

revalidating the first patch based upon the subsequent patches; and

utilizing the first patch and the subsequent patches to indicate a not clear path for use in navigating the vehicle.

15. Method for detecting a clear path of travel for a vehicle utilizing analysis of an image generated by a camera device located upon the vehicle, the method comprising:

monitoring the image;

identifying through patch-based clear path detection analysis of the image from a plurality of candidate patches a first patch within the image that indicates a not clear path, wherein the analysis includes a non-pixelated analysis comprising;

applying a feature detection filter to each candidate patch for generating a feature-based response,

comparing the feature-based response generated for each candidate patch to a predetermined threshold response, and

classify each candidate patch as corresponding to one of a clear path and an interesting feature indicating an object upon the roadway based on the comparing;

determining whether one of the candidate patches is classified as corresponding to the interesting feature, and if patch is classified as corresponding to the interesting feature, identifying the candidate patch as the first patch;

determining through patch smoothing that the first patch is invalid;

utilizing the invalidated first patch to define a clear path of travel for the vehicle; and

utilizing the clear path of travel to navigate the vehicle.

16. The method of claim 15, wherein the patch smoothing comprises temporal smoothing.

17. The method of claim 15, wherein the patch smoothing comprises spatial smoothing.

18. System for detecting a clear path of travel for a vehicle utilizing analysis of an image generated by a camera device located upon the vehicle, the system comprising:

the camera device; and

a processing module

monitoring the image,

identifying through patch-based clear path detection analysis of the image from a plurality of candidate patches a first patch within the image that indicates a not clear path, wherein the analysis includes a non-pixelated analysis comprising

applying a feature detection filter to each candidate patch for generating a feature-based response,

comparing the feature-based response generated for each candidate patch to a predetermined threshold response, and

classifying each candidate patch as corresponding to one of a clear path and an interesting feature indicating an object upon the roadway based on the comparing,

determining whether one of the candidate patches is classified as corresponding to the interesting feature, and if patch is classified as corresponding to the interesting feature, identifying the candidate patch as the first patch;

analyzing through patch smoothing the first patch,

invalidating the first patch based upon the analyzing the patch through patch smoothing,

utilizing the invalidated patch to define a clear path of travel for the vehicle, and

utilize the clear path of travel to aid navigation of the vehicle.

19. The system of claim 18, wherein the patch smoothing comprises one of temporal smoothing and spatial smoothing.