ROAD DEPARTURE WARNING SYSTEM

• US 20120050489A1
• Filed: 08/30/2011
• Published: 03/01/2012
• Est. Priority Date: 08/30/2010
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Gupta, Rakesh, Ranganathan, Ananth, Lim, Jongwoo
• Granted Patent: US 9,077,958 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 13/00 Stereoscopic video systems; Multi-view video systems; Details thereof

Abstract

The prevention of vehicle accidents is targeted. A road texture model is created based on a vehicle camera image. An initial vehicle location estimate is determined, and map imagery is obtained based on this location estimate. A refined vehicle location is determined using visual egomotion. In particular, 3D features of the vehicle image and the retrieved map imagery are identified and aligned. A map image is selected based on this alignment, and the location associated with the map image is modified by a displacement between the selected map image and the vehicle image to produce a refined vehicle location. A road boundary model is created based on the road texture model and the refined vehicle location, and a road departure model is created based on the road boundary model and vehicle odometry information. The operator of the vehicle is warned of a road departure based on the road departure model.

Claims

1. A method of providing a road departure warning alert in a vehicle comprising:

creating a road texture model based on a vehicle image received from a vehicle camera;

retrieving map imagery based on an initial vehicle location estimate;

determining a refined vehicle location using visual egomotion based on the vehicle image and the map imagery;

creating a road boundary model describing the edges of the road on which the vehicle is located based on the road texture model and the refined vehicle location;

creating a road departure model based on the road boundary model and vehicle odometry information; and

warning a vehicle operator based on the road departure model.

2. The method of claim 1, wherein the vehicle image comprises an image of the road on which the vehicle is located in the direction in which the vehicle is moving.

3. The method of claim 1, wherein creating a road texture model further comprises:

processing the vehicle image using a filter bank to produce a texton for each pixel in the vehicle image;

classifying each texton in vehicle image using a set of classifiers; and

smoothing the classified textons based on the classification of nearby textons.

4. The method of claim 3, further comprising:

training the set of classifiers using the filter bank and a set of training images with pre-classified pixels.

5. The method of claim 3, wherein the set of classifiers comprise random forest classifiers.

6. The method of claim 3, further comprising:

updating the set of classifiers based on the classification of textons representing pixels with known classifications.

7. The method of claim 1, wherein the retrieved map imagery comprises one of:

satellite map imagery, aerial map imagery, and street view map imagery.

8. The method of claim 1, wherein the initial vehicle location estimate comprises location coordinates received from a GPS receiver.

9. The method of claim 1, wherein determining a refined vehicle location using visual egomotion further comprises:

identifying 3D features in the vehicle image and the each map image in the retrieved map imagery;

aligning the 3D features between the vehicle image and each map image;

selecting a map image based on the alignment of 3D features;

determining a displacement between a location associated with the selected map image and a location associated with the vehicle image; and

applying the displacement to the location associated with the selected map image to determine a refined vehicle location.

10. The method of claim 9, wherein identifying 3D features comprises using edge detectors to detect edges in images.

11. The method of claim 9, wherein aligning the 3D features between the vehicle image and each map image comprises:

projecting the 3D features onto the ground plane; and

comparing the ground plane location of the vehicle image'"'"'s 3D features to the ground plane location of each map image'"'"'s 3D features.

12. The method of claim 9, wherein the selected map image comprises the map image with the most common 3D features.

13. The method of claim 1, wherein creating a road boundary model comprises:

identifying the edges of the road based on an image of the road;

identifying the width of the road based on the identified road edges;

reducing the identified width of the road by a safety margin; and

fitting one or more curves to the road edges and reduced width of the road to create a road boundary model.

14. The method of claim 13, wherein the image of the road comprises one or more of the vehicle image, a satellite map image of the road, an aerial map image of the road, and a street view map image of the road.

15. The method of claim 13, further comprising:

identifying one or both of the center line of the road and the shape of the road; and

fitting one or more curves to the center line of the road or the shape of the road.

16. The method of claim 1, wherein creating a road departure model comprises identifying the probability of a road departure based on the road boundary model and the vehicle odometry information.

17. The method of claim 16, wherein warning a vehicle operator based on the road departure model comprises warning the vehicle operator when the identified probability of a road departure exceeds a pre-determined threshold.

18. A system of providing a road departure warning alert in a vehicle comprising:

a road texture module configured to create a road texture model based on a vehicle image received from a vehicle camera;

a map module configured to retrieve map imagery based on an initial vehicle location estimate;

a location module configured to determine a refined vehicle location using visual egomotion based on the vehicle image and the map imagery;

a road boundary module configured to create a road boundary model describing the edges of the road on which the vehicle is located based on the road texture model and the refined vehicle location;

a road departure module configured to create a road departure model based on the road boundary model and vehicle odometry information; and

a warning module configured to warn a vehicle operator based on the road departure model.

19. A method of refining a vehicle location estimate comprising:

creating a road texture model based on a vehicle image received from a vehicle camera;

retrieving map imagery based on an initial vehicle location estimate;

determining a refined vehicle location using visual egomotion based on the vehicle image and the map imagery.

20. The method of claim 19, wherein the vehicle comprises a GPS receiver, and wherein the initial vehicle location estimate comprises a GPS location estimate received from the GPS receiver.