Road marking recognition system

• US 8,600,655 B2
• Filed: 08/04/2006
• Issued: 12/03/2013
• Est. Priority Date: 08/05/2005
• Status: Expired due to Fees

• Current Assignee: Aisin AW Company Limited (Aisin Corp.), Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Aisin AW Company Limited (Aisin Corp.), Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Ishikawa, Tomoaki, Nakamura, Masaki, Aisaka, Osamu, Kanba, Motoki, Nakamura, Motohiro
• Primary Examiner(s): NGUYEN, HIEP VAN
• Application Number:
  US11/922,779
• Publication Number:
  US 20090088978A1
• Time in Patent Office: 2,678 Days
• Field of Search: 701/300
• US Class Current: 701/300
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G05D 1/0244 using reflecting strips
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096725 where the received information generates an automatic action on the vehicle control
• CPC Class Codes: G08G 1/09675 where a selection from the received information takes place in the vehicle
• CPC Class Codes: G08G 1/096783 where the origin of the information is a roadside individual element
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Systems, methods, and programs capture an image of a periphery of a vehicle. The systems, methods, and programs, store predetermined patterns representing conditions of road markings formed on a road surface, detect a road marking on the basis of the captured image, and compare the detected road marking to the predetermined patterns. If the detected road marking matches one of the predetermined patterns, the systems, methods, and programs execute a predetermined action associated with the matching predetermined pattern.

Claims

1. A road marking recognition system for a vehicle, comprising:

a camera disposed on the vehicle that captures an image of a periphery of the vehicle;

a memory that stores;

locations of different types of road markings formed on road surfaces; and

a plurality of wear patterns associated with each type of road marking, each wear pattern indicating a wear condition of the associated road marking and each stored wear pattern comprising measurement starting points in a plurality of locations; and

a controller that;

detects a road marking on the basis of the image captured by the camera;

compares the detected road marking to the stored wear patterns;

if the detected road marking matches one of the stored wear patterns;

determines that the detected road marking is the stored type of road marking that is associated with the matching stored wear pattern;

selects one of the measurement starting points from the plurality of measurement starting points on the basis of the matching stored wear pattern;

calculates a distance between the vehicle and the detected road marking based on the selected measurement starting point; and

executes a predetermined action associated with the stored type of road marking that is associated with the matching stored wear pattern and the calculated distance.

2. The road marking recognition system according to claim 1, further comprising:

a current position detector that detects a current location of the vehicle;

wherein the controller;

extracts the plurality of wear patterns associated with a type of road marking located within a predetermined range of the detected current location from the memory;

compares the extracted plurality of wear patterns to the detected road marking; and

if one of the extracted wear patterns matches the detected road marking, executes the predetermined action associated with the stored type of road marking that is associated with the matching stored wear pattern.

3. The road marking recognition system according to claim 1, wherein the stored wear patterns comprise approaching pedestrian crossing wear patterns that are associated with a type of road marking indicating an approaching pedestrian crossing, each of the approaching pedestrian crossing wear patterns indicating various types of deterioration of the type of road marking indicating the approaching pedestrian crossing.

4. The road marking recognition system according to claim 1, wherein the stored wear patterns comprise main-access road boundary wear patterns that are associated with a type of road marking representing a traffic lane boundary line positioned at a connection portion between a main road and an access road of an expressway, each of the main-access road boundary wear patterns representing various types of deterioration of the type of road marking representing the traffic lane boundary line positioned at the connection portion between the main road and the access road of the expressway.

5. The road marking recognition system according to claim 4, wherein the stored wear patterns comprise traffic lane boundary line wear patterns that are associated with a type of road marking representing a traffic lane boundary line, each of the traffic lane boundary line wear patterns representing the type of road marking representing the traffic lane boundary line which has been divided in a length direction due to various types of deterioration.

6. The road marking recognition system according to claim 5, further comprising:

a current position detector that detects a current location of the vehicle;

wherein the controller;

extracts the plurality of wear patterns associated with a type of road marking located within a predetermined range of the detected current location from the memory;

recognizes a shape of the road marking formed on the road surface on the basis of the captured image; and

when the recognized shape includes double lines that are parallel to each other via a predetermined gap and when the extracted plurality of wear patterns do not include information of a road marking containing double lines, detects the recognized shape as the type of road marking representing the traffic lane boundary line.

7. The road marking recognition system according to claim 1, wherein the camera is a rear camera located on a rear of the vehicle that captures an image of a rear periphery of the vehicle.

8. The road marking recognition system according to claim 7, further comprising a second camera located on a front of the vehicle that captures an image of a front periphery of the vehicle.

9. The road marking recognition system according to claim 8, wherein the controller:

detects the road marking based on the image of the front periphery of the vehicle;

calculates a distance from the vehicle to the detected road marking; and

determines a timing for capturing an image of the road marking with the rear camera based on the calculated distance.

10. A road marking recognition method for use with a vehicle, comprising:

capturing an image of a periphery of the vehicle with a camera;

accessing a memory containing;

locations of different types of road markings formed on road surfaces; and

a plurality of wear patterns associated with each type of road marking, each wear pattern indicating a wear condition of the associated road marking and each stored wear pattern comprising measurement starting points in a plurality of locations;

detecting a road marking on the basis of the image captured by the camera;

comparing the detected road marking to the stored wear patterns;

if the detected road marking matches one of the stored wear patterns, determining that the detected road marking is the stored type of road marking that is associated with the matching stored wear pattern;

selecting one of the measurement starting points from the plurality of measurement starting points on the basis of the matching stored wear pattern; and

calculating a distance between the vehicle and the detected road marking based on the selected measurement starting point; and

executing a predetermined action associated with the stored type of road marking that is associated with the matching stored wear pattern and the calculated distance.

11. The road marking recognition method according to claim 10, further comprising:

detecting a current location of the vehicle;

extracting the plurality of wear patterns associated with a type of road marking located within a predetermined range of the detected current location from the memory;

comparing the extracted plurality of wear patterns to the detected road marking; and

if one of the extracted wear patterns matches the detected road marking, executing the predetermined action associated with the stored type of road marking that is associated with the matching stored wear pattern.

12. The road marking recognition method according to claim 10, wherein the stored wear patterns comprise approaching pedestrian crossing wear patterns that are associated with a type of road marking indicating an approaching pedestrian crossing, each of the approaching pedestrian crossing wear patterns indicating various types of deterioration of the type of road marking indicating the approaching pedestrian crossing.

13. The road marking recognition method according to claim 10, wherein the stored wear patterns comprise main-access road boundary wear patterns that are associated with a type of road marking representing a traffic lane boundary line positioned at a connection portion between a main road and an access road of an expressway, each of the main-access road boundary wear patterns representing various types of deterioration of the type of road marking representing the traffic lane boundary line positioned at the connection portion between the main road and the access road of the expressway.

14. The road marking recognition method according to claim 13, wherein the stored wear patterns comprise traffic lane boundary line wear patterns that are associated with a type of road marking representing a traffic lane boundary line, each of the traffic lane boundary line wear patterns representing the type of road marking representing the traffic lane boundary line which has been divided in a length direction due to various types of deterioration.

15. The road marking recognition method according to claim 13, further comprising:

detecting a current location of the vehicle;

extracting the plurality of wear patterns associated with a type of road marking located within a predetermined range of the detected current location from the memory;

recognizing a shape of the road marking formed on the road surface on the basis of the captured image; and

when the recognized shape includes double lines that are parallel to each other via a predetermined gap and when the extracted information does not include information of a road marking containing double lines, detecting the recognized shape as the type of road marking representing the traffic lane boundary line.

16. The road marking recognition method according to claim 10, wherein the captured image an image of a rear periphery of the vehicle captured by a camera located on a rear of the vehicle.

17. The road marking recognition method according to claim 16, further comprising capturing an image of a front periphery of the vehicle with a second camera located on a front of the vehicle.

18. The road marking recognition method according to claim 17, further comprising:

detecting the road marking based on the image of the front periphery of the vehicle;

calculating a distance from the vehicle to the detected road marking; and

determining a timing for capturing an image of the road marking with the rear camera based on the calculated distance.

19. A non-transitory computer-readable storage medium storing a computer-executable program usable to recognize road markings, the program comprising:

instructions for capturing an image of a periphery of a vehicle;

instructions for accessing a memory containing;

locations of different types of road markings formed on road surfaces; and

a plurality of wear patterns associated with each type of road marking, each wear pattern indicating a wear condition of the associated road marking and each stored wear pattern comprising measurement starting points in a plurality of locations;

instructions for detecting a road marking on the basis of the image captured by the camera;

instructions for comparing the detected road marking to the stored wear patterns;

instructions for, if the detected road marking matches one of the stored wear patterns, determines that the detected road marking is the stored type of road marking that is associated with the matching stored wear pattern;

instructions for selecting one of the measurement starting points from the plurality of measurement starting points on the basis of the matching stored wear pattern; and

instructions for calculating a distance between the vehicle and the detected road marking based on the selected measurement starting point; and

instructions for, executing a predetermined action associated with the stored type of road marking that is associated with the matching stored wear pattern and the calculated distance.