PAVEMENT MARKER RECOGNITION DEVICE, PAVEMENT MARKER RECOGNITION METHOD AND PAVEMENT MARKER RECOGNITION PROGRAM

• US 20100259609A1
• Filed: 12/03/2008
• Published: 10/14/2010
• Est. Priority Date: 12/05/2007
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Takahashi, Katsuhiko
• Granted Patent: US 9,123,242 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096791 where the origin of the information is another vehicle
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A pavement marker recognition device for recognizing a pavement marker accurately is provided, the device being capable of recognizing an object even when it is difficult to be recognized only from tones or color information. A pavement marker candidate extraction unit extracts an image area having a feature similar to that of a pavement marker from an image captured by a car-mounted camera as a pavement marker candidate area, a road shape analysis unit analyzes the road shape from the image captured by the car-mounted camera, and a pavement marker verification unit determines whether the road shape analyzed by the road shape analysis unit and the pavement marker candidate area extracted by the pavement marker candidate extraction unit are in a correspondence relationship indicated by a road shape and pavement marker correspondence table, and based on the determination result, recognizes the pavement marker candidate area as a true pavement marker.

Claims

1. A pavement marker recognition device for recognizing a marker on a pavement based on a captured image, the device comprising:

a correspondence table data storage unit which stores, in advance, a road shape and pavement marker correspondence table showing correspondence relationships between a plurality of types of pavement markers and road shapes;

a pavement marker candidate extraction unit which extracts an image area having a feature similar to a feature of a pavement maker from a captured image as a pavement marker candidate area;

a road shape analysis unit which analyzes and recognizes a road shape from the captured image; and

a pavement marker verification unit which determines whether the road shape recognized by the road shape analysis unit and the pavement marker candidate area extracted by the pavement marker candidate extraction unit are in a correspondence relationship shown in the road shape and pavement marker correspondence table, and based on a determination result, recognizes the pavement marker candidate area as a true pavement marker.

2. The pavement marker recognition device, according to claim 1, wherein

the road shape analysis unit and the pavement marker candidate extraction unit respectively use images captured from different angles,

the pavement marker recognition device includes a road shape storage unit which stores information of the road shape analyzed by the road shape analysis unit, and

the pavement marker verification unit acquires the information of the road shape by reading the information from the road shape storage unit.

3. The pavement marker recognition device, according to claim 2, further comprising a travel amount measurement unit which measures a travel amount of a vehicle provided with a car-mounted camera, wherein

the road shape analysis unit modifies the road shape information stored in the road shape storage unit based on the measured travel amount, and additionally writes information of the road shape analyzed from an image of the car-mounted camera.

4. The pavement marker recognition device, according to claim 3, wherein

the pavement marker candidate extraction unit extracts the pavement marker candidate area from a captured image by a car-mounted camera facing downward of back of the vehicle.

5. The pavement marker recognition device, according to claim 4, wherein

the road shape analysis unit analyzes a road shape from a captured image by a car-mounted camera of at least one of a car-mounted camera facing front of the vehicle, a car-mounted camera facing a side of the vehicle, and an all-round car-mounted camera.

6. The pavement marker recognition device, according to claim 1, further comprising a pavement marker recognition unit in place of the pavement marker candidate extraction unit and the pavement marker verification unit, wherein

the pavement marker recognition unit has a function of identifying a type of a pavement marker corresponding to a road shape analyzed by the road shape analysis unit based on the road shape and pavement marker correspondence table, and recognizing a pavement marker of the identified type from the image captured by the car-mounted camera.

7. The pavement marker recognition device, according to claim 1, wherein

the road shape and pavement marker correspondence table is information indicating at least one of road shapes of places where the pavement markers are present, road shapes of places where the pavement markers are absent, and both.

8. A pavement marker recognition method for recognizing a marker on a pavement based on a captured image, the method comprising:

a road shape analyzing step for analyzing and recognizing a road shape from the captured image;

a pavement marker candidate area extracting step for extracting an image area having a feature similar to a feature of a pavement marker from the captured image as a pavement marker candidate area; and

a pavement marker verifying step for determining whether or not the road shape analyzed in the road shape analyzing step and the pavement marker candidate area extracted in the pavement marker candidate area extraction step are in a correspondence relationship shown in a road shape and pavement marker correspondence table previously stored in a storage section, and based on a determination result, recognizing the pavement marker candidate area as a true pavement marker.

9. The pavement marker recognition method, according to claim 8, wherein

in the road shape analyzing step and the pavement marker candidate area extracting step, images captured from different angles are used respectively,

a road shape storing step for storing information of the road shape analyzed in the road shape analyzing step in the storage section is performed, and

in the pavement marker verifying step, the information of the road shape is read from the storage section.

10. The pavement marker recognition method, according to claim 8, further comprising:

before the road shape analyzing step, a travel amount measuring step for measuring a travel amount of the vehicle provided with the car-mounted camera, wherein

in the road shape analyzing step, the road shape information previously stored in the storage section is modified based on the measured travel amount, and information of the road shape analyzed from the car-mounted camera is additionally written in the storage section.

11. The pavement marker recognition method, according to claim 10, wherein

in the pavement marker candidate area extracting step, the pavement marker candidate area is extracted from a captured image by a car-mounted camera facing downward of back of the vehicle.

12. The pavement marker recognition method, according to claim 11, wherein

in the road shape analyzing step, the road shape is analyzed from a captured image of a car-mounted camera of at least one of a car-mounted camera facing front of the vehicle, a car-mounted camera facing a side of the vehicle, and an all-round car-mounted camera.

13. The pavement marker recognition method, according to claim 8, further comprising,

after the road shape analyzing step, a pavement marker recognizing step in place of the pavement marker candidate area extracting step and the pavement marker verifying step, wherein

in the pavement marker recognizing step, a type of a pavement marker corresponding to the road shape analyzed by the road shape analysis unit is identified based on the road shape and pavement marker correspondence table, and a pavement marker of the identified type is recognized from the image captured by the car-mounted camera.

14. A non-transitory computer readable recording medium storing a pavement marker recognition program for controlling recognition of a marker on a pavement based on a captured image, the program causing a computer to perform:

a road shape analyzing function for analyzing and recognizing a road shape from the captured image;

a pavement marker candidate area extracting function for extracting an image area having a feature similar to a feature of a pavement marker from the captured image as a pavement marker candidate area; and

a pavement marker verifying function for determining whether or not the road shape analyzed by the road shape analyzing function and the pavement marker candidate area extracted by the pavement marker candidate area extracting function are in a correspondence relationship shown in a road shape and pavement marker correspondence table previously stored in a storage section, and based on a determination result, recognizing the pavement marker candidate area as a true pavement marker.

15. The non-transitory computer readable recording medium storing the pavement marker recognition program, according to claim 14, wherein

in the road shape analyzing function and the pavement marker candidate area extracting function, images captured from different angles are used respectively, and

in addition to a road shape storing function for storing information of the road shape analyzed by the road shape analyzing function in the storage section,

the program causes the computer to perform the pavement marker verifying function as a function of reading the information of the road shape is from the storage section, determining whether or not a road shape indicated by this information and the pavement marker candidate area extracted by a pavement marker candidate extraction unit is in a correspondence relationship shown in the road shape and pavement marker correspondence table, and based on a determination result, recognizing the pavement marker candidate area as a true pavement marker.

16. The non-transitory computer readable recording medium storing the pavement marker recognition program, according to claim 15, wherein

in addition to a travel amount measuring function for measuring a travel amount of the vehicle,

the program causes the computer to perform the road shape analyzing function as a function for modifying the road shape information previously stored in the storage section based on the measured travel amount, and additionally writing information of the road shape analyzed from an image captured by the car-mounted camera in the storage section.

17. The non-transitory computer readable recording medium storing the pavement marker recognition program, according to claim 16, wherein

the pavement marker candidate area extracting function is a function of extracting the pavement marker candidate area from a captured image by a car-mounted camera facing downward of back of the vehicle.

18. The non-transitory computer readable recording medium storing the pavement marker recognition program, according to claim 17, wherein

the road shape analyzing function is a function of analyzing the road shape from a captured image of a car-mounted camera of at least one of a car-mounted camera facing front of the vehicle, a car-mounted camera facing a side of the vehicle, and an all-round car-mounted camera.

19. The non-transitory computer readable recording medium storing the pavement marker recognition program, according to claim 14, wherein

in place of the pavement marker candidate area extracting function and the pavement marker verifying function, the program causes the computer to perform a pavement marker recognizing function of reading, from the storage section, and referring to the road shape and pavement marker correspondence table, identifying a type of a pavement marker corresponding to the road shape analyzed by the road shape analyzing function, and recognizing a pavement marker of the identified type from the image captured by the car-mounted camera.

20. The non-transitory computer readable recording medium storing the pavement marker recognition program, according to claim 14, wherein

the road shape and pavement marker correspondence table is information indicating at least one of road shapes of places where the pavement markers are present, road shapes of places where the pavement markers are absent, and both.