Method and device for recognizing traffic signs for a vehicle

• US 10,223,600 B2
• Filed: 10/21/2013
• Issued: 03/05/2019
• Est. Priority Date: 11/06/2012
• Status: Active Grant

• Current Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Inventors: Fischer, Marc, van der Vegte, Peter, Moller, Ulrich
• Primary Examiner(s): Jones, Heather R
• Application Number:
  US14/427,427
• Publication Number:
  US 20150220795A1
• Time in Patent Office: 1,961 Days
• Field of Search: None
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 2013/9321 Velocity regulation, e.g. cruise control
• CPC Class Codes: G01S 2013/9322 using additional data, e.g. driver condition, road state or weather data
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/9329 cooperating with reflectors or transponders
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

The invention provides a method and a device for recognizing traffic signs by a sensor system for recording the surrounding environment for a vehicle, which includes at least one camera sensor and one radar sensor or LiDAR sensor. Based on the data of the at least one radar sensor or LiDAR sensor, at least one element of the presence of a traffic sign, or the size of a traffic sign, position of a traffic sign, distance and/or orientation of a traffic sign, bridge-like structure, position and/or type of the vehicle ahead, position of lane delimiting structures is obtained and used for traffic sign recognition.

Claims

1. A method for recognizing traffic signs by a sensor system for recording the surrounding environment for a vehicle, which comprises a processor in the vehicle, at least one camera sensor in the vehicle and one radar sensor or LiDAR sensor in the vehicle, the method comprising:

detecting, by the at least one radar sensor or LiDAR sensor, at least one of the following;

a) the presence of a traffic sign,

b) the size of a traffic sign,

c) position of a traffic sign,

d) distance and/or orientation of a traffic sign,

e) bridge-like structure,

f) position or type of the vehicle ahead,

g) position of lane delimiting structures;

allocating, by the processor, a probability to the presence of a traffic sign detected by the at least one radar sensor or LiDAR sensor, the probability being determined by correlating at least one of a)-g) with roadway location information provided by a navigation system;

capturing, by the at least one camera sensor, an image of the traffic sign;

determining, by the processor, plausibility of the traffic sign captured in the image being relevant to a roadway that the vehicle is traveling on based on the allocated probability of the information detected by the at least one radar sensor or LiDAR sensor;

determining a roadway to which the traffic sign belongs;

using, by the processor, the traffic sign when controlling the vehicle, when the processor determines that the roadway that the vehicle is traveling is the same as the roadway to which the traffic sign belongs; and

disregarding, by the processor, the traffic sign when controlling the vehicle, when the processor determines that the roadway that the vehicle is traveling is different from the roadway to which the traffic sign belongs.

2. The method according to claim 1, wherein the at least one piece of information obtained from radar or LiDAR data is used to check the plausibility of data provided by the camera sensor.

3. The method according to claim 1, further comprising image processing for the analysis of camera data in such a manner that an area where information according to a) e) has been recorded is specifically searched for traffic signs.

4. The method according to claim 1, further comprising it includes image processing for the analysis of camera data in such a manner that only an area where information according to a)-e) has been recorded is searched for traffic signs.

5. The method according to claim 1, wherein either of the elements of the information under c) or d) is analyzed to determine the area where a traffic sign is valid.

6. The method according to claim 1, further comprising image processing for the analysis of camera data in such a manner that an area where information according to f) has been recorded is not searched for traffic signs.

7. The method according to claim 1, wherein a traffic sign is recognized on the basis of data provided by a radar or LiDAR system, depending on at least one of the following parameters:

position of the object

distance

intensity of reflection

relative speed

dimensions of the object.

8. The method according to claim 1, wherein objects are recognized as potential traffic signs on the basis of data provided by a radar or LiDAR sensor, wherein these objects are allocated a probability value which indicates the probability that the relevant object is a traffic sign.

9. The method according to claim 1, wherein at least one element of the information under a)-e) is used to check the plausibility of data provided by the camera sensor.

10. The method according to claim 1, wherein either of the elements of the information under c) or d) is analyzed to determine the area where a traffic sign is valid when there are several traffic lanes in the surrounding environment of the vehicle.

11. A device for a vehicle for recognizing traffic signs, comprising:

a radar or LiDAR sensor system;

a camera sensor system; and

a processor,

wherein the radar sensor or LiDAR sensor is configured to detect at least one of the following;

a) the presence of a traffic sign,

b) the size of a traffic sign,

c) position of a traffic sign,

d) distance and/or orientation of a traffic sign,

e) bridge-like structure,

f) position or type of the vehicle ahead,

g) position of lane delimiting structures,

wherein the camera sensor is configured to capture an image of the traffic sign, and

wherein the processor is configured to;

allocate a probability to the presence of a traffic sign detected by the radar sensor or LiDAR sensor, the probability being determined by correlating at least one of a)-g) with roadway location information provided by a navigation system, and

determine a plausibility of the traffic sign captured in the image being relevant to a roadway that the vehicle is traveling on based on the allocated probability of the information detected by the radar sensor or LiDAR sensor;

determine a roadway to which the traffic sign belongs;

use the traffic sign when controlling the vehicle, when the processor determines that the roadway that the vehicle is traveling is the same as the roadway to which the traffic sign belongs; and

disregard the traffic sign when controlling the vehicle, when the processor determines that the roadway that the vehicle is traveling is different from the roadway to which the traffic sign belongs.