Method for recognizing traffic signs

• US 9,436,879 B2
• Filed: 07/10/2012
• Issued: 09/06/2016
• Est. Priority Date: 08/04/2011
• Status: Active Grant

• Current Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Conti Temic Microelectronic GmbH (Continental AG)
• Inventors: Zobel, Matthias
• Primary Examiner(s): Bali, Vikkram
• Assistant Examiner(s): Mangialaschi, Tracy
• Application Number:
  US14/126,098
• Publication Number:
  US 20140119605A1
• Time in Patent Office: 1,519 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G08G 1/09623 Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle G08G1/0967 takes precedence

Abstract

For recognizing traffic signs that include a main sign and an associated additional sign, a method involves using a camera to record image data of an environment outside a vehicle, evaluating the image data to detect the presence and the class of the main sign and to detect the presence and partially classify the content of the additional sign by pattern recognition, performing text recognition to read and interpret text of the additional sign, and comparing the recognized text with actual current situational information to determine whether the main sign is currently applicable.

Claims

1. A method of traffic sign recognition in a driver assistance system of a motor vehicle, said method comprising the step:

a) using a camera, recording image data of at least one image of a vehicle environment outside of the motor vehicle driving along a road;

and further comprising the following steps in an automatic machine interpretation process performed by said driver assistance system;

b) from the image data, recognizing the presence and class of at least one main sign present in the vehicle environment;

c) from the image data, recognizing the presence of at least one additional sign assigned to the main sign;

d) classifying the additional sign using a pattern recognition method to an extent possible;

e) reading and interpreting and thereby recognizing text of the additional sign using a text recognition method; and

f) in a relevance assessment unit of the driver assistance system, automatically assessing a relevance of the main sign in an automatic machine-based relevance assessment, in which the recognized text of the additional sign is compared to situational information regarding the motor vehicle, a current operation of the motor vehicle, or a condition of the vehicle environment.

2. The method according to claim 1, wherein the main sign is assessed as not relevant based on the relevance assessment if the situational information does not fall within a scope of the recognized text of the additional sign.

3. The method according to claim 2, further comprising performing a check whether the relevance assessment for an expected change in situational information will remain constant or not, and if not, then repeating the relevance assessment when the situational information changes.

4. The method according to claim 1, wherein the at least one additional sign comprises two additional signs which comprise two similar signs detected on two opposite sides of the road, and further comprising performing a consistency check of respective text found on the two additional signs.

5. The method according to claim 1, further comprising taking into account, for the relevance assessment, other information from a digital map, a positioning device and/or other vehicle sensors.

6. A method of traffic sign recognition in a driver assistance system of a motor vehicle, said method comprising the step:

a) using a camera, recording image data of at least one image of a vehicle environment outside of the motor vehicle driving along a road;

and further comprising the following steps in an automatic machine interpretation process performed by said driver assistance system;

b) from the image data, recognizing the presence and class of at least one main sign present in the vehicle environment;

c) from the image data, recognizing the presence of at least one additional sign assigned to the main sign, wherein the additional sign includes text;

d) reading and interpreting and thereby recognizing the text of the additional sign using a text recognition method;

e) deciding whether the additional sign includes other information, and if so, then classifying the other information using a pattern recognition method; and

f) in a relevance assessment unit of the driver assistance system, automatically assessing a relevance of the main sign in an automatic machine-based relevance assessment, in which the recognized text of the additional sign is compared to situational information regarding the motor vehicle, a current operation of the motor vehicle, or a condition of the vehicle environment.

7. The method according to claim 6, wherein the main sign is assessed as not relevant based on the relevance assessment if the situational information does not fall within a scope of the recognized text of the additional sign.

8. The method according to claim 7, further comprising performing a check whether the relevance assessment for an expected change in situational information will remain constant or not, and if not, then repeating the relevance assessment when the situational information changes.

9. The method according to claim 6, wherein the at least one additional sign comprises two additional signs which comprise two similar signs detected on two opposite sides of the road, and further comprising performing a consistency check of respective text found on the two additional signs.

10. The method according to claim 6, further comprising taking into account, for the relevance assessment, other information from a digital map, a positioning device and/or other vehicle sensors.

11. A method of traffic sign recognition in a driver assistance system of a motor vehicle driving along a road, wherein the driver assistance system includes a camera, said method comprising the following steps performed in the driver assistance system:

a) with the camera, recording image data of at least one image of a vehicle environment outside of the vehicle that includes a main sign and an auxiliary sign associated with the main sign;

b) evaluating the image data and thereby recognizing a presence of the main sign and classifying main sign information of the main sign;

c) evaluating the image data and thereby recognizing a presence of the auxiliary sign;

d) performing pattern recognition on the image data and thereby recognizing and classifying any available pattern information of the auxiliary sign;

e) performing text recognition on the image data and thereby reading and interpreting textual information of the auxiliary sign;

f) obtaining situational information regarding the motor vehicle, a current operation of the motor vehicle, or a condition of the vehicle environment;

g) automatically assessing a relevance of the main sign by comparing the situational information with the interpreted textual information of the auxiliary sign and with any available classified pattern information of the auxiliary sign, and thereby determining whether that the main sign is relevant only when the situational information falls within a scope of the interpreted textual information of the auxiliary sign and the available classified pattern information of the auxiliary sign; and

h) in response to the step g) having determined that the main sign is relevant, activating a driver assistance feature of the driver assistance system based on the classified main sign information of the main sign.

12. The method according to claim 11, wherein the textual information of the auxiliary sign includes at least one of letters and numbers.

13. The method according to claim 11, wherein the pattern information of the auxiliary sign includes at least one non-textual symbol.

14. The method according to claim 11, wherein the pattern information of the auxiliary sign includes letters or numbers matching a fixed pre-specified pattern.

15. The method according to claim 11, wherein the driver assistance feature comprises searching for an upcoming further sign in subsequent image data recorded by the camera at a subsequent time, in response to the situational information at the subsequent time falling within the scope of the interpreted textual information and the available classified pattern information of the auxiliary sign at a previous time.

16. The method according to claim 11, wherein, if the situational information changes over time, then at a subsequent time when the situational information has changed, again comparing and determining whether the changed situational information at the subsequent time falls within the scope of the interpreted textual information and the available classified pattern information of the auxiliary sign from a previous time.

17. The method according to claim 11, wherein the situational information comprises a distance the vehicle has traveled since passing the main sign and the auxiliary sign.

18. The method according to claim 11, wherein the situational information comprises at least one of an outside temperature and an indication whether the road is wet.

19. The method according to claim 11, wherein the situational information comprises at least one of an indication whether a turn signal of the vehicle is activated and a position of a lane of the road in which the vehicle is traveling.