Device and method for recognizing traffic signs
First Claim
1. A process for recognizing and displaying traffic signs, comprising:
- analyzing and classifying image data from an image sensor in an information processing unit, on the basis of the results of the classification, storing image segments and/or a symbolic image representing image data in a memory unit, and displaying this by means of a display unit, wherein in the framework of the analysis in the information processing unit it is determined whether the image data from the image sensor contains one or more objects which, with sufficient confidence, are traffic signs, and subsequently supplies these objects to a further processing and classification comprising;
separating the image data of the objects, in the data processing, into traffic sign specific upper classes and lower classes, extracting, during the separation, appropriate class,specific characterizing data, supplying the class-specific characterizing data to separate classifications, replacing the class-specific characterizing data, if they have been correctly recognized with high confidence by the classification system, by appropriate image data representing the symbolic representation stored in a memory unit, or by the appropriate original image data originating from the image sensor, and synthesizing the image data resulting from this replacement into a synthetic image of a traffic sign, storing this image in the memory unit, and bringing it to display in the display unit.
2 Assignments
0 Petitions
Accused Products
Abstract
The invention concerns a process and a device, which make it possible to reliably, with the least amount of computing power, recognize traffic signs and to display these as memory aids for an observer. Image data of an image sensor are analyzed and classified in an information processing unit, and on the basis of the results classification image segments and/or symbolic image representing data stored in a memory unit are recorded in a memory unit and are displayed by means of a display unit. In the further processing of the image data, extracted objects are separated and classified into traffic sign specific upper classes and lower classes. Subsequently, a synthetic image of a traffic sign is generated, stored in a memory unit, and displayed by means of a display unit.
-
Citations
62 Claims
-
1. A process for recognizing and displaying traffic signs, comprising:
-
analyzing and classifying image data from an image sensor in an information processing unit, on the basis of the results of the classification, storing image segments and/or a symbolic image representing image data in a memory unit, and displaying this by means of a display unit, wherein in the framework of the analysis in the information processing unit it is determined whether the image data from the image sensor contains one or more objects which, with sufficient confidence, are traffic signs, and subsequently supplies these objects to a further processing and classification comprising;
separating the image data of the objects, in the data processing, into traffic sign specific upper classes and lower classes, extracting, during the separation, appropriate class,specific characterizing data, supplying the class-specific characterizing data to separate classifications, replacing the class-specific characterizing data, if they have been correctly recognized with high confidence by the classification system, by appropriate image data representing the symbolic representation stored in a memory unit, or by the appropriate original image data originating from the image sensor, and synthesizing the image data resulting from this replacement into a synthetic image of a traffic sign, storing this image in the memory unit, and bringing it to display in the display unit. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36)
a) first, only the characterizing data associated with the upper class of the object is supplied to a classifier, aa) upon successful classification, wherein the class is recognized correctly with a high degree of confidence, replacing the characterizing data with the appropriate symbolic representation of the upper class stored in the memory unit, ab) upon unsuccessful classification, wherein the class could not be correctly recognized with a high degree of confidence, replacing the characterizing data for the upper class and the charactering data for the lower class with the corresponding original image data originating from the image sensor, b) if the classification of the upper class was successful, then subsequently the characterizing data associated with the lower class of the object are supplied to a classifier, ba) upon successful classification, wherein the class is recognized with a high degree of confidence, the characterizing data are replaced by the appropriate symbolic representation of the lower class stored in the memory unit, bb) upon unsuccessful classification, wherein the class could not be correctly recognized with a high degree of confidence, the characterizing data for the lower class are substituted by the original image data from the image sensor.
-
-
4. A process according to claim 1, wherein the classifiers employed in the classification are trained using multiple learning samples.
-
5. A process according to claim 1, wherein the classifiers used in the classification make decisions on the basis of a comparison of the distance of the characterizing data supplied to them from the class-typical characterizing data sets stored in the memory unit.
-
6. A process according to claim 1, wherein, in the case that in the framework of the analysis of the image data from the image sensor a supplemental sign is recognized attached to the traffic sign, and when during the classification at least the upper class of the traffic sign is recognized with high confidence, this supplemental sign is displayed on the display unit.
-
7. A process according to claim 1, wherein in the case that in the framework of the classification at least the upper class of a traffic sign has been recognized, the respective image data which originated from the area of the environment of the traffic sign in which a supplemental sign is typically located is displayed by means of the display unit.
-
8. A process according to claim 7, wherein the display of the image data is subjected to a pre-classification, which rejects obviously useless image information and does not forward this to the display.
-
9. A process according to claim 1, wherein in the framework of the analysis in the information processing unit, during which it is determined whether an image contains one or more objects which with sufficiently high confidence are traffic signs, this is divided into two process steps, wherein
a) in a first process step image areas are determined which contain traffic signs with an above-average degree of confidence, and b) in a second process step objects possessing a predetermined shape are determined within these image areas by means of geometric analysis. -
10. A process according to claim 9, wherein said first process step is carried out on the basis of color values and/or spatial positions, and wherein said second process step is carried out by means of a correlation processes.
-
11. Process according to claim 9, wherein said shape is round or triangular.
-
12. A process according to claim 1, wherein said system is carried onboard a vehicle,
wherein a sequence of images is recorded of a recognized object as a vehicle drives once past the traffic sign, from which sequence respectively the best image is selected, wherein the selection is carried out on the basis of defined quality criteria, and only the synthesized traffic sign image generated on the basis of this best image is stored in the memory unit and displayed on the display unit. -
13. A process according to claim 12, wherein said defined quality criteria include size, contrast, and brightness of the object recognized to be a traffic sign.
-
14. A process according to claim 1, wherein during the combination of the image data into a synthesized image of a traffic sign, the respective image data, which corresponds to the image data originally originating from the image sensor, are processed for improvement with respect to a better recognizability, in such a manner that they are standardized with respect to at least one of:
-
a) size, b) contrast, c) intensity, and d) color shade.
-
-
15. A process according to claim 1, further comprising determining the position of one of the objects recognized to be a traffic sign by means of a position determining unit.
-
16. A process according to claim 1, wherein the display unit is so controlled by the information processing unit via a program for using a decision logic stored in the memory unit, that it displays the synthesized image of a traffic sign only until a new object is recognized as a traffic sign, at which time the simplified representation of the new traffic sign replaces the previous image.
-
17. A process according to claim 1, wherein the maximal duration of the display of the synthesized image of a traffic sign is adjusted by means of an adjustment unit included in the display unit.
-
18. A process according to claim 1, wherein the system is carried onboard a vehicle, wherein the traveled distance between the actual vehicle position and the position of the last recognized traffic sign of a particular class of traffic signs is determined via a program in the memory unit and with the aid of a device or sensor coupled to the information unit.
-
19. A process according to claim 18, wherein the determination of the distance occurs by means of a vehicle tachometer coupled to the information unit or a wheel sensor for determining the traveled distance and a clock signal producer.
-
20. A process according to claim 18, wherein the data transmission of the devices coupled to the information unit and the sensors for determining the traveled distance occur via a vehicle on-board data bus.
-
21. A process according to claim 18, wherein, via a program stored in the memory unit, a decision logic is controlled via which the information processing unit controls the display unit in such a manner that it displays the synthesized image of the traffic sign so long, until a predetermined traveled path stored in the memory unit is traveled, which is preferably class-specific.
-
22. A device according to claim 18, wherein in the memory unit a program is stored for application of a decision logic by means of which the information processing unit so controls the display unit, that the synthesized image of a traffic sign is display so long, until a travel distance preset in the memory unit has been traveled, which is preferably class specific.
-
23. A process according to claim 1, further comprising controlling a decision logic via a program stored in the memory unit by means of which the information processing unit directs the display unit in such a manner, that it displays the synthesized image of a traffic sign so long, until a new object is recognized as a traffic sign, optionally including recognizing whether the a traffic sign is of the same or a corresponding class, and replaces the previous one with the simplified representation of the new one.
-
24. A process according to claim 1, further comprising emitting an acoustic signal via an acoustic signal emitter within the display unit as soon as an object is recognized as a traffic sign, wherein said acoustic signal is optionally class-specific.
-
25. A process according to claim 1, wherein said system is carried onboard a vehicle, wherein by a suitable arrangement of one or more digital cameras within the digital image sensor, all relevant traffic signs for a vehicle along a traveled path are detected independent of the horizontal and/or vertical tilting of the vehicle path.
-
26. A process according to claim 25, wherein said digital cameras are monochromic.
-
27. A process according to claim 1, wherein the system for recognizing and displaying traffic signs utilizes information which originated from at least one map-based navigation system and/or a traffic information system.
-
28. A process according to claim 27, wherein the system for recognizing and displaying traffic signs communicates with the map-based navigation system and/or traffic information system via an on-board vehicle data bus.
-
29. A process according to claim 27, wherein the system for traffic sign recognition begins to operate at a higher level of performance when on the basis of map-based information it is recognized that the vehicle is passing through an area in which the recognition of traffic signs may be problematic.
-
30. A process according to claim 29, wherein within said problematic areas supplemental subdivided recognition processes are carried out, in such a manner that
a) in the area of community entrances and community exits there is a specific search for signs indicating entry of a community and departing of a community, b) in the area of traffic influencing facilities, changing signs or traffic lights, separately attention is paid to a change in the type and manner of the representation of the traffic sign, c) in an area, in which poor visibility due to fog or rain is present and a higher probability of contrast-poor images is likely, an increase in the contrast enhancement of the image data is effected. -
31. A process according to claim 27, wherein in a data memory unit integrated in the overall system, in addition to or complimentary to possibly already existing information, at least one of the type, the position, the condition, the visibility and the size of a recognized traffic sign is stored.
-
32. A process according to claim 31, wherein by means of a signal producer contained in the display unit, an acoustic or optical signal is emitted when a change with respect to the position or a change as to the presence or absence of a traffic sign occurs along a previously traveled road segment.
-
33. A process according to claim 27, wherein information from the map-based navigation system is drawn upon to determine the length of a road segment traveled by the vehicle.
-
34. A process according to claim 1, wherein a data memory unit is integrated in the overall system, in which, additionally or complimentary to possibly already existing information, at least one of the type, the position, the condition, the visibility and the size of a recognized traffic sign is recorded.
-
35. A process according to claim 34, wherein the display unit includes a signal emitter which emits an acoustic or optical signal when, while traveling along a path previously traveled at least once, a change with respect to the position or the presence or absence of a traffic sign occurs.
-
36. A process according to claim 1, wherein said system is carried onboard a vehicle, and wherein the information from the map-based navigation system is utilized in order to determine the distance of a road segment traveled by the vehicle.
-
37. A device for recognizing and displaying a traffic sign, comprising
a) an image sensor, b) an information processing unit, comprised of an analyzer, a supplemental processing unit and a classifier for processing image data supplied by the image sensor, c) a memory unit, which contains symbolic images representing image data, and in which image segments and/or symbolic images representing image data are stored by the information processing unit on the basis of the results of information supplied by the classifier, and d) a display unit, upon which the image data stored in the memory unit can be brought to display, wherein the analyzer is programmed to determine whether an image contains one or more objects which with high probability are traffic signs, and as a result thereof to supply this to the classifier, wherein that the supplemental processing unit includes a separator, which separates the image data supplied to the analyzer into traffic sign specific upper classes and lower classes and extracts class specific characterizing data, which are subsequently supplied to a classifier, wherein the classifier included in the information processing unit is constructed in such a manner, that the class-specific characterizing data, depending upon whether it has been recognized correctly by the classifier with high degree of confidence, is replaced by image data of representative symbolic images stored in the memory unit or the appropriate original image data originating from the image sensor, and wherein the information processing unit includes an element for combination of image data, which combines the image data resulting from this replacement into a synthetic image of a traffic sign and stores this image in the memory unit, and by means of the display unit causes this to be displayed.
-
61. A method for enhancing the safety of a vehicle in operation, said method comprising providing to the operator of said vehicle a memory aid, said memory aid, obtained by operating an onboard system designed to
analyze and classify image data from an image sensor in an information processing unit, on the basis of the results of the classification, store image segments and/or a symbolic image representing image data in a memory unit, and display this by means of a display unit, wherein in the framework of the analysis in the information processing unit it is determined whether the image data from the image sensor contains one or more objects which, with sufficient confidence, are traffic signs, and subsequently supplies these objects to a further processing and classification comprising: -
separating the image data of the objects, in the data processing, into traffic sign specific upper classes and lower classes, extracting, during the separation, appropriate class-specific characterizing data, supplying the class-specific characterizing data to separate classifications, replacing the class-specific characterizing data, if they have been correctly recognized with high confidence by the classification system, by appropriate image data representing the symbolic representation stored in a memory unit, or by the appropriate original image, data originating from the image sensor, and synthesizing the image data resulting from this replacement into a synthetic image of a traffic sign, storing this image in the memory unit, and bringing it to display in the display unit. - View Dependent Claims (62)
-
Specification