Device, method and computer program for identifying a traffic sign in an image

• US 8,170,340 B2
• Filed: 12/18/2007
• Issued: 05/01/2012
• Est. Priority Date: 12/18/2006
• Status: Active Grant

• Current Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Klefenz, Frank
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): HEIDEMANN, JASON E
• Application Number:
  US12/519,594
• Publication Number:
  US 20100067805A1
• Time in Patent Office: 1,596 Days
• Field of Search: None
• US Class Current: 382/187
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

A device for identifying a traffic sign in an image includes a Hough transformer implemented to identify a plurality of line sections running in different directions through the image in the image or in an edge image derived from same. The device further includes a shape detector implemented to detect a predefined shape in the image or in the edge image derived from same based on the identified line sections. The device apart from that includes a pattern identifier implemented to select an image section corresponding to the detected predefined shape based on the detected predefined shape and to identify a traffic sign based on the selected image section.

Claims

1. A device for identifying a traffic sign in an image, comprising:

a Hough transformer which is implemented to identify a plurality of line sections running through the image in different directions in the image or in an edge image derived from same;

a shape detector which is implemented to detect a shape corresponding to a traffic sign in the image or in an edge image derived from same based on the identified line sections; and

a pattern identifier which is implemented to select an image section corresponding to the detected shape whose shape corresponds to the shape detected by the shape detector based on the detected shape and to identify a traffic sign based on the selected image section using a comparative image pattern,

wherein the pattern identifier is implemented to adapt the section of the image and the comparative image pattern to each other with respect to a shape by mapping;

wherein the Hough transformer is further implemented to identify circular arc segments or elliptical arc segments in the image or in the edge image derived from same;

wherein the device further comprises an ellipse detector which is implemented to detect a location of an ellipse in the image or in the edge image derived from same based on the identified circular arc segments or elliptical arc segments;

wherein the Hough transformer is implemented to determine extreme points of circular arc segments or of elliptical arc segments, and

wherein the ellipse detector is implemented to select at least a first set of four extreme points from the extreme points determined by the Hough transformer, and a second set of four extreme points from the extreme points determined by the Hough transformer, wherein the first set of four extreme points is different from the second set of four extreme points,

to determine parameters of a first fit ellipse by the four extreme points of the first set of extreme points,

to determine a deviation of the four extreme points of the first set of extreme points from the first fit ellipse,

to determine parameters of a second fit ellipse by the four extreme points of the second set of extreme points,

to determine a deviation of the four extreme points of the second set of extreme points from the fit ellipse, and

to determine, using the deviation of the four extreme points of the first set of extreme points from the first fit ellipse and using the deviation of the four extreme points of the second set of extreme points from the second fit ellipse, whether the four extreme points of the first set of extreme points belong to an ellipse in the image, and whether the four extreme points of the second set of extreme points belong to an ellipse in the image.

2. The device according to claim 1, wherein the pattern identifier is implemented to select the image section based on the detected shape so that its shape corresponds to the shape detected by the shape detector.

3. The device according to claim 1, wherein the shape detector is implemented to detect the predetermined shape in the image or in the edge image derived from same based on a relative location of the identified line sections.

4. The device according to claim 1, wherein the shape detector is implemented to select a subset of selected identified line sections from an entirety of identified line sections,

to determine relative location parameters for the selected identified line sections, which describe a relative location of the selected identified line sections to each other or a relative location of intersections of the selected identified line sections, and

to decide, based on the relative location parameters, whether the selected identified line sections describe a predetermined comparative shape.

5. The device according to claim 4, wherein the shape detector is implemented to compare the relative location parameters for the selected identified line sections to relative location parameters of comparative shapes to be identified, to acquire a quantitative description of a deviation between the relative location parameters for the selected identified line sections and the relative location parameters of the comparative shapes to be identified, and

to detect a shape corresponding to the comparative shape in the image or in an edge image derived from same, when the relative location parameters for the selected identified line sections deviate from the relative location parameters of the comparative shapes by at most a predetermined, maximum admissible deviation.

6. The device according to claim 1, wherein the Hough transformer is implemented to identify the circular arc segments or elliptical arc segments in the image or in an edge image derived from same, and

wherein the shape detector is implemented to describe a description of an object in the form of a list of adjacent identified straight line sections and of identified circular arc segments or elliptical arc segments.

7. The device according to claim 6, wherein the list comprises information about a length, a direction, a position or a curvature radius of the identified straight line section and the identified circular arc segments or elliptical arc segments.

8. The device according to claim 6, wherein the shape detector is implemented to compare the information of the list to a description of a comparative shape to be identified,

to determine a measure for a difference between the object described by the list and the comparative shape, and

to decide based on the measure for the difference whether the object described by the list corresponds to the comparative shape.

9. The device according to claim 6, wherein the shape detector is implemented to set up a plurality of descriptions of objects comprising different combinations of adjacent, identified straight line sections, identified circular arc segments or identified elliptical arc segments, and

to select selected descriptions of objects corresponding to at least one comparative shape from the plurality of descriptions of objects.

10. The device according to claim 6, wherein the shape detector is implemented to detect two identified straight line sections, circular arc segments or elliptical arc segments as adjacent, identified straight line sections, circular arc segments or elliptical arc segments, when a distance between the identified straight line sections, circular arc segments or elliptical arc segments is smaller than a predetermined minimum distance.

11. The device according to claim 1, wherein the device comprises an edge detector which is implemented to detect edges in the image, and to generate an edge image based on the image;

wherein the Hough transformer is implemented to identify circular arc segments or elliptical arc segments in the edge image, to acquire a location information about the identified circular arc segments or elliptical arc segments, and

to identify a plurality of straight line sections running in different directions through the edge image in the edge image to acquire a location information about the identified straight line sections;

wherein the device further comprises an ellipse detector which is implemented to detect a location of an ellipse in the image based on the location information about the identified circular arc segments or elliptical arc segments and to provide location information about the detected ellipse;

wherein the shape detector is implemented to determine relative location parameters based on a plurality of selected identified line sections from a entirety of identified line sections, describing a relative location of the selected identified line sections with regard to each other,

to compare the relative location parameters of the selected identified line sections to relative location parameters of comparative shapes to be identified, wherein the comparative shapes describe contours of traffic signs,

to detect a comparative shape in the edge image, when the relative location parameters of the selected identified line sections deviate from the relative location parameters of the comparative shapes to be identified by no more than a predetermined maximum admissible deviation, and

to acquire the location information about the comparative shape detected in the edge image based on the location information about the identified straight line sections;

wherein the pattern identifier is implemented to select an image section of the image corresponding to the identified ellipse or the identified comparative shape depending on the fact whether in the edge image an ellipse or a comparative shape was identified, and

to compare the image section or a mapped image section resulting from a mapping of the image section to one or several comparative images, to provide information with regard to the fact whether the image section or the mapped image section shows a traffic sign depending on how well the image section matches the at least one comparative image.

12. The device according to claim 1, wherein the pattern identifier comprises a writing detector,

wherein the writing detector comprises a character description generator which is implemented to acquire a character description describing the location of the identified straight line sections, the identified circular arc segments or the identified elliptical arc segments based on information provided by the Hough transformer about straight line sections, identified circular arc segments or identified elliptical arc segments passing through the image;

wherein the writing detector comprises a database comparator which is implemented to compare the character description to a plurality of comparative character descriptions to which character codes are associated to provide a character code of a detected character as a result of the comparison; and

wherein the device is implemented to use the character code for an identification of a traffic sign.

13. A method of identifying traffic signs in an image, comprising:

Hough transforming the image or an edge image derived from same to identify circular arc segments or elliptical arc segments;

detecting a shape corresponding to a traffic sign in the image or an edge image derived from same based on the identified circular arc segments or elliptical arc segments; and

identifying a traffic sign in an image section corresponding to the detected shape using a comparative image pattern,

wherein the section of the image and the comparative image pattern are adapted to each other with respect to a shape by mapping;

wherein detecting a shape corresponding to a traffic sign comprises detecting a location of an ellipse in the image or in the edge image derived from same based on the identified circular arc segments or elliptical arc segments;

wherein Hough transforming comprises determining extreme points of circular arc segments or of elliptical arc segments, and

wherein detecting the ellipse comprises selecting a first set of four extreme points from the extreme points determined by the Hough transformer, and a second set of four extreme points from the extreme points determined by the Hough transformer, wherein the first set of four extreme points is different from the second set of four extreme points,

wherein detecting the ellipse comprises determining parameters of a first fit ellipse by the four extreme points of the first set of extreme points,

determining a deviation of the four extreme points of the first set of extreme points from the first fit ellipse,

determining parameters of a second fit ellipse by the four extreme points of the second set of extreme points,

determining a deviation of the four extreme points of the second set of extreme points from the fit ellipse, and

determining, using the deviation of the four extreme points of the first set of extreme points from the first fit ellipse and using the deviation of the four extreme points of the second set of extreme points from the second fit ellipse, whether the four extreme points of the first set of extreme points belong to an ellipse in the image, and whether the four extreme points of the second set of extreme points belong to an ellipse in the image.

14. The method according to claim 13, the method comprising selecting an image section whose shape corresponds to the detected shape and identifying a traffic sign in the image section corresponding to the detected shape.

15. A non-transitory computer readable medium having a computer program for executing, when the computer program is executed on a computer, the method of identifying traffic signs in an image, the method comprising:

Hough transforming the image or an edge image derived from same to identify circular arc segments or elliptical arc segments;

detecting a shape corresponding to a traffic sign in the image or an edge image derived from same based on the identified circular arc segments or elliptical arc segments; and

identifying a traffic sign in an image section corresponding to the detected shape using a comparative image pattern, wherein the section of the image and the comparative image pattern are adapted to each other with respect to a shape by mapping;

wherein detecting a shape corresponding to a traffic sign comprises detecting a location of an ellipse in the image or in the edge image derived from same based on the identified circular arc segments or elliptical arc segments;

wherein Hough transforming comprises determining extreme points of circular arc segments or of elliptical arc segments, and

wherein detecting the ellipse comprises selecting a first set of four extreme points from the extreme points determined by the Hough transformer, and a second set of four extreme points from the extreme points determined by the Hough transformer, wherein the first set of four extreme points is different from the second set of four extreme points,

wherein detecting the ellipse comprises determining parameters of a first fit ellipse by the four extreme points of the first set of extreme points,

determining a deviation of the four extreme points of the first set of extreme points from the first fit ellipse, determining parameters of a second fit ellipse by the four extreme points of the second set of extreme points,

determining a deviation of the four extreme points of the second set of extreme points from the fit ellipse, and

determining, using the deviation of the four extreme points of the first set of extreme points from the first fit ellipse and using the deviation of the four extreme points of the second set of extreme points from the second fit ellipse, whether the four extreme points of the first set of extreme points belong to an ellipse in the image, and whether the four extreme points of the second set of extreme points belong to an ellipse in the image.