Method for traffic sign detection

• US 7,466,841 B2
• Filed: 04/19/2005
• Issued: 12/16/2008
• Est. Priority Date: 08/16/2004
• Status: Active Grant

• Current Assignee: Continental Automotive GmbH (Continental AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Ramesh, Visvanathan, Zhu, Ying, Bahlmann, Claus, Pellkofer, Martin, Köhler, Thorsten
• Primary Examiner(s): Desire; Gregory M
• Application Number:
  US11/109,106
• Publication Number:
  US 20060034484A1
• Time in Patent Office: 1,337 Days
• Field of Search: 382/103, 382/104, 382/107, 382/156, 382/159, 382/165, 382/181, 382/291, 700/47, 700/48, 700/51, 706/20
• US Class Current: 382/103
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

A method for detecting and recognizing at least one traffic sign is disclosed. A video sequence having a plurality of image frames is received. One or more filters are used to measure features in at least one image frame indicative of an object of interest. The measured features are combined and aggregated into a score indicating possible presence of an object. The scores are fused over multiple image frames for a robust detection. If a score indicates possible presence of an object in an area of the image frame, the area is aligned with a model. A determination is then made as to whether the area indicates a traffic sign. If the area indicates a traffic sign, the area is classified into a particular type of traffic sign. The present invention is also directed to training a system to detect and recognize traffic signs.

Claims

1. A method for detecting and recognizing at least one traffic sign, the method comprising the steps of:

a. receiving a video sequence comprised of a plurality of image frames;

b. using one or more filters to measure features in at least one image frame indicative of an object of interest;

c. aggregating and combining features into a score indicating possible presence of an object;

d. fusing the scores over multiple image frames for a robust detection;

e. if a score indicates possible presence of an object in an area of the image frame, aligning the area with a model;

f. determining if the area indicates a traffic sign; and

g. if so, classifying the area into a particular type of traffic sign.

2. The method of claim 1 wherein the image frames are color image frames.

3. The method of claim 2 wherein at least one filter is a color representation filter.

4. The method of claim 3 wherein the color filter incorporates one or more color channels.

5. The method of claim 4 wherein at least one color channel is a normalized color channel.

6. The method of claim 4 wherein at least one channel is a gray channel.

7. The method of claim 4 wherein at least one color channel is a linear transformation of a color channel.

8. The method of claim 1 wherein the image frames are grayscale image frames.

9. The method of claim 8 wherein at least one filter is a grayscale representation filter.

10. The method of claim 1 wherein at least one filter is a ring filter.

11. The method of claim 1 wherein the step of combining features into a score further comprises an aggregated evaluation of simple classifiers and wavelet features.

12. The method of claim 11 wherein the aggregated classifier is evaluated using a sequential hypothesis testing scheme and partial classifier decisions.

13. The method of claim 1 wherein the step of detecting traffic signs further comprises the steps of:

accumulating scores for consecutive image frames to obtain a cumulative score;

comparing the accumulated score after each image frame to determine if the score is above a predetermined threshold;

if the score is above the threshold, adding a feature score from the next image frame to the accumulated score; and

if the score is below the threshold, determining that the feature is not associated with an object of interest.

14. The method of claim 1 wherein the step of aligning the area of the image further comprises the steps of:

defining an edge region, a boundary region, a ring region and an interior region;

defining a cost function that measures dissimilarity of a sign model and a candidate sign image patch; and

estimating correct sign parameters from a minimization of an error function in the parameter space.

15. The method of claim 14 wherein the step of estimating sign parameters is used for image normalization and further comprises the steps of:

normalizing position and scale of the sign parameters, and

normalizing image intensity, by using an area defined on x, y, and r, and statistical properties of the area.

16. The method of claim 14 wherein the estimated sign parameters are x, y, and r and are used to form an image region-of-interest (ROI).

17. The method of claim 16 wherein a classification feature vector is obtained from a linear discriminant analysis (LDA) of pixels in the ROI.

18. The method of claim 17 wherein a linear transformation is determined by linear discriminant analysis (LDA) of object and non-object pixels in the color space.

19. The method of claim 14 wherein adaptation of parameters α

_i in

$\begin{array}{c}E<br><br>\left(a,b,r\right)=<br><br>\underset{i=1}{\overset{6}{\sum <br><br>}}<br><br>{\mathrm{\alpha <br><br>}}_i<br><br>E_i<br><br>\left(a,b,r\right)\\ =<br><br>{\mathrm{\alpha <br><br>}}_i<br><br>{\mathrm{grad}}_{X_{\mathrm{Edge}}^{\left(a,b,r\right)}}+\\ <br><br>{\mathrm{\alpha <br><br>}}_2<br><br>{\mathrm{grad}}_{X_{\mathrm{Boundary}}^{\left(a,b,r\right)}}+\\ <br><br>{\mathrm{\alpha <br><br>}}_3<br><br>\log <br><br>\left({\mathrm{\sigma <br><br>}}_{X_{\mathrm{Ring}}^{\left(a,b,r\right)}}^2\right)+\\ <br><br>{\mathrm{\alpha <br><br>}}_4<br><br>\log <br><br>\left({\mathrm{\sigma <br><br>}}_{X_{\mathrm{PictBG}}^{\left(a,b,r\right)}}^2\right)+\\ <br><br>{\mathrm{\alpha <br><br>}}_5<br><br>\log <br><br>\left({\mathrm{\sigma <br><br>}}_{X_{\mathrm{PictFG}}^{\left(a,b,r\right)}}^2\right)+\\ <br><br>{\mathrm{\alpha <br><br>}}_6<br><br>\frac{{\left({\mathrm{\mu <br><br>}}_{X_{\mathrm{Ring}}^{\left(a,b,r\right)}}-{\mathrm{\mu <br><br>}}_{X_{\mathrm{PictBG}}^{\left(a,b,r\right)}}\right)}^2}{{\mathrm{\sigma <br><br>}}_{X_{\mathrm{Ring}}^{\left(a,b,r\right)}}^2+{\mathrm{\sigma <br><br>}}_{X_{\mathrm{PictBG}}^{\left(a,b,r\right)}}^2}\end{array}$

is performed by a gradient decent with respect to classification error.

20. The method of claim 1 wherein the classification of a sign image is obtained using multivariate Gaussian probability density functions of the feature space and a maximum-a-priori (MAP) or maximum likelihood (ML) approach.

21. The method of claim 20 wherein the step of classification further comprises the step of accumulating classification scores for consecutive image frames to obtain a cumulative classification score.

22. The method of claim 20 wherein training of sign classes involves an automatic alignment of the training images, an image normalization, a region-of-interest formation, a linear discriminant analysis (LDA) feature transform, and the estimation of Gaussian probability density functions.

23. The method of claim 1 wherein the traffic sign being detected is a circular sign having a colored ring around its perimeter.

24. The method of claim 1 wherein filters ƒ

_t, weights α

_t and classifier thresholds θ

_t are adapted using AdaBoost.

25. The method of claim 24 wherein thresholds θ

_t for sequential hypothesis testing are determined from

1−

d_t≦

1−

d_t, ∀

t∈

{1, . . . T−

1}.

with a target false negative rate 1−

d_t.

26. The method of claim 24 wherein thresholds θ

_t for sequential hypothesis testing are determined from

Ø

_t=r_min,pos,^(t),t∈

{1, . . . , T−

1}.

27. The method of claim 1 wherein geometric parameters of filters ƒ

_t are adapted using AdaBoost.

28. The method of claim 1 wherein color representation of filters ƒ

_t are adapted using AdaBoost.