Real time traffic sign recognition

• US 10,255,511 B2
• Filed: 12/30/2016
• Issued: 04/09/2019
• Est. Priority Date: 01/04/2016
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Kudana, Arun Shankar, Mathew, Manu, Nagori, Soyeb
• Primary Examiner(s): Tsai, Tsung Yin
• Application Number:
  US15/395,141
• Publication Number:
  US 20170193313A1
• Time in Patent Office: 830 Days
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 10/7747 Organisation of the process, e.g. bagging or boosting
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

Advanced driver assistance systems need to be able to recognize and to classify traffic signs under real time constraints, and under a wide variety of visual conditions. The invention shown employs binary masks extracted by color space segmentation, with a different binary mask generated for each sign shape. Temporal tracking is employed to add robustness to the detection system. The system is generic, and is trainable to the signs used in various countries.

Claims

1. A method for recognition of traffic signs, the method comprising:

receiving an input image of a captured scene;

identifying one or more approximate locations of interest in the input image based at least partially on extracting color information from the input image, wherein extracting the color information includes extracting initial binary masks from the input image that include a red initial binary mask, a blue initial binary mask, a yellow initial binary mask, and a white initial binary mask, and applying, to each of the red, blue, yellow, and white initial binary masks, a morphological opening operation that includes an erosion operation follows by a dilation operation to produce respective red, blue, yellow, and white final binary masks;

processing each of the one or more approximate locations of interest in the input image to identify one or more localized windows in the input image in which a traffic sign is present;

for each of the one or more localized windows, determining initial classification information of the respective traffic sign present in the localized window based on a plurality of traffic sign classification categories;

applying temporal smoothing to each of the one or more localized windows to determine, for each of the one or more localized windows, final classification information of the respective traffic sign present in the localized window; and

outputting information that identifies, for each traffic sign present in the captured scene, a traffic sign type based on the final classification information.

2. The method of claim 1, wherein the input image is initially in RGB format and extracting the color information from the input image includes converting the input image from RGB format to YUV format and extracting the red, blue, yellow, and white initial binary masks from the input image in YUV format using thresholding.

3. The method of claim 2, comprising applying contrast stretching to the input image in YUV format prior to extracting the red, blue, yellow, and white initial binary masks by using histogram equalization on a Y-plane of the input image in YUV format.

4. The method of claim 1, wherein identifying the one or more approximate locations of interest in the input image is also based at least partially on applying shape detection to the input image.

5. The method of claim 4, wherein applying shape detection to the input image includes applying an extended radial symmetry transform to the red, blue, yellow, and white final binary masks to detect locations of one or more of a plurality of shape types in the input image.

6. The method of claim 5, wherein the plurality of shape types include a circle, a triangle, a square, and an octagon.

7. The method of claim 1, wherein processing each of the one or more approximate locations of interest in the input image to identify the one or more localized windows in the input image in which a traffic sign is present includes applying a feature extraction operation to the input image.

8. The method of claim 7, wherein applying the feature extraction operation to the input image includes determining a plurality of feature planes based on the input image, the plurality of feature plans including a gradient magnitude feature plane, at least one histogram of oriented gradients (HOG) feature plane, and three feature planes based on the input image in YUV format.

9. The method of claim 8, wherein the at least one HOG feature plane includes 6 HOG feature planes.

10. The method of claim 7, wherein processing each of the one or more approximate locations of interest in the input image to identify the one or more localized windows in the input image in which a traffic sign is present includes applying an adaptive boost classifier operation after applying the feature extraction operation to the input image.

11. The method of claim 10, wherein applying the adaptive boost classifier operation includes using a plurality of decision trees as classifiers.

12. The method of claim 11, wherein each of the plurality of decision trees has a depth of 2.

13. The method of claim 11, wherein the plurality of decision trees is 1024 decision trees.

14. The method of claim 1, wherein determining, for each of the one or more localized windows, the initial classification information of the respective traffic sign present in the localized window based on a plurality of traffic sign classification categories includes using a linear discriminant analyzer (LDA).

15. The method of claim 14, wherein using the LDA includes minimizing a Mahalanobis distance between a feature vector and a mean vector of the plurality of traffic sign classification categories.

16. The method of claim 1, wherein the input image is received from an image capture device is configured to capture video and the input image is one frame of the captured video.

17. The method of claim 1, wherein the image capture device is part of an automobile advanced driver assistance system (ADAS).