Apparatus and method for recognizing traffic signs

• US 10,445,602 B2
• Filed: 11/09/2017
• Issued: 10/15/2019
• Est. Priority Date: 05/30/2017
• Status: Active Grant

• Current Assignee: HL Klemove Corporation
• Original Assignee: Mando-Hella Electronics Corp.
• Inventors: Park, Ha-Bit, Kim, Ki-Man, Kim, Beom-Kyu, Kim, Shinwook
• Primary Examiner(s): Chang, Jon
• Application Number:
  US15/808,873
• Publication Number:
  US 20180349716A1
• Time in Patent Office: 705 Days
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 10/759 Region-based matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/582 of traffic signs

Abstract

An apparatus for recognizing traffic signs normalizes a window of a predetermined size for a region of interest set in an image frame inputted from an image sensor, generates a first input vector, extracts a candidate region of a traffic sign based on feature pattern information of a neuron having a feature pattern vector, stores the coordinates of the extracted traffic sign candidate region, converts the image size of the extracted candidate region, normalizes a window of a predetermined size for the candidate region of the converted image size, generates a second input vector, determines traffic sign content information of a neuron having a content pattern vector, stores the determined traffic sign content information, and recognizes the location and content of the traffic sign based on the coordinates of the stored candidate regions and the content information of the stored traffic sign when the traffic sign disappears.

Claims

1. An apparatus for recognizing traffic signs comprising:

an image sensor unit for acquiring an image frame;

a neuromorphic system in which a plurality of neurons storing a feature pattern vector capable of specifying a sign including an edge of a traffic sign and a content pattern vector related to content information of the traffic sign are connected by a parallel bus; and

a controller that (a) sets a certain region as a region of interest based on an upper end of the image frame inputted from the image sensor unit, (b) normalizes a window of a first size, which is preset for the set region of interest, by moving the window in such a way to overlap by a first number of pixels, (c) generates a first input vector that vectorizes the normalized window, (d) inputs the generated first input vector to the neuromorphic system, (e) extracts a candidate region of the traffic sign based on feature pattern information of a neuron having a feature pattern vector most similar to the inputted first input vector among the plurality of neurons stored in the neuromorphic system, (f) stores the coordinates of the extracted traffic sign candidate region, (g) converts the image size of the extracted candidate region, (h) normalizes a window of a second size, which is preset for the candidate region of the converted image size, by moving the window in such a way to overlap by a second number of pixels, (i) generates a second input vector that vectorizes the normalized window, (j) inputs the generated second input vector to the neuromorphic system, (k) determines traffic sign content information of a neuron having a content pattern vector most similar to the inputted second input vector among the plurality of neurons stored in the neuromorphic system, (l) stores the determined traffic sign content information, (m) performs processes (a) to (l) until the traffic sign is beyond the field of view, and (n) recognizes the location and content of the traffic sign based on the coordinates of the stored candidate regions and the content information of the stored traffic sign when the traffic sign disappears.

2. The apparatus according to claim 1,

wherein a region corresponding to 70% on the basis of the upper end of the image frame inputted from the image sensor unit is set as the region of interest.

3. The apparatus according to claim 1,

wherein the controller recognizes the location and content of the traffic sign by using a mode value among the coordinates of the stored traffic sign candidate regions and the traffic sign content information determined in the candidate regions or by weighting the coordinates of the traffic sign candidate region extracted from a last image frame and the traffic sign content information determined in the extracted traffic sign candidate region.

4. A method for recognizing traffic signs by using an image sensor unit for acquiring an image frame and a neuromorphic system in which a plurality of neurons storing a feature pattern vector capable of specifying a sign including an edge of a traffic sign and a content pattern vector related to content information of the traffic sign are connected by a parallel bus, comprising:

(a) setting a certain region as a region of interest based on an upper end of the image frame inputted from the image sensor unit,

(b) normalizing a window of a first size, which is preset for the set region of interest, by moving the window in such a way to overlap by a first number of pixels,

(c) generating a first input vector that vectorizes the normalized window,

(d) inputting the generated first input vector to the neuromorphic system,

(e) extracting a candidate region of the traffic sign based on feature pattern information of a neuron having a feature pattern vector most similar to the inputted first input vector among the plurality of neurons stored in the neuromorphic system,

(f) storing the coordinates of the extracted traffic sign candidate region,

(g) converting the image size of the extracted candidate region,

(h) normalizing a window of a second size, which is preset for the candidate region of the converted image size, by moving the window in such a way to overlap by a second number of pixels,

(i) generating a second input vector that vectorizes the normalized window,

(j) inputting the generated second input vector to the neuromorphic system,

(k) determining traffic sign content information of a neuron having a content pattern vector most similar to the inputted second input vector among the plurality of neurons stored in the neuromorphic system,

(l) storing the determined traffic sign content information,

(m) performing processes (a) to (1) until the traffic sign is beyond the field of view, and

(n) recognizing the location and content of the traffic sign based on the coordinates of the stored candidate regions and the content information of the stored traffic sign when the traffic sign disappears.

5. The method according to claim 4, wherein a region corresponding to 70% on the basis of the upper end of the image frame inputted from the image sensor unit is set as the region of interest.

6. The method according to claim 4, further comprising:

recognizing the location and content of the traffic sign by using a mode value among the coordinates of the stored traffic sign candidate regions and the traffic sign content information determined in the candidate regions or by weighting the coordinates of the traffic sign candidate region extracted from a last image frame and the traffic sign content information determined in the extracted traffic sign candidate region.