IMAGE RECOGNITION APPARATUS

• US 20140169631A1
• Filed: 07/25/2012
• Published: 06/19/2014
• Est. Priority Date: 08/05/2011
• Status: Active Grant

• Current Assignee: Megachips Corporation
• Original Assignee: Megachips Corporation
• Inventors: Mizuno, Yusuke, Togo, Daisuke
• Granted Patent: US 9,361,534 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

An image recognition apparatus determines whether an image of a pedestrian is captured in a frame of video data captured by a vehicle mounted camera. A pre-processing unit determines a detection block from within a frame, and cuts out block image data corresponding to the detection block from the frame. Block data with a predetermined size that is smaller than the size of the detection block is created from the block image data. A neuro calculation unit executes neuro calculation on the block data, and calculates an output synapse. A post-processing unit determines whether a pedestrian exists within the detection block on the basis of the output synapse. When a pedestrian is detected, the post-processing unit creates result data, which is obtained by superimposing the detection block within which the pedestrian was detected onto the frame.

Claims

1. An image recognition apparatus configured to determine whether or not an object which is to be detected is included in a frame, comprising:

a pre-processing unit configured to generate first calculation image data from a first area image in the frame, a size of the first calculation image data being smaller than that of the first area image;

a neuro calculation unit configured to calculate a neuro calculation value which indicates whether or not the object which is to be detected is included in the first area image by performing neuro calculation on a pixel value of the first calculation image data; and

a post-processing unit configured to generate result data which indicates whether or not the object which is to be detected is included in the frame by using the neuro calculation value.

2. The image recognition apparatus according to claim 1, further comprising:

a first block buffer configured to store the first calculation image data; and

a second block buffer configured to store second calculation image data generated from a second area image in the frame by the pre-processing unit, the second area image being different from the first area image, wherein

the pre-processing unit generates the second calculation image data and stores the second calculation image data in the second block buffer while the neuro calculation unit performs neuro calculation by using the first calculation image data stored in the first block buffer, and

the pre-processing unit generates the first calculation image data and stores the first calculation image data in the first block buffer while the neuro calculation unit performs neuro calculation by using the second calculation image data stored in the second block buffer.

3. The image recognition apparatus according to claim 1, wherein

the pre-processing unit includes:

a block parameter calculation unit configured to statistically process pixel values in the first area image to calculate a block statistical value; and

a normalization unit configured to normalize pixel values in the first area image based on the block statistical value and a standard distribution data of pixel values in the first area image.

4. The image recognition apparatus according to claim 3, wherein

the pre-processing unit includes a frame parameter calculation unit configured to statistically process pixel values in the frame to calculate a frame statistical value,

the normalization unit normalizes pixel values in the frame based on the frame statistical value and a standard distribution data of pixel values in the frame, and

the pre-processing unit normalizes pixel values in the first area image by using the frame which is normalized.

5. The image recognition apparatus according to claim 1, further comprising:

an input interface configured to receive a plurality of frames;

a first frame buffer; and

a second frame buffer, wherein

the input interface stores a frame which is input next to a first frame stored in the first frame buffer into the second frame buffer while the first calculation image data is generated from the first frame.

6. The image recognition apparatus according to claim 1, wherein

the neuro calculation unit includes:

an intermediate synapse calculation unit configured to carry out an intermediate synapse calculation process of performing neuro calculation on synapses in an input layer to calculate synapses in an intermediate layer, at least one time; and

an output synapse calculation unit configured to perform neuro calculation on the synapses in the intermediate layer to calculate an output synapse, and output the output synapse as the neuro calculation value, wherein

each of the synapses in the input layer in the first intermediate synapse calculation process is pixel values of the first calculation image data.

7. The image recognition apparatus according to claim 6, wherein

the intermediate synapse calculation unit includes:

an object-for-calculation designation unit configured to designate N intermediate synapses from the synapses in the intermediate layer, as an object for calculation;

an input synapse selection unit configured to carry out a selecting process of selecting one input synapse from the synapses in the input layer; and

an intermediate operation unit configured to determine a weighting coefficient associated with the selected input synapse and the K-th (K is a natural number equal to or smaller than N) intermediate synapse, and perform an intermediate operation process of adding a result of multiplication of the selected input synapse by the determined weighting coefficient, to the K-th intermediate synapse, wherein

the intermediate operation unit performs respective intermediate operation processes on the first to N-th intermediate synapses, in parallel, and

the selecting process and the intermediate operation process are repeated until all of input synapses are selected.

8. The image recognition apparatus according to claim 7, wherein

the input synapse selection unit determines a weighting coefficient used for the next intermediate operation process in a period in which the intermediate operation process is carried out by the intermediate operation unit.

9. The image recognition apparatus according to claim 7, wherein

the synapses in the input layer are divided into a plurality of input groups, and

the input synapse selection unit selects one from the plurality of input groups, selects an input synapse on which the intermediate operation process is to be performed from the selected input group, and selects the next input group after all of input synapses included in the selected input group are selected.

10. The image recognition apparatus according to claim 9, further comprising:

an input synapse buffer configured to store the synapses in the input layer; and

a first coefficient table configured to store weighting coefficients used by the intermediate synapse calculation unit, wherein

the intermediate synapse calculation unit reads out all of input synapses included in the selected input group from the input synapse buffer by a one-time reading process, and reads out weighting coefficients determined by the intermediate operation unit from the first coefficient table by a one-time reading process.

11. The image recognition apparatus according to claim 9, wherein

the image recognition apparatus carries out an intermediate operation process which is carried out on an input synapse lastly selected from the selected input group, and a process of selecting the next input group, in parallel.

12. The image recognition apparatus according to claim 6, wherein

the intermediate synapse calculation unit uses intermediate synapses which is calculated immediately before, as input synapses when each of the second and subsequent intermediate operation processes is performed.

13. The image recognition apparatus according to claim 6, wherein

the intermediate synapse calculation unit performs sigmoid calculation on intermediate synapses after intermediate calculation processes on all of input synapses are finished, and

an approximate function obtained by linearly approximating a sigmoid function is used in the sigmoid calculation.

14. The image recognition apparatus according to claim 6, wherein

the output synapse calculation unit includes:

a partial addition value setting unit configured to set Q partial addition values which are used for calculation of the output synapse;

a partial addition value calculation unit configured to select Q intermediate synapses from the synapses in the intermediate layer, and perform a partial addition process of adding a result of multiplication of the R-th (R is a natural number equal to or smaller than Q) intermediate synapse included in the Q intermediate synapses by a weighting coefficient associated with the R-th intermediate synapse, to the R-th partial addition value; and

a summing unit configured to calculate a total value of the Q partial addition values after partial addition processes on all of intermediate synapses included in the synapses in the intermediate layer are finished, to calculate the output synapse, wherein

the partial addition processes using the Q intermediate synapses are carried out in parallel.

15. The image recognition apparatus according to claim 14, further comprising:

an intermediate synapse buffer configured to store the synapses in the intermediate layer; and

a second coefficient table configured to store weighting coefficients respectively associated with the synapses in the intermediate layer, wherein

the output synapse calculation unit reads out the Q intermediate synapses from the intermediate synapse buffer by a one-time reading process, and reads out the weighting coefficients respectively associated with the Q intermediate synapses from the second coefficient table by a one-time reading process.

16. The image recognition apparatus according to claim 14, wherein

the output synapse calculation unit designates another Q intermediate synapses and weighting coefficients which are used for the next partial addition process in a period in which the partial addition process is carried out.

17. The image recognition apparatus according to claim 14, wherein

the intermediate synapse calculation unit performs sigmoid calculation on the total value of the Q partial addition values to calculate the output synapse after partial addition processes on all of the intermediate synapses are finished, and

an approximate function obtained by linearly approximating a sigmoid function is used in the sigmoid calculation.

18. The image recognition apparatus according to claim 1, wherein

the neuro calculation unit is implemented by a configurable processor.