CONVOLUTION OPERATION CIRCUIT AND OBJECT RECOGNITION APPARATUS

• US 20110239032A1
• Filed: 12/03/2009
• Published: 09/29/2011
• Est. Priority Date: 12/04/2008
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ito, Yoshinori, Yamamoto, Takahisa, Kato, Masami
• Granted Patent: US 8,832,004 B2
• Time in Patent Office: Days
• US Class Current: 713/500
• CPC Class Codes: G06F 17/15 Correlation function computation including computation of convolution operations arithmetic circuits for sum of products per se, e.g. multiply-accumulators G06F7/5443; digital filters, e.g. FIR, IIR, adaptive filters H03H17/00
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/06 Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

In a convolution operation circuit, a first and a second shift registers provide data to a first and a second inputs of a plurality of multipliers, a first and a second storage units store data to be supplied to the first and the second shift registers, a plurality of cumulative adders accumulate output from the plurality of multipliers, a third storage unit latches output from the plurality of cumulative adders at predetermined timing, a fourth storage unit stores data to be stored in the first and the second storage units and data output from the third storage unit, and a control unit sets data stored in the first and the second storage units to the first and the second shift registers at predetermined timing, causes the first and the second shift registers to perform shift operations in synchronization with an operation of the cumulative adder.

Claims

1. A convolution operation circuit comprising:

a plurality of multipliers;

a first shift register configured to provide data to a first input of the plurality of multipliers;

a first storage unit configured to store a plurality of data to be supplied to the first shift register;

a second shift register configured to provide data to a second input of the plurality of multipliers;

a second storage unit configured to store a plurality of data to be supplied to the second shift register;

a plurality of cumulative adders configured to accumulate output from the plurality of multipliers;

a third storage unit configured to latch output from the plurality of cumulative adders at predetermined timing;

a fourth storage unit configured to store data to be stored in the first storage unit and the second storage unit and data output from the third storage unit; and

a control unit configured to control operations of the first storage unit, the second storage unit, the third storage unit, the fourth storage unit, the first shift register, the second register, and the cumulative adder,

wherein the control unit sets a plurality of data stored in the first storage unit to the first shift register at predetermined timing, sets a plurality of data stored in the second storage unit to the second shift register at predetermined timing, causes the first shift register and the second shift register to perform shift operations in synchronization with an operation of the cumulative adder, and transfers during the shift operation data stored in the fourth storage unit to at least one of the first storage unit and the second storage unit.

2. The convolution operation circuit according to claim 1, wherein the third storage unit is a shift register including a number of registers which is greater than or equal to a number of the plurality of multipliers.

3. The convolution operation circuit according to claim 1, further comprising a non-linear conversion unit configured to perform non-linear conversion on an output from the third storage unit.

4. The convolution operation circuit according to claim 3, wherein the control unit stores in a predetermined region in the fourth storage unit during the shift operation an operation result stored in the third storage unit or a result of non-linear conversion performed by the non-linear conversion unit.

5. The convolution operation circuit according to claim 1, wherein a register length of the first shift register and a storage capacity of the first storage unit is greater than or equal to a value acquired by subtracting 1 from a sum of a number of the multipliers and a size of a column or a row of a two-dimensional convolution kernel, and

wherein the predetermined timing at which data is set to the first shift register is timing at which a process of performing a convolution operation in a column direction or a row direction is started or ended.

6. The convolution operation circuit according to claim 1, wherein a register length of the second shift register and a storage capacity of the second storage unit is greater than or equal to a size of a column or a row of a two-dimensional convolution kernel, and

wherein predetermined timing at which data is set to the second shift register is timing at which a process of performing a convolution operation in a column direction or a row direction is started or ended.

7. The convolution operation circuit according to claim 1, wherein a register length of the second shift register and a storage capacity of the second storage unit is greater than or equal to a number of coefficients in a two-dimensional convolution kernel, and

wherein predetermined timing at which data is set to the second shift register is timing at which a process of performing one convolution operation is started or ended.

21. An object recognition apparatus including a convolution operation circuit according to claim 1.

8. A hierarchical convolution operation circuit comprising:

a convolution operation unit including;

a multiplier;

a first storage unit configured to provide data to a first input of the multiplier;

a second storage unit configured to provide data to a second input of the multiplier;

a cumulative adder configured to accumulate output from the multiplier;

a third storage unit configured to latch output from the cumulative adder;

a fourth storage unit configured to store data to be supplied to the first storage unit and the second storage unit and data output from the third storage unit; and

a control unit configured to control operations performed by the fourth storage unit and the convolution operation unit,

wherein the control unit maps the fourth storage unit in a common address space, controls a reading address and a writing address in the fourth storage unit and performs data transfer from the fourth storage unit to the first storage unit and the second storage unit and from the third storage unit to the fourth storage unit, and causes the convolution operation unit to perform a convolution operation on data to be processed, stored in the fourth storage unit and again perform convolution operation on a result of the convolution operation.

9. The hierarchical convolution operation circuit according to claim 8, wherein the control unit includes a storage region which is accessible from outside and determines a hierarchical connection relation according to a content of the storage region.

10. The hierarchical convolution operation circuit according to claim 9, wherein the control unit determines an address of the storage region according to a content of the storage region which is accessible from outside.

11. The hierarchical convolution operation circuit according to claim 9, wherein the control unit includes information for determining a hierarchical connection according to a content of the storage region which is accessible from outside for each convolution operation to be performed on all data regions in a previous layer to be connected.

12. The hierarchical convolution operation circuit according to claim 9, wherein the control unit determines a read address of the fourth storage unit according to a content of the storage region which is accessible from outside.

13. The hierarchical convolution operation circuit according to claim 9, wherein the control unit determines a write address of the fourth storage unit according to a content of the storage region which is accessible from outside.

14. The hierarchical convolution operation circuit according to claim 9, wherein the storage region which is accessible from outside includes information about a pointer of data in a previous layer to be connected.

15. The hierarchical convolution operation circuit according to claim 9, wherein the storage region which is accessible from outside includes information about a pointer of storage destination of feature data to be processed.

16. The hierarchical convolution operation circuit according to claim 9 wherein the storage region which is accessible from outside includes information for defining a number of previous layers to be connected.

17. The hierarchical convolution operation circuit according to claim 9, wherein the storage region which is accessible from outside includes information for defining whether a unit of convolution operations to be performed on all data regions in a previous layer to be connected is a final process.

18. The hierarchical convolution operation circuit according to claim 9, wherein the storage region which is accessible from outside includes information about a size of weight coefficients.

19. The hierarchical convolution operation circuit according to claim 9, wherein the storage region which is accessible from outside includes information about a pointer of weight coefficient data.

20. The hierarchical convolution operation circuit according to claim 9, wherein the control unit determines whether to perform non-linear conversion according to a content of the storage region which is accessible from outside.

22. An object recognition apparatus including a convolution operation circuit according to claim 8.