Vector computation unit in a neural network processor

• US 10,074,051 B2
• Filed: 12/22/2016
• Issued: 09/11/2018
• Est. Priority Date: 05/21/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Thorson, Gregory Michael, Clark, Christopher Aaron, Luu, Dan
• Primary Examiner(s): Wong, Lut
• Application Number:
  US15/389,288
• Publication Number:
  US 20170103315A1
• Time in Patent Office: 628 Days
• Field of Search: 706 33
• CPC Class Codes: G06F 5/08 having a sequence of storage locations, the intermediate ones not being accessible for either enqueue or dequeue operations, e.g. using a shift register G06F5/065 takes precedence; shift registers per se G11C19/00
• CPC Class Codes: G06F 7/544 for evaluating functions by calculation G06F7/4824 takes precedence
• CPC Class Codes: G06N 3/0464 Convolutional networks [CNN, ConvNet]
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising: activation circuitry configured to receive a vector of accumulated values and configured to apply a function to each accumulated value to generate a vector of activation values; and normalization circuitry coupled to the activation circuitry and configured to generate a respective normalized value from each activation value.

Claims

1. A circuit for performing neural network computations for a neural network comprising a plurality of layers, the circuit comprising:

a vector computation unit, wherein the vector computation unit includes;

activation circuitry configured to receive a vector of accumulated values, where the accumulated values correspond to products of a matrix multiplication between an input to a layer of the neural network and a parameter matrix for the layer, and where the activation circuitry is configured to apply a function to each accumulated value to generate a vector of activation values; and

normalization circuitry coupled to the activation circuitry and configured to receive activation values of the vector of activation values and to generate a respective normalized value for each activation value, the normalization circuitry comprising a plurality of normalization register columns, each normalization register column comprising a plurality of normalization registers, and wherein at least one normalization register in each normalization register column is configured to receive a respective activation value for generating at least the respective normalized value for the activation value.

2. The circuit of claim 1, where the activation circuitry receives the vector of accumulated values from a systolic array in the circuit.

3. The circuit of claim 1, where the plurality of normalization registers in each normalization register column are connected in series, each normalization register column is configured to receive the respective activation value, where the normalization circuitry is configured to form groups around one or more normalization registers, each group corresponding to a normalization unit, and each normalization unit is configured to calculate a respective normalized value for the respective activation value.

4. The circuit of claim 3, where each normalization register is configured to pass a distinct activation value to an adjacent normalization register column.

5. The circuit of claim 3, where each group is formed using a normalization radius parameter.

6. The circuit of claim 3, where each normalization unit is configured to:

receive the respective activation value;

generate a respective intermediate normalized value from the respective activation value; and

send the respective intermediate normalized value to one or more neighboring normalization units.

7. The circuit of claim 6, where generating the respective intermediate normalized value comprises generating a square of the respective activation value.

8. The circuit of claim 6, where each normalization unit is further configured to:

receive, from one or more neighboring normalization units, one or more intermediate normalized values generated from activation values;

sum each intermediate normalized value to generate an index;

use the index to access one or more values from a lookup table;

generate a scaling factor from the one or more values and the index; and

generate the respective normalized value from the scaling factor and the respective activation value.

9. The circuit of claim 1, further comprising pooling circuitry configured to receive the normalized values and configured to pool the normalized values to generate a pooled value.

10. The circuit of claim 6, where the pooling circuitry is configured to store the plurality of normalized values in a plurality of registers and a plurality of memory units,

where the plurality of registers and the plurality of memory units are connected in series, where each register stores one normalized value and each memory unit stores a plurality of normalized values,

where the pooling circuitry is configured to, after every clock cycle, shift a given normalized value to a subsequent register or memory unit, and

where the pooling circuitry is configured to generate the pooled value from the normalized values.

11. The circuit of claim 1, further comprising pooling circuitry configured to receive the activation values and configured to pool the activation values to generate a pooled value.

12. The circuit of claim 11, where the pooling circuitry is configured to store the plurality of activation values in a plurality of registers and a plurality of memory units,

where the plurality of registers and the plurality of memory units are connected in series, where each register stores one normalized value and each memory unit stores a plurality of activation values,

where the pooling circuitry is configured to, after every clock cycle, shift a given activation value to a subsequent register or memory unit, and

where the pooling circuitry is configured to generate the pooled value from the activation values.

13. A method for performing neural network computations for a neural network comprising a plurality of layers, the method comprising:

receiving, at an activation circuit in a vector computation unit and from a systolic array, a vector of accumulated values, where the accumulated values correspond to products of a matrix multiplication between an input to a layer of the neural network and a parameter matrix for the layer;

applying, at the activation circuit in the vector computation unit, a function to each accumulated value to generate a vector of activation values;

receiving, at a normalization circuit in the vector computation unit, the vector of activation values from the activation circuit, wherein the normalization circuit is coupled to the activation circuit in the vector computation unit, and wherein the normalization circuit comprises multiple normalization register columns, each normalization register column comprising multiple normalization registers, and wherein at least one normalization register in each normalization register column is configured to receive a respective activation value of the vector of activation values; and

generating, at the normalization circuit in the vector computation unit, a respective normalized value for each activation value received by the at least one normalization register in a normalization register column.

14. The method of claim 13, where receiving the vector of activation values at the normalization circuit, comprises:

receiving, at the normalization circuit, the respective activation value for each normalization register column in the normalization circuit, wherein each normalization register column comprises multiple normalization registers that are connected in series.

15. The method of claim 14, further comprising:

forming, at the normalization circuit, groups around one or more normalization registers, where each group corresponds to a normalization unit; and

calculating, at each normalization unit, a respective normalized value for the respective activation value.

16. The method of claim 14, further comprising:

passing, using a normalization register of the normalization circuit, a distinct activation value to an adjacent normalization register column; and

using, at the normalization circuit, a normalization radius parameter to form the groups around the one or more normalization registers.