Convolutional neural network using a binarized convolution layer

• US 9,563,825 B2
• Filed: 11/20/2014
• Issued: 02/07/2017
• Est. Priority Date: 11/20/2014
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Shen, Xiaohui, Li, Haoxiang, Lin, Zhe, Brandt, Jonathan W.
• Primary Examiner(s): TUCKER, WESLEY J
• Application Number:
  US14/549,350
• Publication Number:
  US 20160148078A1
• Time in Patent Office: 810 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A convolutional neural network is trained to analyze input data in various different manners. The convolutional neural network includes multiple layers, one of which is a convolution layer that performs a convolution, for each of one or more filters in the convolution layer, of the filter over the input data. The convolution includes generation of an inner product based on the filter and the input data. Both the filter of the convolution layer and the input data are binarized, allowing the inner product to be computed using particular operations that are typically faster than multiplication of floating point values. The possible results for the convolution layer can optionally be pre-computed and stored in a look-up table. Thus, during operation of the convolutional neural network, rather than performing the convolution on the input data, the pre-computed result can be obtained from the look-up table.

Claims

1. A method comprising:

obtaining, by at least one computing device, input data that is a set of training inputs for a convolutional neural network, the convolutional neural network including multiple layers, one of the multiple layers including a convolution layer, the convolution layer including one or more filters;

training, by the at least one computing device, the convolutional neural network based on the set of training inputs;

binarizing, by the at least one computing device, the convolution layer by converting parameters of the one or more filters from floating point to binary values;

generating, by the at least one computing device, look-up table values of a look-up table for the binary values, the look-up table storing pre-computed results that are possible for the convolution layer; and

outputting, by the at least one computing device, a result of the generating.

2. A method as recited in claim 1, the binarizing the convolution layer comprising converting each filter in the convolution layer into a set of coefficients and binary vectors.

3. A method as recited in claim 1, further comprising binarizing, prior to training the convolutional neural network, the input data.

4. A method as recited in claim 3, the binarizing the input data comprising converting each training input into a set of coefficients and binary vectors.

5. A method as recited in claim 1, further comprising fine-tuning the convolutional neural network, after binarizing the convolution layer, by again applying the training inputs to the convolutional neural network to adjust ones of the multiple layers in the convolutional neural network later than the convolution layer.

6. A method as recited in claim 1, further comprising repeating the binarizing the convolution layer and fine-tuning the convolutional neural network.

7. A method as recited in claim 1, the multiple layers including multiple convolution layers, and the binarizing comprising binarizing each of the multiple convolution layers.

8. A method as recited in claim 1, further comprising pre-computing the possible results for the convolution layer and storing the pre-computed results in the look-up table.

19. A method as recited in claim 1, wherein the input data comprises audio, and the result comprising an indication of whether an audio sample matches a particular pattern.

9. A method comprising:

obtaining, by at least one computing device, input data that is to be analyzed by a convolutional neural network, the convolutional neural network including multiple layers, one of the multiple layers including a convolution layer, the convolution layer including one or more filters, each of the one or more filters having been binarized by being converted from floating point to binary values;

binarizing, by the at least one computing device, the input data by converting the input data to a set of binary vectors;

determining, by the at least one computing device, at the convolution layer, a convolution of the binarized one or more filters over the binarized input data, the determining including accessing, based on a binary vector of the set of binary vectors, a look-up table storing pre-computed results for the convolution layer, the pre-computed results used as the convolution of the binarized one or more filters over the binarized input data; and

outputting, by the at least one computing device, a result of the determining.

10. A method as recited in claim 9, the binarizing the input data comprising converting the input data into a first set of coefficients and binary vectors, and the one or more filters having been binarized by being converted into a second set of coefficients and binary vectors.

11. A method as recited in claim 9, the result comprising an indication of one or more faces detected in the image.

12. A method as recited in claim 9, the multiple layers including multiple convolution layers, each of the multiple convolution layers having been binarized by being converted from floating point to binary values.

13. A method as recited in claim 9, the binarized one or more filters including a first set of binary vectors and the binarized input data including a second set of binary vectors, the determining a convolution of the binarized one or more filters over the binarized input data comprising using bitwise logical AND and population count instructions to compute an inner product of one of the first set of binary vectors and one of the second set of binary vectors.

14. A convolutional neural network system comprising:

a data access module implemented by at least one computing device to obtain input data;

a convolutional neural network implemented by the at least one computing device and including multiple layers, one of the multiple layers including a convolution layer, the convolution layer including one or more filters, each of the one or more filters having been binarized by being converted from floating point to binary values as a first set of binary vectors;

a binarization module implemented by the at least one computing to binarize the input data by converting the input data to binary values as a second set of binary vectors;

the convolutional neural network being further configured by the at least one computing device to determine, at the convolution layer, a convolution of the binarized one or more filters over the binarized input data using bitwise logical AND and population count instructions to compute an inner product of one of the first set of binary vectors and one of the second set of binary vectors; and

an output module implemented by the at least one computing device to output a result of the analyzing.

15. A convolutional neural network system as recited in claim 14, the binarization module being configured to binarize the input data by converting the input data into a set of coefficients and binary vectors.

16. A convolutional neural network system as recited in claim 14, the one or more filters having been binarized by being converted into a set of coefficients and binary vectors.

17. A convolutional neural network system as recited in claim 14, the input data comprising an image, and the result comprising an indication of one or more faces detected in the image.

18. A convolutional neural network system as recited in claim 14, the binarized input data including a set of binary vectors, the determining a convolution of the binarized one or more filters over the binarized input data comprising accessing, based on a binary vector of the set of binary vectors, a look-up table storing pre-computed results for the convolution layer, and using the pre-computed results as the convolution of the binarized one or more filters over the binarized input data.

20. A convolutional neural network system as recited in claim 14, wherein the input data comprises audio, and the result comprising an indication of whether an audio sample matches a particular pattern.