Systems and methods for processing convolutional neural network operations using textures

• US 10,339,443 B1
• Filed: 02/24/2017
• Issued: 07/02/2019
• Est. Priority Date: 02/24/2017
• Status: Active Grant

• Current Assignee: GoPro, Inc.
• Original Assignee: GoPro, Inc.
• Inventors: Medioni, Tom
• Primary Examiner(s): Thomas, Mia M
• Application Number:
  US15/441,963
• Time in Patent Office: 858 Days
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/06 Physical realisation, i.e. hardware implementation of neural networks, neurons or parts of neurons
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/80 Shading
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: H03H 17/0202 Two or more dimensional filters; Filters for complex signals multidimensional convolutions G06F17/153

Abstract

Convolutional neural network information may define a convolutional neural network including layers. The layers may define operations on an input to the convolutional neural network. The layers in the convolutional neural network information may be formatted as shaders. Input information defining the input to the convolutional neural network may be accessed. The input information may be formatted as an array of textures. The shaders may be applied to the textures to effectuate processing the input to the convolutional neural network through the layers of the convolutional neural network. One or more results may be obtained from applying the shaders to the array of textures.

Claims

1. A system for processing convolutional neural network operations using textures, the system comprising:

physical storage media storing convolutional neural network information, the convolutional neural network information defining a convolutional neural network, the convolutional neural network including layers, the layers defining operations on an input to the convolutional neural network, wherein the layers in the convolutional neural network information are formatted as shaders; and

one or more physical processors configured by machine-readable instructions to;

access the input to the convolutional neural network;

format the input as an array of textures;

store the array of textures as input information;

access the input information, the input information defining the input to the convolutional neural network, the input information formatted as the array of textures, the textures characterized by a width, a height, and a number of channels;

apply the shaders to the array of textures, wherein applying the shaders to the array of textures effectuates processing the input to the convolutional neural network through the layers of the convolutional neural network; and

obtain one or more results from applying the shaders to the array of textures.

2. The system of claim 1, wherein the one or more physical processors are further configured by machine-readable instructions to:

access layer information, the layer information defining one or more of the layers of the convolutional neural network;

format one or more of the operations defined by the one or more of the layers as one or more of the shaders; and

add the shaders to the convolutional neural network information.

3. The system of claim 1, wherein the number of channels of the textures is four.

4. The system of claim 3, wherein individual shaders output information for four channels.

5. The system of claim 1, further comprising a graphics processing unit, wherein the shaders are applied to the array of textures by the graphics processing unit.

6. The system of claim 5, further comprising a graphics processing unit memory, wherein one or more inputs to the shaders and one or more outputs of the shaders are stored in the graphics processing unit memory.

7. The system of claim 5, wherein the graphics processing unit is included in a mobile device.

8. The system of claim 1, wherein the layers of the convolutional neural network include one or more convolution layers, the convolution layers processed using binary convolution.

9. The system of claim 1, wherein the array of textures are stored using an int32 structure, the int32 structure including four int8 structures, an individual int8 structure storing a four by two patch of a single channel of an individual texture.

10. A method for processing convolutional neural network operations using textures, the method comprising:

accessing the input to a convolutional neural network;

formatting the input as an array of textures;

storing the array of textures as input information

accessing the input information, the input information defining the input to the convolutional neural network, the convolutional neural network defined by convolutional neural network information, the convolutional neural network including layers, the layers defining operations on the input to the convolutional neural network, wherein the layers in the convolutional neural network information are formatted as shaders and the input information is formatted as the array of textures, the textures characterized by a width, a height, and a number of channels;

applying the shaders to the array of textures, wherein applying the shaders to the array of textures effectuates processing the input to the convolutional neural network through the layers of the convolutional neural network; and

obtaining one or more results from applying the shaders to the array of textures.

11. The method of claim 10, further comprising:

accessing layer information, the layer information defining one or more of the layers of the convolutional neural network;

formatting one or more of the operations defined by the one or more of the layers as one or more of the shaders; and

adding the shaders to the convolutional neural network information.

12. The method of claim 10, wherein the number of channels of the textures is four.

13. The method of claim 12, wherein individual shaders output information for four channels.

14. The method of claim 10, wherein the shaders are applied to the array of textures by a graphics processing unit.

15. The method of claim 14, wherein one or more inputs to the shaders and one or more outputs of the shaders are stored in a graphics processing unit memory.

16. The method of claim 14, wherein the graphics processing unit is included in a mobile device.

17. The method of claim 10, wherein the layers of the convolutional neural network include one or more convolution layers, the convolution layers processed using binary convolution.

18. The method of claim 10, wherein the array of textures are stored using an int32 structure, the int32 structure including four int8 structures, an individual int8 structure storing a four by two patch of a single channel of an individual texture.