PERFORMING MULTI-CONVOLUTION OPERATIONS IN A PARALLEL PROCESSING SYSTEM

• US 20160062947A1
• Filed: 08/27/2015
• Published: 03/03/2016
• Est. Priority Date: 08/29/2014
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: NVIDIA Corporation
• Inventors: CHETLUR, Sharanyan, CATANZARO, Bryan
• Granted Patent: US 10,223,333 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 17/153 Multidimensional correlation or convolution
• CPC Class Codes: G06N 3/045 Combinations of networks

Abstract

In one embodiment of the present invention a convolution engine configures a parallel processing pipeline to perform multi-convolution operations. More specifically, the convolution engine configures the parallel processing pipeline to independently generate and process individual image tiles. In operation, for each image tile, the pipeline calculates source locations included in an input image batch. Notably, the source locations reflect the contribution of the image tile to an output tile of an output matrix—the result of the multi-convolution operation. Subsequently, the pipeline copies data from the source locations to the image tile. Similarly, the pipeline copies data from a filter stack to a filter tile. The pipeline then performs matrix multiplication operations between the image tile and the filter tile to generate data included in the corresponding output tile. To optimize both on-chip memory usage and execution time, the pipeline creates each image tile in on-chip memory as-needed.

Claims

1. A computer-implemented method for performing a multi-convolution operation, the method comprising:

calculating a first source location included in an image batch that is stored in a first memory based on a first destination location included in a first image tile that is stored in a second memory;

copying data from the first source location to the first destination location;

copying data from a filter source location included in a filter stack that is stored in the first memory to a filter destination location included in a first filter tile that is stored in the second memory; and

performing one or more matrix multiplication operations between the first image tile and the first filter tile to generate a first output tile associated with an output matrix that is stored in the second memory.

2. The computer-implemented method of claim 1, wherein the first memory comprises off-chip memory and the second memory comprises on-chip memory.

3. The computer-implemented method of claim 1, further comprising calculating the filter source location based on the first destination location.

4. The computer-implemented method of claim 1, wherein calculating the first source location comprises:

associating the first destination location with a first virtual location included in a virtual image matrix; and

performing one or more indexing operations that map the first virtual location to a location included in the image batch.

5. The computer-implemented method of claim 4, wherein associating the first destination location comprises performing one or more arithmetic calculations based on at least one of a size of the second memory and the number of threads included in a thread group.

6. The computer-implemented method of claim 1, further comprising, assigning the first image tile to a first thread group, and configuring a first thread included in the first thread group to calculate the first source location based on the first destination location.

7. The computer-implemented method of claim 6, further comprising, assigning a second image tile to a second thread group, and configuring a second thread included in the second thread group to calculate a second source location included in the image batch based on a second destination location included in the second image tile.

8. The computer-implemented method of claim 7, wherein the first source location and the second source location are computed substantially in parallel.

9. The computer-implemented method of claim 7, further comprising discarding the first image tile prior to calculating the second source location.

10. The computer-implemented method of claim 1, further comprising performing one or more output formatting operations based on the output matrix to generate an output batch, and storing the output batch in the first memory.

11. The method of claim 10, wherein the image batch comprises a first layer included in a convolutional neural network, and the output batch comprises a second layer included in the convolutional neural network.

12. A non-transitory, computer-readable storage medium including instructions that, when executed by a processor, cause the processor to perform a multi-convolution operation, by performing the steps of:

calculating a first source location included in an image batch that is stored in a first memory based on a first destination location included in a first image tile that is stored in a second memory;

copying data from the first source location to the first destination location;

copying data from a filter source location included in a filter stack that is stored in the first memory to a filter destination location included in a first filter tile that is stored in the second memory; and

performing one or more matrix multiplication operations between the first image tile and the first filter tile to generate a first output tile associated with an output matrix that is stored in the second memory.

13. The non-transitory computer-readable storage medium of claim 12, wherein the first memory comprises off-chip memory and the second memory comprises on-chip memory.

14. The non-transitory computer-readable storage medium of claim 12, further comprising calculating the filter source location based on the first destination location.

15. The non-transitory computer-readable storage medium of claim 12, further comprising, assigning the first image tile to a first thread group, and configuring a first thread included in the first thread group to calculate the first source location based on the first destination location.

16. The non-transitory computer-readable storage medium of claim 15, further comprising, assigning a second image tile to a second thread group, and configuring a second thread included in the second thread group to calculate a second source location included in the image batch based on a second destination location included in the second image tile.

17. The non-transitory computer-readable storage medium of claim 16, wherein the first source location and the second source location are computed substantially in parallel.

18. The non-transitory computer-readable storage medium of claim 12, wherein a plurality of dimensions of the image batch comprise a batch size, a total number of color planes, an image width, and an image height.

19. The non-transitory computer-readable storage medium of claim 12, wherein a plurality of dimensions of the filter stack comprise a total number of filter sets, a total number of feature planes, a filter width, and a filter height.

20. A system configured to perform a multi-convolution operation, the system comprising:

a first memory;

a second memory; and

a convolution engine coupled to both the first memory and the second memory, and configured to;

calculate a first source location included in an image batch that is stored in the first memory based on a first destination location included in a first image tile that is stored in the second memory,

copy data from the first source location to the first destination location,

copy data from a filter source location included in a filter stack that is stored in the first memory to a filter destination location included in a first filter tile that is stored in the second memory, and

perform one or more matrix multiplication operations between the first image tile and the first filter tile to generate a first output tile associated with an output matrix that is stored in the second memory.