System and Method for Parallelizing and Accelerating Learning Machine Training and Classification Using a Massively Parallel Accelerator

• US 20090304268A1
• Filed: 06/04/2009
• Published: 12/10/2009
• Est. Priority Date: 06/04/2008
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Sankaradass, Murugan, Durdanovic, Igor, Jakkula, Venkata, Graf, Hans Peter, Cadambi, Srihari, Cosatto, Eric, Chakradhar, Srimat T.
• Granted Patent: US 8,359,281 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]

Abstract

A method system for training an apparatus to recognize a pattern includes providing the apparatus with a host processor executing steps of a machine learning process; providing the apparatus with an accelerator including at least two processors; inputting training pattern data into the host processor; determining coefficient changes in the machine learning process with the host processor using the training pattern data; transferring the training data to the accelerator; determining kernel dot-products with the at least two processors of the accelerator using the training data; and transferring the dot-products back to the host processor.

Claims

1. A method for training an apparatus to recognize a pattern, the method comprising the steps of:

providing the apparatus with a host processor executing steps of a machine learning process;

providing the apparatus with an accelerator including at least two processors;

inputting training pattern data into the host processor;

transferring the training data to the accelerator;

determining coefficient changes in the machine learning process with the host processor using the training pattern data;

transferring indices pertaining to one or more training vectors to the accelerator

determining kernel dot-products with the at least two processors of the accelerator using the training data; and

transferring the dot-products back to the host processor.

2. The method of claim 1, further comprising the step of determining kernels of the machine learning process with the host processor using the kernel dot-products.

3. The method of claim 2, further comprising the step of determining gradients of the machine learning process with the host processor using the kernels.

4. The method of claim 1, further comprising the step of reducing the precision of the training data with the host processor prior to the step of transferring the training data to the accelerator.

5. The method of claim 4, further comprising the step of reducing the precision of the dot-products with the accelerator prior to the step of transferring the dot-products back to the host processor.

6. The method of claim 1, further comprising the step of reducing the precision of the kernel dot-products with the accelerator prior to the step of transferring the dot-products back to the host processor.

7. The method of claim 6, wherein the accelerator further includes a memory bank associated with each one of the at least two processors, and further comprising the step of partitioning the reduced precision kernel dot-products into groups and storing each of the groups of the kernel dot-products in one of the memory banks prior to the step of transferring the kernel dot-products back to the host processor.

8. The method of claim 1, wherein the kernel dot-products are determined in a parallel manner with the at least two processors of the accelerator.

9. The method of claim 1, wherein the kernel dot-products are determined in separate and discrete chunks.

10. The method of claim 1, wherein the accelerator further includes a memory banks associated with each one of the at least two processors, and further comprising the step of partitioning the kernel dot-products into groups and storing each of the groups of the kernel dot-product in one of the memory banks prior to the step of transferring the kernel dot-products back to the host processor.

11. A system for training an apparatus to recognize a pattern, the system comprising:

a host processor of the apparatus for determining coefficient changes of a machine learning process from input training data;

an accelerator including at least two processors for determining kernel dot-products using the training data; and

at least one conduit for transferring the training data from the host processor to the accelerator and for transferring the kernel dot-products from the accelerator to the host processor.

12. The system of claim 11, wherein the host processor uses the kernel dot-products to determine kernels of the machine learning process.

13. The system of claim 12, wherein the host processor uses the kernels to determine gradients of the machine learning process.

14. The system of claim 11, wherein the host processor reduces the precision of the training data prior to its transfer to the accelerator.

15. The system of claim 14, wherein the accelerator reduces the precision of the kernel dot-products prior to their transfer to the host processor.

16. The system of claim 11, wherein the accelerator reduces the precision of the kernel dot-products prior to their transfer to the host processor.

17. The system of claim 16, wherein the accelerator further includes a memory bank associated with each one of the at least two processors, and wherein the kernel dot-products are partitioned into groups and each of the groups of the kernel dot-products are stored in one of the memory banks prior to being transferred to the host processor.

18. The system of claim 11, wherein the kernel dot-products are determined in a parallel manner by the at least two processors of the accelerator.

19. The system of claim 11, wherein the kernel dot-products are determined in separate and discrete chunks.

20. The system of claim 11, wherein the accelerator further includes a memory bank associated with each one of the at least two processors, and wherein the kernel dot-products are partitioned into groups and each of the groups of the kernel dot-products are stored in one of the memory banks prior to being transferred to the host processor.

21. A method for recognizing patterns, the method comprising the steps of:

providing host processor executing steps of a support vector machine learning process;

providing an accelerator including at least two processors and a memory bank associated with each of the at least two processors;

storing support vectors in the memory banks of the accelerator;

reducing the precision of unlabeled pattern data with the host processor;

transferring unlabeled pattern data from the host processor to the accelerator;

calculating labels for the unlabeled pattern data with the at least two processors of the accelerator using the support vectors stored in the memory banks of the accelerator; and

transferring the labeled pattern data back to the host processor.