MACHINE LEARNING CLASSIFICATION ON HARDWARE ACCELERATORS WITH STACKED MEMORY

• US 20160379137A1
• Filed: 06/29/2015
• Published: 12/29/2016
• Est. Priority Date: 06/29/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Burger, Douglas C., Chiou, Derek, Chung, Eric, Putnam, Andrew R.
• Granted Patent: US 10,452,995 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A method is provided for processing on an acceleration component a machine learning classification model. The machine learning classification model includes a plurality of decision trees, the decision trees including a first amount of decision tree data. The acceleration component includes an acceleration component die and a memory stack disposed in an integrated circuit package. The memory die includes an acceleration component memory having a second amount of memory less than the first amount of decision tree data. The memory stack includes a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW. The method includes slicing the model into a plurality of model slices, each of the model slices having a third amount of decision tree data less than or equal to the second amount of memory, storing the plurality of model slices on the memory stack, and for each of the model slices, copying the model slice to the acceleration component memory, and processing the model slice using a set of input data on the acceleration component to produce a slice result.

Claims

1. A method for processing on an acceleration component a machine learning classification model comprising a plurality of decision trees, the decision trees comprising a first amount of decision tree data, the acceleration component comprising an acceleration component die and a memory stack disposed in an integrated circuit package, the memory die comprising an acceleration component memory having a second amount of memory less than the first amount of decision tree data, the memory stack comprising a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW, the method comprising:

slicing the model into a plurality of model slices, each of the model slices having a third amount of decision tree data less than or equal to the second amount of memory;

storing the plurality of model slices on the memory stack; and

for each of the model slices;

copying the model slice to the acceleration component memory; and

processing the model slice using a set of input data on the acceleration component to produce a slice result.

2. The method of claim 1, wherein the acceleration component comprises one or more of a field-programmable gate array device, a massively parallel processor array device, a graphics processing unit, and an application-specific integrated circuit.

3. The method of claim 1, wherein the memory stack comprises one or more memory die.

4. The method of claim 1, wherein the acceleration component further comprises an interposer, and the acceleration component die and the memory stack are disposed on the interposer.

5. The method of claim 1, wherein the memory stack is disposed above the acceleration component die.

6. The method of claim 1, further comprising:

storing a plurality sets of input data on the memory stack; and

for each of the model slices;

copying the model slice to the acceleration component memory; and

processing the model slice using each of the plurality of sets of input data on the acceleration component.

7. The method of claim 1, further comprising summing the slice results for each of the slices.

8. The method of claim 1, wherein providing an acceleration component further comprises providing the an acceleration component in a server unit component in a data center server.

9. A system for processing a machine learning classification model comprising a plurality of decision trees, the decision trees comprising a first amount of decision tree data, the system comprising:

an acceleration component comprising an acceleration component die and a memory stack disposed in an integrated circuit package, the memory die comprising an acceleration component memory having a second amount of memory less than the first amount of decision tree data, the memory stack comprising a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW; and

a model slicing component configured to slice the model into a plurality of model slices, each of the model slices having a third amount of decision tree data less than or equal to the second amount of memory, and store the plurality of model slices on the memory stack,

wherein for each of the model slices, the acceleration component is configured to copy the model slice to the acceleration component memory and is configured to process the model slice using a set of input data on the acceleration component to produce a slice result.

10. The system of claim 9, wherein the acceleration component comprises one or more of a field-programmable gate array device, a massively parallel processor array device, a graphics processing unit, and an application-specific integrated circuit.

11. The system of claim 9, wherein the memory stack comprises one or more memory die.

12. The system of claim 9, wherein the acceleration component further comprises an interposer, and the acceleration component die and the memory stack are disposed on the interposer.

13. The system of claim 9, wherein the memory stack is disposed above the acceleration component die.

14. The system of claim 9, wherein the acceleration component is configured to:

store a plurality sets of input data on the memory stack; and

for each of the model slices;

copy the model slice to the acceleration component memory; and

process the model slice using each of the plurality of sets of input data on the acceleration component.

15. The system of claim 9, wherein the acceleration component is configured to sum the slice results for each of the slices.

16. The system of claim 9, further comprising a server unit component comprising the acceleration component, wherein the server unit component comprises a data center.

17. A method for processing on an acceleration component a machine learning classification model comprising a plurality of decision trees, the decision trees comprising a first amount of decision tree data, the acceleration component comprising an acceleration component die and a memory stack disposed in an integrated circuit package, the memory die comprising an acceleration component memory having a second amount of memory less than the first amount of decision tree data, the memory stack comprising a memory bandwidth greater than about 50 GB/sec and a power efficiency of greater than about 20 MB/sec/mW, the method comprising:

storing the plurality of decision trees on the memory stack;

for each of the decision trees;

copying a first portion of the decision tree to the acceleration component memory;

processing the first portion using a set of input data on the acceleration component; and

copying a second portion of decision tree to the acceleration component memory based on a result of processing the first portion of the decision tree.

18. The method of claim 17, wherein the first portion of each decision tree comprises a top portion of the decision tree.

19. The method of claim 17, wherein the acceleration component comprises one or more of a field-programmable gate array device, a massively parallel processor array device, a graphics processing unit, and an application-specific integrated circuit.

20. The method of claim 17, wherein the acceleration component further comprises an interposer, and the acceleration component die and the memory stack are disposed on the interposer.