Sharing preprocessing, computations, and hardware resources between multiple neural networks

• US 20200134467A1
• Filed: 10/23/2019
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: Runai Labs Ltd.
• Original Assignee: Run.Ai Labs Ltd.
• Inventors: Dar, Ronen, Anholt, Micha
• Granted Patent: US 11,551,095 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 9/3885 using a plurality of independent parallel functional units
• CPC Class Codes: G06F 9/5033 considering data affinity
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

A method for training a Neural-Network (NN), the method includes receiving a plurality of NN training tasks, each training task including a respective preprocessing phase that preprocesses data to be provided as input data to the NN, and (ii) a respective computation phase that trains the NN using the preprocessed data. The plurality of NN training tasks is executed, including: (a) a commonality is identified between the input data required by computation phases of two or more of the training tasks, and (b) in response to identifying the commonality, one or more preprocessing phases are executed that produce the input data jointly for the two or more training tasks.

Claims

1. A method for training a Neural-Network (NN), the method comprising:

receiving a plurality of NN training tasks, each training task comprising (i) a respective preprocessing phase that preprocesses data to be provided as input data to the NN, and (ii) a respective computation phase that trains the NN using the preprocessed data; and

executing the plurality of NN training tasks, including;

identifying a commonality between the input data required by computation phases of two or more of the training tasks; and

in response to identifying the commonality, executing one or more preprocessing phases that produce the input data jointly for the two or more training tasks.

2. The method according to claim 1, wherein executing the plurality of NN training tasks comprises, in response to identifying the commonality, assigning two more computation tasks to a same group of one or more processors.

3. The method according to claim 1, wherein computation phases of the training tasks are executed by multiple processors, and wherein executing the plurality of NN training tasks comprises assigning the computation phases to the processors in accordance with predefined assignment criterion.

4. The method according to claim 3, wherein the assignment criterion aims to minimize a total execution time of the training tasks.

5. The method according to claim 3, wherein the assignment criterion aims to minimize idle times during computation phases.

6. The method according to claim 3, wherein assigning the computation phases comprises estimating durations of execution of the computation phases, and assigning the computation phases to the processors based on the durations of execution.

7. The method according to claim 6, wherein estimating the durations comprises re-estimating the durations during execution of the computation phases, and reassigning one or more of the computation phases to the processors based on the re-estimated durations of execution.

8. The method according to claim 1, wherein preprocessing phases of the training tasks are executed by multiple processors, and wherein executing the plurality of NN training tasks comprises assigning the preprocessing phases to the processors in accordance with predefined assignment criterion.

9. The method according to claim 8, wherein the assignment criterion aims to minimize a total execution time of the training tasks.

10. The method according to claim 8, wherein assigning the preprocessing phases comprises estimating durations of execution of the preprocessing phases, and assigning the preprocessing phases to the processors based on the durations of execution.

11. The method according to claim 1, wherein receiving the plurality of NN training tasks comprises deciding on a maximal number of training tasks for which to produce the input data jointly, based on a total execution time of the training tasks.

12. A system for training a Neural-Network (NN), the system comprising:

an interface, configured to receive a plurality of NN training tasks, each training task comprising (i) a respective preprocessing phase that preprocesses data to be provided as input data to the NN, and (ii) a respective computation phase that trains the NN using the preprocessed data; and

one or more processors, configured to execute the plurality of NN training tasks, including;

identifying a commonality between the input data required by computation phases of two or more of the training tasks; and

in response to identifying the commonality, executing one or more preprocessing phases that produce the input data jointly for the two or more training tasks.

13. The system according to claim 12, wherein the one or more processors are configured to execute the plurality of NN training tasks by, in response to identifying the commonality, assigning two or more computation tasks to a same group of one or more processors.

14. The system according to claim 12, wherein the one or more processors are configured to execute the plurality, of NN training tasks by assigning the computation phases to the processors in accordance with a predefined assignment criterion.

15. The system according to claim 14, wherein the assignment criterion aims to minimize a total execution time of the training tasks.

16. The system according to claim 14, wherein the assignment criterion aims to minimize idle times during computation phases.

17. The system according to claim 14, wherein the multiple processors are configured to assign the computation phases by estimating durations of execution of the computation phases, and assigning the computation phases to the processors based on the durations of execution.

18. The system according to claim 17, wherein the multiple processors are configured to re-estimate the durations during execution of the computation phases, and to reassign one or more of the computation phases to the processors based on the re-estimated durations of execution.

19. The system according to claim 12, wherein the one or more processors are configured to execute the plurality of NN training tasks by assigning the preprocessing phases to the processors in accordance with a predefined assignment criterion.

20. The system according to claim 19, wherein the assignment criterion aims to minimize a total execution time of the training tasks.

21. The system according to claim 19, wherein the multiple processors are configured to assign the preprocessing phases by estimating durations of execution of the preprocessing phases, and assigning the preprocessing phases to the processors based on the durations of execution.

22. The system according to claim 12, the multiple processors are configured to decide on a maximal number of training tasks for which to produce the input data jointly based on a total execution time of the training tasks.

23. A computer software product, the product comprising a tangible non-transitory computer readable medium in which program instructions are stored, which instructions, when read by one or more processors, cause the one or more processors to:

receive a plurality of NN training tasks, each training task comprising (i) a respective preprocessing phase that preprocesses data to be provided as input data to the NN, and (ii) a respective computation phase that trains the NN using the preprocessed data; and

execute the plurality of NN training tasks, including;

identifying a commonality between the input data required by computation phases of two or more of the training tasks; and

in response to identifying the commonality, executing one or more preprocessing phases that produce the input data jointly for the two or more training tasks.