Interoperable neural network operation scheduler
First Claim
1. A computer-implemented method comprising:
- obtaining information describing a first neural network (NN) model in an interoperable data format;
processing the information to determine a plurality of operations representing discrete execution units of the first NN model;
identifying attributes from the information to create descriptor information as annotations describing execution criteria of each operation from the plurality of operations;
obtaining information about hardware functionality and availability for a plurality of hardware processors configured to process the first NN model;
matching a first operation to a first selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the first operation with functionality of the plurality of hardware processors;
matching a second operation to a second selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the second operation with functionality of the plurality of hardware processors;
creating a first schedule entry and executing the first operation on the first selected hardware processor; and
creating a second schedule entry and executing the second operation on the second selected hardware processor,wherein hardware performance capabilities of the first selected hardware processor and the second selected hardware processor are different,at least one of the annotations for the first operation align with capabilities of the first selected hardware processor not available for the second selected hardware processor, andat least one of the annotations for the second operation align with capabilities of the second selected hardware processor not available for the first selected hardware processor.
1 Assignment
0 Petitions
Accused Products
Abstract
A Neural Network (NN) scheduler and techniques to implement features of different possible NN schedulers are disclosed. In a first example, an NN scheduler that accepts NN models in an interoperable format and performs optimizations on this interoperable format as part of converting it to a run-time format is provided. In a second example, an NN scheduler analyzes operations and annotations associated with those operations to determine scheduling options based on hardware availability, data availability, hardware efficiency, processor affinity, etc. In a third example, an NN scheduler that may be integrated with a feed-back loop to recognize actual run-time attributes may be used to “learn” and adapt to change its future scheduling behavior. Each of these examples may be integrated individually, or together, to provide an NN scheduler that optimizes and adapts processing functions for an NN model either prior to processing or for just-in-time determination of operation scheduling.
-
Citations
20 Claims
-
1. A computer-implemented method comprising:
-
obtaining information describing a first neural network (NN) model in an interoperable data format; processing the information to determine a plurality of operations representing discrete execution units of the first NN model; identifying attributes from the information to create descriptor information as annotations describing execution criteria of each operation from the plurality of operations; obtaining information about hardware functionality and availability for a plurality of hardware processors configured to process the first NN model; matching a first operation to a first selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the first operation with functionality of the plurality of hardware processors; matching a second operation to a second selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the second operation with functionality of the plurality of hardware processors; creating a first schedule entry and executing the first operation on the first selected hardware processor; and creating a second schedule entry and executing the second operation on the second selected hardware processor, wherein hardware performance capabilities of the first selected hardware processor and the second selected hardware processor are different, at least one of the annotations for the first operation align with capabilities of the first selected hardware processor not available for the second selected hardware processor, and at least one of the annotations for the second operation align with capabilities of the second selected hardware processor not available for the first selected hardware processor. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A non-transitory computer readable medium comprising computer executable instructions stored thereon that when executed by one or more processing units, perform a method to provide a neural network (NN) operation creation and scheduling function, the method comprising:
-
obtaining information describing a first neural network (NN) model in an interoperable data format; processing the information to determine a plurality of operations representing discrete execution units of the first NN model; identifying attributes from the information to create descriptor information as annotations describing execution criteria of each operation from the plurality of operations; obtaining information about hardware functionality and availability for a plurality of hardware processors configured to process the first NN model; matching a first operation to a first selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the first operation with functionality of the plurality of hardware processors; matching a second operation to a second selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the second operation with functionality of the plurality of hardware processors; creating a first schedule entry and executing the first operation on the first selected hardware processor; and creating a second schedule entry and executing the second operation on the second selected hardware processor, wherein hardware performance capabilities of the first selected hardware processor and the second selected hardware processor are different, at least one of the annotations for the first operation align with capabilities of the first selected hardware processor not available for the second selected hardware processor, and at least one of the annotations for the second operation align with capabilities of the second selected hardware processor not available for the first selected hardware processor. - View Dependent Claims (15, 16, 17)
-
-
18. A computer system, comprising:
-
a network communications interface; a memory; and one or more processing units, communicatively coupled to the memory and the network communications interface, wherein the memory stores instructions, that when executed by the one or more processing units, cause the one or more processing units to provide a neural network (NN) operation creation and scheduling function, the NN operation creation and scheduling function configured to; obtain information describing a first neural network (NN) model in an interoperable data format; process the information to determine a plurality of operations representing discrete execution units of the first NN model; identify attributes from the information to create descriptor information as annotations describing execution criteria of each operation from the plurality of operations; obtain information about hardware functionality and availability for a plurality of hardware processors configured to process the first NN model; match a first operation to a first selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the first operation with functionality of the plurality of hardware processors; match a second operation to a second selected hardware processor from the plurality of hardware processors, in part, by comparing the annotations for the second operation with functionality of the plurality of hardware processors; create a first schedule entry and executing the first operation on the first selected hardware processor; and create a second schedule entry and executing the second operation on the second selected hardware processor, wherein hardware performance capabilities of the first selected hardware processor and the second selected hardware processor are different, at least one of the annotations for the first operation align with capabilities of the first selected hardware processor not available for the second selected hardware processor, and at least one of the annotations for the second operation align with capabilities of the second selected hardware processor not available for the first selected hardware processor. - View Dependent Claims (19, 20)
-
Specification