Automated transition from non-neuromorphic to neuromorphic processing

• US 10,346,211 B2
• Filed: 07/19/2018
• Issued: 07/09/2019
• Est. Priority Date: 02/05/2016
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Bequet, Henry Gabriel Victor, Chen, III, Huina, Du, Juan
• Primary Examiner(s): Patel, Hiren P
• Application Number:
  US16/039,949
• Publication Number:
  US 20190026155A1
• Time in Patent Office: 355 Days
• CPC Class Codes: G06F 16/9014 hash tables
• CPC Class Codes: G06F 16/90344 by using string matching techniques
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: H04L 67/025 for remote control or remote monitoring of applications
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 69/40 for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection management of faults, events, alarms or notifications in data switching networks H04L41/06

Abstract

An apparatus includes a processor to: assign a portion of currently available instruction-based processing resources to a first non-neuromorphic performance of an analytical function; in response to availability of sufficient remaining processing resources for a first neuromorphic performance of the analytical function with the same input values, assign a portion of the remaining processing resources to the first neuromorphic performance; analyze the output values generated by the first neuromorphic and non-neuromorphic performances to determine a degree of accuracy of the neural network in performing the analytical function; in response to at least the degree of accuracy exceeding a predetermined threshold, assign a portion of currently available processing resources to a second neuromorphic performance of the analytical function; and in response to availability of sufficient remaining processing resources for a second non-neuromorphic performance of the analytical function, assign a portion of the remaining instruction-based processing resources to the second non-neuromorphic performance.

Claims

1. An apparatus comprising a processor and a storage to store instructions that, when executed by the processor, cause the processor to perform operations comprising:

receive a first request to perform an analytical function with a first data set comprising multiple sets of input values to generate multiple corresponding sets of output values;

assign, as part of a first assignment of processing resources, at least a portion of currently available instruction-based processing resources to a first non-neuromorphic performance of the analytical function with the first data set, and with at least a predetermined level of throughput, through execution of instructions implementing the analytical function by one or more processor cores;

after the assignment of instruction-based processing resources to the first non-neuromorphic performance, analyze a state of remaining processing resources; and

in response to current availability of sufficient remaining processing resources to enable a first neuromorphic performance of the analytical function with at least a subset of the sets of input values of the first data set through use of a neural network defined by at least a set of hyperparameters, and at least partly in parallel with the first non-neuromorphic performance;

assign, as part of the first assignment, at least a portion of the remaining processing resources to the first neuromorphic performance;

analyze the sets of output values generated from the subset of the sets of input values by the first neuromorphic performance relative to corresponding sets of output values generated by the first non-neuromorphic performance to determine a first degree of accuracy of the neural network in performing the analytical function; and

in response to at least the first degree of accuracy exceeding a predetermined higher threshold, in response to receipt of a second request from a requesting device to perform the analytical function with a second data set comprising multiple sets of input values to generate multiple corresponding sets of output values, and in response to current availability of sufficient processing resources to enable a second neuromorphic performance of the analytical function with the second data set through use of the neural network, and with at least the predetermined level of throughput;

assign, as part of a second assignment of processing resources, at least a portion of currently available processing resources to the second neuromorphic performance;

after the assignment of processing resources to the second neuromorphic performance, analyze a state of remaining instruction-based processing resources currently available; and

in response to current availability of sufficient remaining instruction-based processing resources to enable a second non-neuromorphic performance of the analytical function with at least a subset of the sets of input values of the second data set through execution of instructions implementing the analytical function by one or more processor cores, and at least partly in parallel with the second neuromorphic performance;

assign, as part of the second assignment, at least a portion of the remaining instruction-based processing resources to the second non-neuromorphic performance;

analyze the sets of output values generated from the subset of the sets of input values by the second neuromorphic performance relative to corresponding sets of output values generated by the second non-neuromorphic performance to determine a second degree of accuracy of the neural network in performing the analytical function; and

in response to at least the second degree of accuracy exceeding the predetermined higher threshold, transmit the multiple sets of output values generated from the second data set by the second neuromorphic performance to the requesting device.

2. The apparatus of claim 1, wherein the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises an assignment of one or more processor cores of one or more processors comprising at least one of one or more central processing units (CPUs), or one or more graphics processing units (GPUs).

3. The apparatus of claim 2, wherein:

the one or more processor cores of the one or more processors are processors distributed among multiple node devices of a grid of node devices; and

the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises an assignment of processing resources of the multiple node devices to the first non-neuromorphic performance.

4. The apparatus of claim 2, wherein:

the assignment of at least a portion of remaining processing resources to the first neuromorphic performance comprises an assignment of at least a subset of one or more remaining processor cores, or an assignment of at least a portion of each of one or more neuromorphic devices;

each of the remaining processor cores are programmable to instantiate at least a portion of the neural network;

each of the neuromorphic devices comprises at least one of a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC); and

each of the neuromorphic devices comprises sets of circuits that each implement an artificial neuron able to be included in the neural network.

5. The apparatus of claim 1, wherein:

the set of hyperparameters specifies a quantity of artificial neurons within the neural network; and

the analysis of the state of remaining processing resources after the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises a determination of whether sufficient processing resources are available to instantiate the specified quantity of artificial neurons.

6. The apparatus of claim 1, wherein the processor is caused, in response to at least the second degree of accuracy exceeding the predetermined higher threshold, in response to receipt of a third request from a requesting device to perform the analytical function with a third data set comprising multiple sets of input values to generate multiple corresponding sets of output values, and in response to current availability of sufficient processing resources to enable a third neuromorphic performance of the analytical function with the third data set through use of the neural network, and with at least the predetermined level of throughput, to:

assign, as part of a third assignment of processing resources, at least a portion of currently available processing resources to the third neuromorphic performance; and

refrain from assigning processing resources to a third non-neuromorphic performance of the analytical function with the third data set through execution of instructions implementing the analytical function by one or more processor cores.

7. The apparatus of claim 1, wherein the processor is caused, in response to at least the first degree of accuracy falling below a predetermined lower threshold that is lower than the predetermined higher threshold, provide an indication that the performance of the analytical function by the neural network is deemed insufficiently accurate to be used.

8. The apparatus of claim 7, wherein the processor is caused, in response to at least the first degree of accuracy falling below the predetermined higher threshold and above the predetermined lower threshold, to perform operations comprising:

assign at least a portion of currently available processing resources to instantiating the neural network in a training mode; and

use backpropagation to train the neural network with the multiple sets of input values of the first data set and the corresponding multiple sets of output values generated by the first non-neuromorphic performance.

9. The apparatus of claim 7, wherein:

the first neuromorphic performance and the first non-neuromorphic performance occur at least partly within a testing federated area maintained by one or more storage devices to store the first data set, a first neuromorphic job flow definition, at least one task routine to perform at least one task defined by the first neuromorphic job flow definition, a first non-neuromorphic job flow definition, and at least one task routine to perform at least one task defined by the first non-neuromorphic job flow definition;

the neural network was trained at least partly within a training federated area maintained by the one or more storage devices to store at least one training data set, a training job flow definition, and at least one task routine to perform at least one task defined by the training job flow definition; and

in response to at least the first degree of accuracy falling below the predetermined lower threshold, the processor is caused to transfer the indication that the performance of the analytical function by the neural network is deemed insufficiently accurate to be used from the testing federated area to the training federated area to trigger retraining of the neural network within another set of hyperparameters.

10. The apparatus of claim 9, wherein:

the second neuromorphic performance and the second non-neuromorphic performance occur at least partly within a usage federated area maintained by the one or more storage devices to store the second data set, a second neuromorphic job flow definition, at least one task routine to perform at least one task defined by the second neuromorphic job flow definition, a second non-neuromorphic job flow definition, and at least one task routine to perform at least one task defined by the second non-neuromorphic job flow definition; and

in response to at least the second degree of accuracy exceeding the predetermined higher threshold, the processor is caused to transfer neural network configuration data that defines the neural network from the testing federated area to the usage federated area to enable instantiation of the neural network to support the second neuromorphic performance.

11. A computer-program product tangibly embodied in a non-transitory machine-readable storage medium, the computer-program product including instructions operable to cause a processor to perform operations comprising:

receive a first request to perform an analytical function with a first data set comprising multiple sets of input values to generate multiple corresponding sets of output values;

assign, as part of a first assignment of processing resources, at least a portion of currently available instruction-based processing resources to a first non-neuromorphic performance of the analytical function with the first data set, and with at least a predetermined level of throughput, through execution of instructions implementing the analytical function by one or more processor cores;

after the assignment of instruction-based processing resources to the first non-neuromorphic performance, analyze a state of remaining processing resources; and

in response to current availability of sufficient remaining processing resources to enable a first neuromorphic performance of the analytical function with at least a subset of the sets of input values of the first data set through use of a neural network defined by at least a set of hyperparameters, and at least partly in parallel with the first non-neuromorphic performance;

assign, as part of the first assignment, at least a portion of the remaining processing resources to the first neuromorphic performance;

analyze the sets of output values generated from the subset of the sets of input values by the first neuromorphic performance relative to corresponding sets of output values generated by the first non-neuromorphic performance to determine a first degree of accuracy of the neural network in performing the analytical function; and

in response to at least the first degree of accuracy exceeding a predetermined higher threshold, in response to receipt of a second request from a requesting device to perform the analytical function with a second data set comprising multiple sets of input values to generate multiple corresponding sets of output values, and in response to current availability of sufficient processing resources to enable a second neuromorphic performance of the analytical function with the second data set through use of the neural network, and with at least the predetermined level of throughput;

assign, as part of a second assignment of processing resources, at least a portion of currently available processing resources to the second neuromorphic performance;

after the assignment of processing resources to the second neuromorphic performance, analyze a state of remaining instruction-based processing resources currently available; and

in response to current availability of sufficient remaining instruction-based processing resources to enable a second non-neuromorphic performance of the analytical function with at least a subset of the sets of input values of the second data set through execution of instructions implementing the analytical function by one or more processor cores, and at least partly in parallel with the second neuromorphic performance;

assign, as part of the second assignment, at least a portion of the remaining instruction-based processing resources to the second non-neuromorphic performance;

analyze the sets of output values generated from the subset of the sets of input values by the second neuromorphic performance relative to corresponding sets of output values generated by the second non-neuromorphic performance to determine a second degree of accuracy of the neural network in performing the analytical function; and

in response to at least the second degree of accuracy exceeding the predetermined higher threshold, transmit the multiple sets of output values generated from the second data set by the second neuromorphic performance to the requesting device.

12. The computer-program product of claim 11, wherein the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises an assignment of one or more processor cores of one or more processors comprising at least one of one or more central processing units (CPUs), or one or more graphics processing units (GPUs).

13. The computer-program product of claim 12, wherein:

the one or more processor cores of the one or more processors are processors distributed among multiple node devices of a grid of node devices; and

the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises an assignment of processing resources of the multiple node devices to the first non-neuromorphic performance.

14. The computer-program product of claim 12, wherein:

the assignment of at least a portion of remaining processing resources to the first neuromorphic performance comprises an assignment of at least a subset of one or more remaining processor cores, or an assignment of at least a portion of each of one or more neuromorphic devices;

each of the remaining processor cores are programmable to instantiate at least a portion of the neural network;

each of the neuromorphic devices comprises at least one of a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC); and

each of the neuromorphic devices comprises sets of circuits that each implement an artificial neuron able to be included in the neural network.

15. The computer-program product of claim 11, wherein:

the set of hyperparameters specifies a quantity of artificial neurons within the neural network; and

the analysis of the state of remaining processing resources after the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises a determination of whether sufficient processing resources are available to instantiate the specified quantity of artificial neurons.

16. The computer-program product of claim 11, wherein the processor is caused, in response to at least the second degree of accuracy exceeding the predetermined higher threshold, in response to receipt of a third request from a requesting device to perform the analytical function with a third data set comprising multiple sets of input values to generate multiple corresponding sets of output values, and in response to current availability of sufficient processing resources to enable a third neuromorphic performance of the analytical function with the third data set through use of the neural network, and with at least the predetermined level of throughput, to:

assign, as part of a third assignment of processing resources, at least a portion of currently available processing resources to the third neuromorphic performance; and

refrain from assigning processing resources to a third non-neuromorphic performance of the analytical function with the third data set through execution of instructions implementing the analytical function by one or more processor cores.

17. The computer-program product of claim 11, wherein the processor is caused, in response to at least the first degree of accuracy falling below a predetermined lower threshold that is lower than the predetermined higher threshold, provide an indication that the performance of the analytical function by the neural network is deemed insufficiently accurate to be used.

18. The computer-program product of claim 17, wherein the processor is caused, in response to at least the first degree of accuracy falling below the predetermined higher threshold and above the predetermined lower threshold, to perform operations comprising:

assign at least a portion of currently available processing resources to instantiating the neural network in a training mode; and

use backpropagation to train the neural network with the multiple sets of input values of the first data set and the corresponding multiple sets of output values generated by the first non-neuromorphic performance.

19. The computer-program product of claim 17, wherein:

the first neuromorphic performance and the first non-neuromorphic performance occur at least partly within a testing federated area maintained by one or more storage devices to store the first data set, a first neuromorphic job flow definition, at least one task routine to perform at least one task defined by the first neuromorphic job flow definition, a first non-neuromorphic job flow definition, and at least one task routine to perform at least one task defined by the first non-neuromorphic job flow definition;

the neural network was trained at least partly within a training federated area maintained by the one or more storage devices to store at least one training data set, a training job flow definition, and at least one task routine to perform at least one task defined by the training job flow definition; and

in response to at least the first degree of accuracy falling below the predetermined lower threshold, the processor is caused to transfer the indication that the performance of the analytical function by the neural network is deemed insufficiently accurate to be used from the testing federated area to the training federated area to trigger retraining of the neural network within another set of hyperparameters.

20. The computer-program product of claim 19, wherein:

the second neuromorphic performance and the second non-neuromorphic performance occur at least partly within a usage federated area maintained by the one or more storage devices to store the second data set, a second neuromorphic job flow definition, at least one task routine to perform at least one task defined by the second neuromorphic job flow definition, a second non-neuromorphic job flow definition, and at least one task routine to perform at least one task defined by the second non-neuromorphic job flow definition; and

in response to at least the second degree of accuracy exceeding the predetermined higher threshold, the processor is caused to transfer neural network configuration data that defines the neural network from the testing federated area to the usage federated area to enable instantiation of the neural network to support the second neuromorphic performance.

21. A computer-implemented method comprising:

receiving, by a processor, a first request to perform an analytical function with a first data set comprising multiple sets of input values to generate multiple corresponding sets of output values;

assigning, by the processor and as part of a first assignment of processing resources, at least a portion of currently available instruction-based processing resources to a first non-neuromorphic performance of the analytical function with the first data set, and with at least a predetermined level of throughput, through execution of instructions implementing the analytical function by one or more processor cores;

after the assignment of instruction-based processing resources to the first non-neuromorphic performance, analyzing, by the processor, a state of remaining processing resources; and

in response to current availability of sufficient remaining processing resources to enable a first neuromorphic performance of the analytical function with at least a subset of the sets of input values of the first data set through use of a neural network defined by at least a set of hyperparameters, and at least partly in parallel with the first non-neuromorphic performance;

assigning, by the processor and as part of the first assignment, at least a portion of the remaining processing resources to the first neuromorphic performance;

analyzing, by the processor, the sets of output values generated from the subset of the sets of input values by the first neuromorphic performance relative to corresponding sets of output values generated by the first non-neuromorphic performance to determine a first degree of accuracy of the neural network in performing the analytical function; and

in response to at least the first degree of accuracy exceeding a predetermined higher threshold, in response to receipt of a second request from a requesting device to perform the analytical function with a second data set comprising multiple sets of input values to generate multiple corresponding sets of output values, and in response to current availability of sufficient processing resources to enable a second neuromorphic performance of the analytical function with the second data set through use of the neural network, and with at least the predetermined level of throughput, performing operations comprising;

assigning, by the processor and as part of a second assignment of processing resources, at least a portion of currently available processing resources to the second neuromorphic performance;

after the assignment of processing resources to the second neuromorphic performance, analyzing, by the processor, a state of remaining instruction-based processing resources currently available; and

in response to current availability of sufficient remaining instruction-based processing resources to enable a second non-neuromorphic performance of the analytical function with at least a subset of the sets of input values of the second data set through execution of instructions implementing the analytical function by one or more processor cores, and at least partly in parallel with the second neuromorphic performance, performing operations comprising;

assigning, by the processor and as part of the second assignment, at least a portion of the remaining instruction-based processing resources to the second non-neuromorphic performance;

analyzing, by the processor, the sets of output values generated from the subset of the sets of input values by the second neuromorphic performance relative to corresponding sets of output values generated by the second non-neuromorphic performance to determine a second degree of accuracy of the neural network in performing the analytical function; and

in response to at least the second degree of accuracy exceeding the predetermined higher threshold, transmitting, from the processor, the multiple sets of output values generated from the second data set by the second neuromorphic performance to the requesting device.

22. The computer-implemented method of claim 21, wherein the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises an assignment of one or more processor cores of one or more processors comprising at least one of one or more central processing units (CPUs), or one or more graphics processing units (GPUs).

23. The computer-implemented method of claim 22, wherein:

the one or more processor cores of the one or more processors are processors distributed among multiple node devices of a grid of node devices; and

the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises an assignment of processing resources of the multiple node devices to the first non-neuromorphic performance.

24. The computer-implemented method of claim 22, wherein:

the assignment of at least a portion of remaining processing resources to the first neuromorphic performance comprises an assignment of at least a subset of one or more remaining processor cores, or an assignment of at least a portion of each of one or more neuromorphic devices;

each of the remaining processor cores are programmable to instantiate at least a portion of the neural network;

each of the neuromorphic devices comprises at least one of a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC); and

each of the neuromorphic devices comprises sets of circuits that each implement an artificial neuron able to be included in the neural network.

25. The computer-implemented method of claim 21, wherein:

the set of hyperparameters specifies a quantity of artificial neurons within the neural network; and

the analysis of the state of remaining processing resources after the assignment of instruction-based processing resources to the first non-neuromorphic performance comprises a determination of whether sufficient processing resources are available to instantiate the specified quantity of artificial neurons.

26. The computer-implemented method of claim 21, comprising, in response to at least the second degree of accuracy exceeding the predetermined higher threshold, in response to receipt of a third request from a requesting device to perform the analytical function with a third data set comprising multiple sets of input values to generate multiple corresponding sets of output values, and in response to current availability of sufficient processing resources to enable a third neuromorphic performance of the analytical function with the third data set through use of the neural network, and with at least the predetermined level of throughput, performing operations comprising:

assigning, by the processor and as part of a third assignment of processing resources, at least a portion of currently available processing resources to the third neuromorphic performance; and

refraining from assigning processing resources to a third non-neuromorphic performance of the analytical function with the third data set through execution of instructions implementing the analytical function by one or more processor cores.

27. The computer-implemented method of claim 21, comprising, in response to at least the first degree of accuracy falling below a predetermined lower threshold that is lower than the predetermined higher threshold, providing an indication that the performance of the analytical function by the neural network is deemed insufficiently accurate to be used.

28. The computer-implemented method of claim 27, comprising, in response to at least the first degree of accuracy falling below the predetermined higher threshold and above the predetermined lower threshold, performing operations comprising:

assigning, by the processor, at least a portion of currently available processing resources to instantiating the neural network in a training mode; and

using backpropagation to train the neural network with the multiple sets of input values of the first data set and the corresponding multiple sets of output values generated by the first non-neuromorphic performance.

29. The computer-implemented method of claim 27, wherein:

the first neuromorphic performance and the first non-neuromorphic performance occur at least partly within a testing federated area maintained by one or more storage devices to store the first data set, a first neuromorphic job flow definition, at least one task routine to perform at least one task defined by the first neuromorphic job flow definition, a first non-neuromorphic job flow definition, and at least one task routine to perform at least one task defined by the first non-neuromorphic job flow definition;

the neural network was trained at least partly within a training federated area maintained by the one or more storage devices to store at least one training data set, a training job flow definition, and at least one task routine to perform at least one task defined by the training job flow definition; and

in response to at least the first degree of accuracy falling below the predetermined lower threshold, transferring the indication that the performance of the analytical function by the neural network is deemed insufficiently accurate to be used from the testing federated area to the training federated area to trigger retraining of the neural network within another set of hyperparameters.

30. The computer-implemented method of claim 29, wherein:

the second neuromorphic performance and the second non-neuromorphic performance occur at least partly within a usage federated area maintained by the one or more storage devices to store the second data set, a second neuromorphic job flow definition, at least one task routine to perform at least one task defined by the second neuromorphic job flow definition, a second non-neuromorphic job flow definition, and at least one task routine to perform at least one task defined by the second non-neuromorphic job flow definition; and

in response to at least the second degree of accuracy exceeding the predetermined higher threshold, transferring neural network configuration data that defines the neural network from the testing federated area to the usage federated area to enable instantiation of the neural network to support the second neuromorphic performance.