System and method for input data load adaptive parallel processing
DCFirst Claim
1. A system, executing on at least one of hardware logic and software logic executing on a plurality of processors, for hosting a plurality of application programs, the system comprising:
- a plurality of processing data input buffers, each input buffer of the plurality of processing data input buffers queuing data for a corresponding instance of one program of the plurality of application programs, wherein each program of the plurality of programs comprises a plurality of instances;
an array of cores of computing capacity;
a first subsystem configured to allocate the array of cores of computing capacity among the plurality of application programs, whereinallocating comprises allocating the array of cores of computing capacity based at least in part ona respective volume of processing data at each buffer of the plurality of processing data input buffers, anda respective processing quota of each application program of at least a portion of the plurality of application programs,andallocating comprises allocating more than one core of the array of cores of computing capacity to at least one of the plurality of application programs;
a second subsystem configured toassign, for each program of the plurality of application programs, each core allocated to the respective program to a different instance of the plurality of instances of the respective program, whereinthe assigning results in assignment of a plurality of selected instances of the plurality of instances of the plurality of application programs, the plurality of instances comprising one or more executable instances of each program of the plurality of application programs, whereina number of the plurality of selected instances is fewer than a maximum number of the plurality of instances, andthe plurality of selected instances is selected based at least in part on respective volumes of processing data available for each instance of the plurality of instances of the respective program at the portion of the plurality of data input buffers queuing data for the respective program, andaccording to the assigning, control connectivity between the plurality of processing data input buffers and the array of cores; and
a third subsystem configured, according to the controlling, to establish direct data access from each input buffer of at least a subset of the plurality of processing data input buffers to the respective core of the array of cores that is assigned to a given corresponding instance of the program for which the respective input buffer is queuing data;
wherein the array of cores is periodically allocated by the first subsystem and assigned by the second subsystem based at least in part on changes in respective volumes of processing data associated with each program of the plurality of application programs.
1 Assignment
Litigations
0 Petitions
Accused Products
Abstract
Systems and methods provide an extensible, multi-stage, realtime application program processing load adaptive, manycore data processing architecture shared dynamically among instances of parallelized and pipelined application software programs, according to processing load variations of said programs and their tasks and instances, as well as contractual policies. The invented techniques provide, at the same time, both application software development productivity, through presenting for software a simple, virtual static view of the actually dynamically allocated and assigned processing hardware resources, together with high program runtime performance, through scalable pipelined and parallelized program execution with minimized overhead, as well as high resource efficiency, through adaptively optimized processing resource allocation.
185 Citations
17 Claims
-
1. A system, executing on at least one of hardware logic and software logic executing on a plurality of processors, for hosting a plurality of application programs, the system comprising:
-
a plurality of processing data input buffers, each input buffer of the plurality of processing data input buffers queuing data for a corresponding instance of one program of the plurality of application programs, wherein each program of the plurality of programs comprises a plurality of instances; an array of cores of computing capacity; a first subsystem configured to allocate the array of cores of computing capacity among the plurality of application programs, wherein allocating comprises allocating the array of cores of computing capacity based at least in part on a respective volume of processing data at each buffer of the plurality of processing data input buffers, and a respective processing quota of each application program of at least a portion of the plurality of application programs, and allocating comprises allocating more than one core of the array of cores of computing capacity to at least one of the plurality of application programs; a second subsystem configured to assign, for each program of the plurality of application programs, each core allocated to the respective program to a different instance of the plurality of instances of the respective program, wherein the assigning results in assignment of a plurality of selected instances of the plurality of instances of the plurality of application programs, the plurality of instances comprising one or more executable instances of each program of the plurality of application programs, wherein a number of the plurality of selected instances is fewer than a maximum number of the plurality of instances, and the plurality of selected instances is selected based at least in part on respective volumes of processing data available for each instance of the plurality of instances of the respective program at the portion of the plurality of data input buffers queuing data for the respective program, and according to the assigning, control connectivity between the plurality of processing data input buffers and the array of cores; and a third subsystem configured, according to the controlling, to establish direct data access from each input buffer of at least a subset of the plurality of processing data input buffers to the respective core of the array of cores that is assigned to a given corresponding instance of the program for which the respective input buffer is queuing data; wherein the array of cores is periodically allocated by the first subsystem and assigned by the second subsystem based at least in part on changes in respective volumes of processing data associated with each program of the plurality of application programs. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 13)
-
-
9. A method, implemented on hardware logic and/or on software logic executing on processors, for executing a plurality of application programs, the method comprising:
-
on each of a plurality of data input buffers, queuing respective data for a corresponding program of the plurality of application programs; periodically allocating an array of units of computing capacity among the plurality of application programs, wherein allocating comprises, for each period of a plurality of allocation periods, identifying expressions of demand for the array of units of computing capacity by each program of the plurality of application programs, wherein the respective expression of demand for a given program of the plurality of application programs is based at least in part on quantifying the respective data queued for the given program in one or more input buffers of the plurality of data input buffers corresponding to the given program as a count of processing data units, and allocating the array of units of computing capacity based at least in part on the expressions of demand of the plurality of application programs; and for each period of a plurality of allocation periods, responsive to the allocating, assigning, for each program of the plurality of application programs, one or more instances of the respective program for execution on a respective one or more units of the array of units of computing capacity, wherein the one or more instances are assigned according to i) a respective number of the units of the array of units of computing capacity allocated to the respective program, and ii) execution priorities among instances of the respective program, and according to the assigning, establishing direct data access from each buffer of at least a subset of the plurality of data input buffers to the cores respective unit of the array of units of computing capacity that is assigned a given corresponding instance of the respective program for which the respective data buffer is queuing input data, wherein establishing direct data access comprises enabling the given corresponding instance to access and process the respective input data at the respective data buffer. - View Dependent Claims (10, 11, 12)
-
-
14. A system, executing on at least one of hardware logic and software logic executing on a plurality of processors, for hosting a plurality of application programs, the system comprising:
-
a plurality of data buffers, each buffer of the plurality of data buffers queuing input data for a respective program of the plurality of application programs, wherein the respective program is configured to access and process the input data during execution of the respective program; a plurality of processing units; a first subsystem configured to allocate the plurality of processing units among the plurality of application programs, wherein allocating comprises allocating the plurality of processing units based at least in part on a respective volume of input data at each buffer of the plurality of data buffers, and allocating, for at least some time periods, more than one processing unit of the plurality of processing units to at least one of the plurality of application programs; a second subsystem configured to, for each program of the plurality of application programs, select one or more instances of a plurality of instances of the respective program, wherein selecting is based at least in part on respective volumes of input data available for each instance of the plurality of instances of the respective program, and a different instance of the respective program is selected for each processing unit allocated to the respective program; a third subsystem configured to assign, for each program of the plurality of application programs, each processing unit allocated to the respective program to a different instance of the one or more instances of the respective program in accordance with the selecting; and a fourth subsystem configured, according to the assigning, to establish direct data access from each buffer of at least a subset of the plurality of data buffers to the respective processing unit of the plurality of processing units that is assigned a given corresponding instance of the respective program for which the respective data buffer is queuing input data, wherein establishing direct data access comprises enabling the given corresponding instance to access and process the respective input data at the respective data buffer; wherein the plurality of processing units is periodically allocated by the first subsystem, selected by the second subsystem, and assigned by the third subsystem based at least in part on changes in respective volumes of input data associated with each program of the plurality of application programs. - View Dependent Claims (15, 16, 17)
-
Specification