System for integrating task and data parallelism in dynamic applications
First Claim
1. A system for integrating task and data parallelism for a dynamic application including at least one task for real-time processing of a continuous input data stream to produce an output data stream, comprising:
- a splitter task partitioning the input data stream into a plurality of individually marked data chunks;
a work queue storing the data chunks;
a variable number of worker tasks processing subsets of the data chunks from the work queue, each worker task being an instance of the at least one task;
a controller queue storing a control item for each data chunk, the control item including task and data strategy provided by the splitter task; and
a joiner task combining the processed data chunks dependent on the control items on the controller queue to produce the output data stream.
6 Assignments
0 Petitions
Accused Products
Abstract
A system for integrating task and data parallelism in a dynamic application that includes at least one task for processing an input data stream to produce an output data stream replaces the at least one task with the following components. A splitter task for partitioning the input data stream into a plurality of data chunks. A plurality of worker tasks for processing subsets of the data chunks, each worker task being an instance of the at least one task, and a joiner task combining the processed data chunks to produce the output data stream.
-
Citations
14 Claims
-
1. A system for integrating task and data parallelism for a dynamic application including at least one task for real-time processing of a continuous input data stream to produce an output data stream, comprising:
-
a splitter task partitioning the input data stream into a plurality of individually marked data chunks;
a work queue storing the data chunks;
a variable number of worker tasks processing subsets of the data chunks from the work queue, each worker task being an instance of the at least one task;
a controller queue storing a control item for each data chunk, the control item including task and data strategy provided by the splitter task; and
a joiner task combining the processed data chunks dependent on the control items on the controller queue to produce the output data stream. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
a run-time system applying the action space to implement the task and data parallel strategy.
-
-
14. A method for integrating task and data parallelism for a dynamic application including at least one task for real-time processing of continuous input data stream to produce an output data stream, comprising the steps of:
-
splitting the input data stream into a plurality of individually marked data chunks by a splitter;
storing the data chunks in a work queue;
processing subsets of the data chunks by a variable number of worker tasks, each worker task being an instance of the at least one task;
storing a control item for each data chunk in a controller queue, the control item including task and a data strategy provided by the splitter task; and
combining the processed data chunks by a joiner task dependent on the control items on the controller queue to produce the output data stream.
-
Specification