Execution environment for data transformation applications
First Claim
1. A method for executing a dataflow application comprising:
- providing a dataflow application comprising a plurality of map components and data ports, wherein a number of the map components are linked between data ports, wherein the map components each comprise one or more map processes, and wherein some of the map components comprise composite components having a plurality of processes;
allocating a processing thread to each map process; and
executing each map process on a separate thread.
8 Assignments
0 Petitions
Accused Products
Abstract
The execution environment provides for scalability where components will execute in parallel and exploit various patterns of parallelism. Dataflow applications are represented by reusable dataflow graphs called map components, while the executable version is called a prepared map. Using runtime properties the prepared map is executed in parallel with a thread allocated to each map process. The execution environment not only monitors threads, detects and corrects deadlocks, logs and controls program exceptions, but also data input and output ports of the map components are processed in parallel to take advantage of data partitioning schemes. Port implementation supports multi-state null value tokens to more accurately report exceptions. Data tokens are batched to minimize synchronization and transportation overhead and thread contention.
67 Citations
9 Claims
-
1. A method for executing a dataflow application comprising:
-
providing a dataflow application comprising a plurality of map components and data ports, wherein a number of the map components are linked between data ports, wherein the map components each comprise one or more map processes, and wherein some of the map components comprise composite components having a plurality of processes; allocating a processing thread to each map process; and executing each map process on a separate thread. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
Specification