Execution Environment for Data Transformation Applications

• US 20100262968A1
• Filed: 05/06/2010
• Published: 10/14/2010
• Est. Priority Date: 11/03/2003
• Status: Active Grant

• Current Assignee: Actian Corporation (HCL Technologies Limited)
• Original Assignee: Pervasive Software Incorporated (HCL Technologies Limited)
• Inventors: Schumacher, Larry Lee, Dingman, Paul C., Gonzales-Tuchmann, Agustin, Yogman, Laurence Tobin
• Granted Patent: US 8,528,000 B2
• Time in Patent Office: Days
• US Class Current: 718/102
• CPC Class Codes: G06F 8/10 Requirements analysis; Specification techniques
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06F 9/524 Deadlock detection or avoidance

Abstract

The execution environment provides for scalability where components will execute in parallel and exploit various patterns of parallelism. Dataflow applications are represented by reusable dataflow graphs called map components, while the executable version is called a prepared map. Using runtime properties the prepared map is executed in parallel with a thread allocated to each map process. The execution environment not only monitors threads, detects and corrects deadlocks, logs and controls program exceptions, but also data input and output ports of the map components are processed in parallel to take advantage of data partitioning schemes. Port implementation supports multi-state null value tokens to more accurately report exceptions. Data tokens are batched to minimize synchronization and transportation overhead and thread contention.

Claims

1. A method for executing a dataflow application comprising:

providing a dataflow application comprising a plurality of map components and data ports, a number of map components being linked between data ports and some map components comprising composite components having a plurality of processes;

allocating a processing thread to each map component; and

executing each map component on a separate thread.

2. The method of claim 1, including assigning runtime properties to the prepared map prior to execution.

3. The method of claim 1, certain map components including input ports and output ports and the executing step including reading data tokens from respective input ports and writing data tokens to respective output ports.

4. The method of claim 1, including choosing the port implementations to a specific data type prior to said executing step.

5. The method of claim 1, some of the ports being hierarchical comprising a plurality of hierarchical ports and the data being partitioned for parallel processing.

6. The method of claim 1, including transporting data tokens among map processes.

7. The method of claim 6, batching the data tokens to regulate the length of time a map process may execute without synchronization.

8. The method of claim 1, including ports associated with each map process for representing and transporting multi-state null value tokens.

9. The method of claim 8, the null value tokens including an error null.

10. A dataflow application having multiple processes executed on one or more multi-threaded central processing units (“

CPUs”

) connected by data input and output ports wherein at least some ports are implemented to indicate multiple state null value data tokens.

11. The application of claim 10, wherein the multiple state null value port operations support at least three state logic.

12. The application of claim 10, wherein the multiple state null values support at least a system defined error null token.

13. The application of claim 12, the system defined error null being passed to a map component for later determination of a system error condition.