STRUCTURED PARALLEL DATA INTENSIVE COMPUTING

• US 20100191716A1
• Filed: 01/25/2009
• Published: 07/29/2010
• Est. Priority Date: 01/25/2009
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Hsu, Meichun, Chen, Qiming
• Granted Patent: US 8,046,373 B2
• Time in Patent Office: Days
• US Class Current: 707/713
• CPC Class Codes: G06F 16/24532 of parallel queries

Abstract

Methods, systems, and computer program products are provided for performing structured parallel data intensive computations. A SQL-FCF framework that defines a SQL construct based interface for performing the computations is provided. The computations within an application are coded as table valued functions (TVFs). Data for the application is partitioned into data partitions to co-locate and load balance the data partitions and corresponding computations. The TVFs are invoked and applied to corresponding ones of the data partitions in a structured order defined by function control forms (FCF).

Claims

1. A computer system comprising:

a computer processor; and

logic instructions on tangible computer readable media and executable by the computer processor to cause the computer processor to;

wrap an application as table valued functions (TVFs), the TVFs being configured to perform computations;

partition data for the application into data partitions;

apply the TVFs to corresponding ones of the data partitions in a structured order.

2. The system of claim 1, the logic instructions further causing the computer processor to:

configure function control forms (FCF) to define the structured order, the FCF controlling application of the TVFs to data objects included in the data partitions.

3. The system of claim 2, the logic instructions further causing the computer processor to:

form new ones of the FCF from existing ones of the FCF using SQL constructs, thereby making the FCF extensible.

4. The system of claim 2, the logic instructions further causing the computer processor to:

construct the FCF as a meta operator for applying a function f to perform at least one of the computations, f being an operand of the meta operator, the meta operator using the function f as a parameter for applying f.

5. The system of claim 2, the logic instructions further causing the computer processor to:

define the FCF to include a primitive CONNECT APPLY to apply a function of the TVFs in the structured order and a primitive CROSS APPLY to apply another function of the TVFs without the structured order.

6. The system of claim 1, the logic instructions further causing the computer processor to:

allocate the data partitions and corresponding ones of the TVFs to co-locate computation and data.

8. The system of claim 1, the logic instructions further causing the computer processor to:

configure a data dependency graph to define the structured order.

9. The system of claim 1, the logic instructions further causing the computer processor to:

apply at least one function configured in the TVFs to data objects included in the corresponding ones of the data partitions.

10. The system of claim 1, the logic instructions further causing the computer processor to:

update the data in response to applying the TVFs to corresponding ones of the data partitions.

11. The system of claim 1, the logic instructions further causing the computer processor to:

configure an input of one of the TVFs as a row representing one of the data partitions and an output of the one of the TVFs as a row set.

12. The system of claim 1, the logic instructions further causing the computer processor to:

perform the computations at a database layer.

13. The system of claim 1, the logic instructions further causing the computer processor to:

configure the structured order to be a reversed tree.

7. The system of claim 7, the logic instructions further causing the computer processor to:

parallelize the data partitions and the corresponding ones of the TVFs across server nodes.

14. A method comprising:

providing a SQL-FCF framework, the SQL-FCF framework defining a SQL construct based interface for performing structured parallel data intensive computations;

configuring table valued functions (TVFs) provided by the SQL-FCF framework, the TVFs being configured to perform the computations;

parallelizing the computations;

configuring function control forms (FCF) to define a structured order for executing the TVFs, the FCF controlling application of the TVFs to data objects, and

invoking the TVFs.

15. A system comprising:

a server node operable to host a database management system (DBMS), the server node comprising a virtual software layer (VSL) coupled to the DBMS, the VSL comprising a virtual data management (VDM) for accessing data from other databases and a virtual task management (VTM) for task partitioning and scheduling;

data partitions to store partitioned original data in the DBMS; and

table valued functions (TVFs) hosted by the server node to perform computations, the TVFs being applied to corresponding ones of the data partitions in a structured order, the TVFs being invoked by the VTM.