Parallel partition-wise aggregation

• US 20060182046A1
• Filed: 02/16/2005
• Published: 08/17/2006
• Est. Priority Date: 02/16/2005
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Zait, Mohamed, Ghosh, Bhaskar, Dageville, Benoit, Cruanes, Thierry, Chen, Rushan
• Granted Patent: US 7,779,008 B2
• Time in Patent Office: Days
• US Class Current: 370/260
• CPC Class Codes: G06F 9/4494 data driven

Abstract

Techniques are provided for performing a parallel aggregation operation on data that resides in a container, such as a relational table. During generation of the execution plan for the operation, it is determined whether partition-wise aggregation should be performed, based on the grouping keys involved in the aggregation and the partition keys used to partition the container. If partition-wise aggregation is to be performed, then the assignments given to the slave processes that are assigned to scan a container are made on a partition-wise basis. The scan slaves themselves may perform full or partial aggregation (depending on whether they are the only scan slaves assigned to the partition). If the scan slaves perform no aggregation, or only partial aggregation, then the scan slaves redistribute the data items to aggregation slaves that are local to the scan slaves.

Claims

1. A method of performing, in parallel, an aggregation operation on data items that reside in one or more containers, wherein the aggregation operation aggregates the data items based on groups established by one or more grouping keys, the method comprising:

determining that at least one of the one or more grouping keys is related to at least one partition key used to establish partitions of the one or more containers;

based on the partitions, establishing scan assignments that dictate which portions of the one or more containers are to be scanned by each slave of a plurality of slaves;

each slave of the plurality of slaves scanning one or more portions of the one or more containers based on the scan assignments; and

based on the one or more grouping keys, aggregating the data items scanned by the plurality of slaves.

2. The method of claim 1 wherein the step of aggregating is performed by each slave of the plurality of slaves on the data items that are scanned by the slave.

3. The method of claim 2 wherein the step of establishing scan assignments includes, for each partition of the container, assigning a single slave of the plurality of slaves to scan all rows of the partition.

4. The method of claim 1 further comprising:

based at least in part on the number of partitions of the container to the number of slaves in the plurality of slaves, determining whether to perform partition-wise aggregation using the same slaves as are to be used to scan the container; and

wherein the step establishing assignments based on partitions is performed in response to determining that partition-wise aggregation should be performed.

5. The method of claim 2 wherein:

a particular slave receives an assignment that involves multiple partitions; and

the particular slave performs scans and aggregates data item from the multiple partitions one partition at a time, and sends the results of aggregating the data items of one partition before starting to scan any other partition.

6. The method of claim 2 wherein said scan assignments guaranty that if a slave scans any data item for a particular group, then the slave will scan all data items for the particular group.

7. The method of claim 1 wherein the step of determining that at least one of the one or more grouping keys is related to at least one partition key includes determining that the at least one grouping key is hierarchically related to the at least one partition key.

8. The method of claim 7 wherein:

the at least one partition key is below the at least one grouping key in the hierarchy; and

the step of establishing scan assignments is performed based on virtual partitions, wherein each virtual partition includes a plurality of partitions of the container.

9. The method of claim 7 wherein the at least one partition key is above the at least one grouping key in the hierarchy.

10. The method of claim 1 wherein the step of establishing scan assignments includes:

establishing a plurality of slave clusters, wherein the plurality of slave clusters includes a particular slave cluster that includes multiple scan slaves; and

establishing scan assignments for the slave clusters on a partition-by-partition basis, such that no two slave clusters are assigned to scan the same partition.

11. The method of claim 10 further comprising determining, based on the location of the slaves of said particular slave cluster, where to execute an aggregation slave that receives data items produced by the particular slave cluster.

12. The method of claim 11 wherein the aggregation slave and the slaves of the particular slave cluster are executed on the same node of a multiple node system.

13. The method of claim 10 each slave of the particular slave cluster performs partial aggregation prior to redistributing the data items to an aggregation slave.

14. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 1.

15. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 2.

16. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 3.

17. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 4.

18. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 5.

19. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 6.

20. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 7.

21. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 8.

22. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 9.

23. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 10.

24. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 11.

25. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 12.

26. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, causes the one or more processors to perform the method recited in claim 13.