PROFILING IN A MASSIVE PARALLEL PROCESSING ENVIRONMENT

• US 20100250563A1
• Filed: 03/27/2009
• Published: 09/30/2010
• Est. Priority Date: 03/27/2009
• Status: Active Grant

• Current Assignee: Business Objects Incorporated (SAP SE), SAP AG (SAP SE)
• Original Assignee: SAP AG (SAP SE)
• Inventors: Ganti, Sridhar, Cao, Wu, Gadhiraju, Balaji
• Granted Patent: US 9,251,212 B2
• Time in Patent Office: Days
• US Class Current: 707/756
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24545 Selectivity estimation or determination

Abstract

A computer-implemented method of profiling a data set in a parallel processing environment includes vertically partitioning an initial data set. One or more attribute subsets are then profiled. A list of subjects is generated each corresponding to a specific attribute value identified in the profiling. Values of multiple attributes are extracted for each identified subject, and the sample results are assembled and merged.

Claims

1. A computer-implemented method of profiling a data set in a parallel processing environment, comprising:

partitioning an initial data set vertically according to multiple attribute subsets;

profiling one or more of the attribute subsets;

generating a list of subjects or otherwise horizontal component values corresponding to a specific attribute value identified in the profiling;

extracting values of multiple attributes for each said identified subject or otherwise horizontal component values;

assembling sample results of said identified subjects or otherwise horizontal component values;

merging the sample results to form a profiled subset of the initial data set; and

transmitting, displaying or storing the profiled subset of the initial data set, a further processed version, or combinations thereof.

2. The method of claim 1, wherein the initial data set comprises a table having m rows and n columns.

3. The method of claim 2, wherein each column corresponds to a specific attribute and each row corresponds to a specific subject or otherwise horizontal component values.

4. The method of claim 1, further comprising assigning a different set of one or more machines to each of the attribute subsets.

5. The method of claim 4, wherein the profiling comprises concurrently profiling the attribute subsets assigned to the different sets of one or more machines.

6. The method of claim 1, wherein the merging comprises removing duplicate subject sets of sample results.

7. A computer network including a parallel processing environment, comprising:

a data source;

multiple projection computers; and

one or more client computers connected to the multiple projection computers and having computer-readable code embedded therein for programming the projection computers to perform a method of profiling a data set in the parallel processing environment, wherein the method comprises;

partitioning an initial data set vertically or otherwise according to multiple attribute subsets;

profiling one or more of the attribute subsets;

generating a list of subjects or otherwise horizontal component values corresponding to a specific attribute value identified in the profiling;

extracting values of multiple attributes for each said identified subject or otherwise horizontal component values;

assembling sample results of said identified subjects or otherwise horizontal component values; and

merging the sample results to form a profiled subset of the initial data set.

8. The computer network of claim 7, wherein the initial data set comprises a table having m rows and n columns.

9. The computer network of claim 8, wherein each column corresponds to a specific attribute and each row corresponds to a specific subject or otherwise horizontal component values.

10. The computer network of claim 7, wherein the method further comprises assigning a different set of one or more machines to each of the multiple attribute data sets.

11. The computer network of claim 10, wherein the profiling comprises concurrently profiling the multiple data sets assigned to the different sets of one or more machines.

12. The computer network of claim 7, wherein the merging comprises removing duplicate subject sets of sample results.

13. The computer network of claim 7, wherein the merging comprises sorting the sample results.

14. One or more processor-readable media having embedded therein processor-readable code to program one or more processors to perform a method of profiling a data set in a parallel processing environment, wherein the method comprises:

partitioning an initial data set vertically or otherwise according to multiple attribute subsets;

profiling one or more of the attribute subsets;

generating a list of subjects or otherwise horizontal component values corresponding to a specific attribute value identified in the profiling;

extracting values of multiple attributes for each said identified subject or otherwise horizontal component values;

assembling sample results of said identified subjects or otherwise horizontal component values; and

merging the sample results to form a profiled subset of the initial data set.

15. The one or more processor-readable media of claim 14, wherein the initial data set comprises a table having m rows and n columns.

16. The one or more processor-readable media of claim 15, wherein each column corresponds to a specific attribute and each row corresponds to a specific subject or otherwise horizontal component values.

17. The one or more processor-readable media of claim 14, wherein the method further comprises assigning a different set of one or more machines to each of the attribute subsets.

18. The one or more processor-readable media of claim 17, wherein the profiling comprises concurrently profiling the attribute subsets assigned to the different sets of one or more machines.

19. The one or more processor-readable media of claim 14, wherein the merging comprises removing duplicate subject sets of sample results.

20. The one or more processor-readable media of claim 14, wherein the merging comprises sorting the sample results.