Parallel processing database tree structure

• US 9,594,803 B2
• Filed: 09/10/2015
• Issued: 03/14/2017
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Welton, Caleb E., Harada, Hitoshi, Cohen, Jeffrey Ira, Chang, Lei, Reddy, Radhika, Ma, Tao, Wang, Zhanwei
• Primary Examiner(s): NGUYEN, KIM T
• Application Number:
  US14/850,814
• Publication Number:
  US 20150379078A1
• Time in Patent Office: 551 Days
• Field of Search: 707/718, 707/797
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1858 Parallel file systems, i.e. file systems supporting multiple processors
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Database system comprising nodes configured in a tree structure is disclosed. The system includes a shared metadata store on the root node. Child nodes may request metadata from their ancestors. Parents will forward the request upward until the metadata is found or the root node is reached.

Claims

1. A method for executing queries in a parallel processing database system, comprising:

receiving a query at a master node, the master node comprising a database catalog including metadata;

deriving a plurality of query plans based at least in part on the query;

transmitting, to a plurality of worker nodes, corresponding ones of the plurality of query plans and associated query metadata, wherein the query metadata includes information used in connection with executing the query plans; and

in the event that additional metadata is needed in connection with executing one of the plurality of query plans, receiving a request for additional metadata corresponding to the query plan for which additional metadata is needed, wherein a parent node is queried for the additional metadata, the parent node comprising a metadata cache, and in the event that the parent node does not comprise the additional metadata, successive ancestor nodes are queried until the additional metadata is found or until all successive ancestor nodes have been queried.

2. The method of claim 1, wherein the metadata included in the database catalog defines database objects.

3. The method of claim 1, wherein the request for the additional metadata is sent to the master node by an ancestor node of the worker node to which the corresponding query plan was transmitted.

4. The method of claim 3, wherein querying the successive ancestor nodes comprises iteratively communicating the request for the additional metadata to successive ancestor nodes of the worker node to which the corresponding query plan was transmitted.

5. The method of claim 3, wherein the request for the additional metadata is iteratively communicated to successive ancestor nodes of the worker node to which the corresponding query plan was transmitted until the master node is reached.

6. The method of claim 1, wherein the master node is a root node in tree structure.

7. The method of claim 1, further comprising checking the metadata cache for the additional metadata.

8. The method of claim 1, wherein the query plans include a set of instructions.

9. The method of claim 1, wherein the query metadata includes one or more of a database table definition, a database function, a database view, and a database index.

10. The method of claim 1, wherein the request for additional metadata is sent in connection with a query for the additional metadata, and in the event that the query is received at the master node, establishing a catalog server session corresponding to the query, and retrieving additional query metadata from a metadata catalog stored on a database catalog server.

11. The method of claim 1, wherein the plurality of worker nodes are connected to the master node, and wherein the plurality of worker nodes respectively comprise a database configured to execute a database query corresponding to the respective query plan and associated query metadata that is received from the master node.

12. A computer program product for executing queries in a parallel processing database system, comprising a non-transitory computer readable medium having program instructions embodied therein for:

receiving a query at a master node, the master node comprising a database catalog including metadata;

deriving a plurality of query plans based at least in part on the query;

transmitting, to a plurality of worker nodes, corresponding ones of the plurality of query plans and associated query metadata, wherein the query metadata includes information used in connection with executing the query plans; and

in the event that additional metadata is needed in connection with executing one of the plurality of query plans, receiving a request for additional metadata corresponding to the query plan for which additional metadata is needed, wherein a parent node is queried for the additional metadata, the parent node comprising a metadata cache, and in the event that the parent node does not comprise the additional metadata, successive ancestor nodes are queried until the additional metadata is found or until all successive ancestor nodes have been queried.

13. The computer program product of claim 12, wherein the metadata included in the database catalog defines database objects.

14. The computer program product of claim 12, wherein the request for the additional metadata is sent to the master node by an ancestor node of the worker node to which the corresponding query plan was transmitted.

15. The computer program product of claim 14, wherein querying the successive ancestor nodes comprises iteratively communicating the request for the additional metadata to successive ancestor nodes of the worker node to which the corresponding query plan was transmitted.

16. The computer program product of claim 14, wherein the request for the additional metadata is iteratively communicated to successive ancestor nodes of the worker node to which the corresponding query plan was transmitted until the master node is reached.

17. The computer program product of claim 12, wherein the master node is a root node in tree structure.

18. The computer program product of claim 12, wherein the parent node is queried, by the worker node, for the additional metadata.

19. The computer program product of claim 18, the non-transitory computer readable medium having program instructions embodied therein for checking the metadata cache for the additional metadata.

20. A system for executing queries in a parallel processing database, comprising a non-transitory computer readable medium and a processor configured to:

receive a query at a master node, the master node comprising a database catalog including metadata;

derive a plurality of query plans based at least in part on the query;

transmit, to a plurality of worker nodes, corresponding ones of the plurality of query plans and associated query metadata, wherein the query metadata includes information used in connection with executing the query plans; and

in the event that additional metadata is needed in connection with executing one of the plurality of query plans, receive a request for additional metadata corresponding to the query plan for which additional metadata is needed, wherein a parent node is queried for the additional metadata, the parent node comprising a metadata cache, and in the event that the parent node does not comprise the additional metadata, successive ancestor nodes are queried until the additional metadata is found or until all successive ancestor nodes have been queried.

21. The system of claim 20, wherein the metadata included in the database catalog defines database objects.

22. The system of claim 20, wherein the request for the additional metadata is sent to the master node by an ancestor node of the worker node to which the corresponding query plan was transmitted.

23. The system of claim 22, wherein querying the successive ancestor nodes comprises iteratively communicating the request for the additional metadata to successive ancestor nodes of the worker node to which the corresponding query plan was transmitted.

24. The system of claim 22, wherein the request for the additional metadata is iteratively communicated to successive ancestor nodes of the worker node to which the corresponding query plan was transmitted until the master node is reached.

25. The system of claim 20, wherein successive ancestor nodes are queried until the additional metadata is found.

26. The system of claim 20, wherein the master node is a root node in tree structure.

27. The system of 20, wherein the parent node checks the metadata cache for the additional metadata.