Parallel processing database tree structure

• US 9,171,042 B1
• Filed: 03/15/2013
• Issued: 10/27/2015
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Welton, Caleb, Harada, Hitoshi, Chang, Lei, Cohen, Jeffrey, Reddy, Radhika, Ma, Tao, Wang, Zhanwei
• Primary Examiner(s): NGUYEN, KIM T
• Application Number:
  US13/839,171
• Time in Patent Office: 956 Days
• Field of Search: 707/718, 707/797
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1858 Parallel file systems, i.e. file systems supporting multiple processors
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Database system comprising nodes configured in a tree structure is disclosed. The system includes a shared metadata store on the root node. Child nodes may request metadata from their ancestors. Parents will forward the request upward until the metadata is found or the root node is reached.

Claims

1. A method for executing queries in a parallel processing database system, comprising:

receiving a query at a master node, the master node comprising a database catalog including metadata defining database objects;

deriving a plurality of query plans from the query;

transmitting the query plans and query metadata to a plurality of worker nodes, wherein the query metadata includes metadata for executing the query plans;

determining additional metadata is needed for executing one of the plurality of query plans; and

querying a parent node for the additional metadata; and

querying successive ancestor nodes until the master node is reached.

2. The method of claim 1, further comprising querying successive ancestor nodes until the additional metadata is found.

3. The method of claim 1, wherein the master node is a root node in tree structure.

4. The method of claim 1, wherein the parent node comprises a metadata cache.

5. The method of claim 4, further comprising checking the cache for the additional metadata.

6. The method of claim 1, wherein the query plans include a set of instructions.

7. The method of claim 1, wherein the query metadata includes one or more of a database table definition, a database function, a database view, and a database index.

8. The method of claim 1, further comprising:

in the event that the query is received at the master node, establishing a catalog server session corresponding to the query, and retrieving the query metadata from a metadata catalog stored on a database catalog server.

9. The method of claim 1, wherein each of the plurality of worker nodes is operatively connected to the master node and comprises a database configured to execute a database query corresponding to the respective query plan and query metadata that is received from the master node.

10. A computer program product for executing queries in a parallel processing database system, comprising a non-transitory computer readable medium having program instructions embodied therein for:

receiving a query at a master node, the master node comprising a database catalog including metadata defining database objects;

deriving a plurality of query plans from the query;

transmitting the query plans and query metadata to a plurality of worker nodes, wherein the query metadata includes metadata for executing the query plans;

determining additional metadata is needed for executing one of the plurality of query plans;

querying a parent node for the additional metadata; and

querying successive ancestor nodes until the master node is reached.

11. The computer program product of claim 10, the non-transitory computer readable medium having program instructions embodied therein for querying successive ancestor nodes until the additional metadata is found.

12. The computer program product of claim 10, wherein the master node is a root node in tree structure.

13. The computer program product of claim 10, wherein the parent node comprises a metadata cache.

14. The computer program product of claim 13, the non-transitory computer readable medium having program instructions embodied therein for checking the cache for the additional metadata.

15. A system for executing queries in a parallel processing database, comprising a non-transitory computer readable medium and a processor configured to:

receive a query at a master node, the master node comprising a database catalog including metadata defining database objects;

derive a plurality of query plans from the query;

transmit the query plans and query metadata to a plurality of worker nodes, wherein the query metadata includes metadata for executing the query plans;

determine additional metadata is needed for executing one of the plurality of query plans; and

query a parent node for the additional metadata; and

query successive ancestor nodes until the master node is reached.

16. The system of claim 15, wherein the processor is further configured to query successive ancestor nodes until the additional metadata is found.

17. The system of claim 15, wherein the master node is a root node in tree structure.

18. The system of claim 15, wherein the parent node comprises a metadata cache.

19. The system of claim 18, wherein the processor is further configured to check the cache for the additional metadata.