Data Management and Indexing Across a Distributed Database

• US 20150169650A1
• Filed: 02/23/2015
• Published: 06/18/2015
• Est. Priority Date: 06/06/2012
• Status: Active Grant

• Current Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Original Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Inventors: Gajic, Natasha
• Granted Patent: US 9,727,590 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

In one embodiment, a distributed database system supporting flexible configuration of data clusters is disclosed. The system includes a cluster manager, an index, and a dataset distributed over one or more database clusters. Where the nodes of the clusters may report ownership of a particular range, the index contains an alternate range. The cluster manager receives requests to access a range of data within database and queries the index to determine the appropriate nodes and/or clusters with which to connect. The cluster manager then directs the requestor to connect to the specified nodes and/or clusters.

Claims

1. A method of establishing a connection to a database node, the method comprising:

receiving a connection request at a cluster manager from a requesting process;

querying an index of a database system to receive a cluster and node configuration for the database system;

determining a cluster/node topography for the database system based on the received cluster and node configuration;

assembling a connection list based on the cluster/node topography of the database system; and

connecting the requesting process to one or more of a specified database node and a specified database cluster based on the connection list.

2. The method of claim 1, wherein the querying of the index includes:

querying a first hierarchical level of the index for cluster and node configuration to receive a first index response;

determining whether the first index response includes a relevant pointer to a second hierarchical level of the index; and

querying the second hierarchical level when it is determined that the first index response includes the relevant pointer to the second hierarchical level.

3. The method of claim 2, wherein the querying of the first hierarchical index includes performing a query command on a first index node in a first index cluster;

and wherein the querying of the second hierarchical index includes performing a query command on a second index node in a second index cluster.

4. The method of claim 3, wherein the first index cluster is different from the second index cluster.

5. The method of claim 1, wherein the connection request includes an attribute of an underlying data access command, wherein the querying of the index includes issuing a query command based on the attribute;

and wherein the assembling of the connection list includes analyzing the cluster/node topography based on the attribute.

6. The method of claim 1, wherein the assembling of the connection list includes analyzing the cluster/node topography based on at least one of an attribute of an underlying data access command, an attribute of a node, and an attribute of a dataset of a node.

7. The method of claim 1,

wherein the connection list includes a reference to the specified database cluster;

and

the method further comprising connecting the requesting process to a database node within the specified database cluster according to a procedure established in a database protocol of the specified database cluster.

8. The method of claim 1, further comprising updating the index based on data access of the requesting process to the one or more of specified data node and specified database cluster.

9. The method of claim 1, further comprising utilizing a proxy to relay commands between the requesting process and the one or more of the specified database node and the specified database cluster.

11. The method of claim 8 further comprising moving data from an existing node to the new node, based on a key range of the new node.

12. The method of claim 8 further comprising:

determining whether an existing node benefits from being relocated from an existing cluster to the new cluster;

relocating the existing node to the new cluster when it is determined that the existing node benefits from being relocated;

repartitioning, into one or more repartioned key ranges, a key range of the existing cluster and the new cluster based on the relocating of the existing node to the new cluster; and

updating the index with the repartitioned key ranges of the existing cluster and the new cluster.

13. The method of claim 8, wherein determining whether the distributed database system benefits from creating a new cluster includes determining whether one or more nodes in the distributed database system have resources available for reallocation.

14. The method of claim 8, wherein determining whether the distributed database system benefits from creating a new cluster includes determining whether one or more clusters in the distributed database system have reached a maximum number of nodes.

10. A method of adding a node to a distributed database system, the method comprising:

receiving, from a computing device in a distributed database system, a trigger to add a new node to a distributed database system;

determining whether the distributed database system benefits from creating a new cluster;

creating index entries for the new node and the new cluster when it is determined that the distributed database system benefits from the new cluster;

establishing the new node and the new cluster as part of the distributed database system, the new node being established within the new cluster;

repartitioning keys within an index; and

updating the index with the created index entries and the repartitioned keys.

15. A database system, comprising:

a plurality of database nodes;

a cluster index;

a cluster manager, implemented on one or more processors, configured to;

receive a connection request from a requesting process;

query the cluster index of the database system to receive a cluster and node configuration for the database system;

determine a cluster/node topography for the database system based on the received cluster and node configuration;

assemble a connection list based on the cluster/node topography of the database system; and

connect the requesting process to one or more of a specified database node and a specified database cluster based on the connection list.

16. The system of claim 15, wherein the cluster manager is further configured to:

query a first hierarchical level of the cluster index for cluster and node configuration to receive a first index response;

determine whether the first index response includes a relevant pointer to a second hierarchical level of the cluster index; and

query the second hierarchical level when it is determined that the first index response includes the relevant pointer to the second hierarchical level.

17. The system of claim 16, wherein the querying of the first hierarchical index includes performing a query command on a first index node in a first index cluster;

and wherein the querying of the second hierarchical index includes performing a query command on a second index node in a second index cluster.

18. The system of claim 17, wherein the first index cluster is different from the second index cluster.

19. The system of claim 15, wherein the connection request includes an attribute of an underlying data access command, wherein the querying of the index includes issuing a query command based on the attribute;

and wherein the assembling of the connection list includes analyzing the cluster/node topography based on the attribute.

20. The system of claim 15,

wherein the connection list includes a reference to the specified database cluster;

and

wherein the cluster manager is further configured to connect the requesting process to a database node within the specified database cluster according to a procedure established in a database protocol of the specified database cluster.