Data management and indexing across a distributed database

• US 9,727,590 B2
• Filed: 02/23/2015
• Issued: 08/08/2017
• Est. Priority Date: 06/06/2012
• Status: Active Grant

• Current Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Original Assignee: Rackspace US Incorporated (Apollo Global Management, Inc.)
• Inventors: Gajic, Natasha
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US14/629,295
• Publication Number:
  US 20150169650A1
• Time in Patent Office: 897 Days
• Field of Search: 707770
• CPC Class Codes: G06F 16/20 of structured data, e.g. relational data
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/285 Clustering or classification

Abstract

In one embodiment, a distributed database system supporting flexible configuration of data clusters is disclosed. The system includes a cluster manager, an index, and a dataset distributed over one or more database clusters. Where the nodes of the clusters may report ownership of a particular range, the index contains an alternate range. The cluster manager receives requests to access a range of data within database and queries the index to determine the appropriate nodes and/or clusters with which to connect. The cluster manager then directs the requestor to connect to the specified nodes and/or clusters.

Claims

1. A method of establishing a connection to a database node, the method comprising:

receiving a connection request at a cluster manager from a requesting process;

querying a first hierarchical level of an index of a database system to receive a cluster and node configuration for the database system, wherein the query results in receipt of a first index response, and wherein querying the first hierarchical level of the index of the database system includes performing a query command on a first index node in a first index cluster;

determining whether the first index response includes a relevant pointer to a second hierarchical level of the index;

querying the second hierarchical level of the index to receive the cluster and node configuration for the database system when it is determined that the first index response includes the relevant pointer to the second hierarchical level, wherein querying the second hierarchical level of the index includes performing a query command on a second index node in a second index cluster, and wherein the first index cluster is different from the second index cluster;

determining a cluster/node topography for the database system based on the received cluster and node configuration;

assembling a connection list based on the cluster/node topography of the database system; and

connecting the requesting process to one or more of a specified database node and a specified database cluster based on the connection list.

2. The method of claim 1, wherein the connection request includes an attribute of an underlying data access command, wherein the querying of the index includes issuing a query command based on the attribute;

and wherein the assembling of the connection list includes analyzing the cluster/node topography based on the attribute.

3. The method of claim 1, wherein the assembling of the connection list includes analyzing the cluster/node topography based on at least one of an attribute of an underlying data access command, an attribute of a node, and an attribute of a dataset of a node.

4. The method of claim 1,

wherein the connection list includes a reference to the specified database cluster;

and

the method further comprising connecting the requesting process to a database node within the specified database cluster according to a procedure established in a database protocol of the specified database cluster.

5. The method of claim 1, further comprising updating the index based on data access of the requesting process to the one or more of specified data node and specified database cluster.

6. The method of claim 1, further comprising utilizing a proxy to relay commands between the requesting process and the one or more of the specified database node and the specified database cluster.

7. A method of adding a node to a distributed database system, the method comprising:

receiving, from a computing device in a distributed database system, a trigger to add a new node to a distributed database system, wherein the trigger is received based at least in part on a determination by the computing device that the new node is needed;

in response to the received trigger, determining whether the distributed database system benefits from creating a new cluster for the new node;

creating index entries for the new node and the new cluster when it is determined that the distributed database system benefits from the new cluster;

establishing the new node and the new cluster as part of the distributed database system, the new node being established within the new cluster;

repartitioning keys within an index;

moving data from an existing node to the new node established within the new cluster, based on a key range of the new node; and

updating the index with the created index entries and the repartitioned keys.

8. The method of claim 7 further comprising:

determining whether an existing node benefits from being relocated from an existing cluster to the new cluster;

relocating the existing node to the new cluster when it is determined that the existing node benefits from being relocated;

repartitioning, into one or more repartitioned key ranges, a key range of the existing cluster and the new cluster based on the relocating of the existing node to the new cluster; and

updating the index with the repartitioned key ranges of the existing cluster and the new cluster.

9. The method of claim 7, wherein determining whether the distributed database system benefits from creating a new cluster includes determining whether one or more nodes in the distributed database system have resources available for reallocation.

10. The method of claim 7, wherein determining whether the distributed database system benefits from creating a new cluster includes determining whether one or more clusters in the distributed database system have reached a maximum number of nodes.

11. A database system, comprising:

one or more processors;

a plurality of database nodes;

a cluster index;

a cluster manager, implemented on the one or more processors, configured to;

receive a connection request from a requesting process;

query a first hierarchical level of the cluster index of the database system to receive a cluster and node configuration for the database system, wherein the query results in receipt of a first index response, and wherein the querying of the first hierarchical level of the cluster index of the database system includes performing a query command on a first index node in a first index cluster;

determine whether the first index response includes a relevant pointer to a second hierarchical level of the cluster index;

query the second hierarchical level of the cluster index to receive the cluster and node configuration for the database system when it is determined that the first index response includes the relevant pointer to the second hierarchical level, wherein querying the second hierarchical level of the index includes performing a query command on a second index node in a second index cluster, and wherein the first index cluster is different from the second index cluster;

determine a cluster/node topography for the database system based on the received cluster and node configuration;

assemble a connection list based on the cluster/node topography of the database system; and

connect the requesting process to one or more of a specified database node and a specified database cluster based on the connection list.

12. The database system of claim 11, wherein the connection request includes an attribute of an underlying data access command, wherein the querying of the index includes issuing a query command based on the attribute;

and wherein the assembling of the connection list includes analyzing the cluster/node topography based on the attribute.

13. The database system of claim 11,

wherein the connection list includes a reference to the specified database cluster;

and

wherein the cluster manager is further configured to connect the requesting process to a database node within the specified database cluster according to a procedure established in a database protocol of the specified database cluster.