RAM daemons

• US 10,437,808 B2
• Filed: 12/16/2011
• Issued: 10/08/2019
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Peters, Keith, Dole, Bryn Robert, Markson, Michael, Saliba, Robert Michael, Skrenta, Rich, Truel, Robert N., Lindahl, Gregory B.
• Primary Examiner(s): Dudek, Jr., Edward J
• Assistant Examiner(s): Russell, Andrew
• Application Number:
  US13/328,682
• Publication Number:
  US 20120210047A1
• Time in Patent Office: 2,853 Days
• Field of Search: 711148
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method of managing a database system using a swarm database system that communicates a request to read data to at least a subset of nodes. Checking the identifier by each respective node in the subset of nodes to determine if the requested read data is stored in the node. Providing the read data to the first node if the respective node in the subset includes read data.

Claims

1. A method comprising, at a swarm database system having a plurality of nodes each comprising one or more processors and memory, the memory of the plurality of nodes storing one or more programs to be executed by the one or more processors, where each of the plurality of nodes include a plurality of partitions mapped by a table stored within the node, the plurality of partitions creating a plurality of storage components within each of the nodes:

identifying data received or generated by one of the plurality of nodes;

replicating the data to a plurality of storage components, where each of the plurality of storage components is located within a unique node of the plurality of nodes;

receiving, at a first node of the plurality of nodes, a request for the data;

determining that a cache of the first node does not include the data;

identifying a second node of the plurality of nodes that has an identifier indicating that the data is stored in the second node, wherein the identifier includes at least one of a key-value pair and a row key;

requesting that the second node of the plurality of nodes service the request, in response to identifying that the second node has the identifier;

determining that the second node of the plurality of nodes has not responded to the request to service the request for the data within a predetermined time period;

identifying a third node of the plurality of nodes that has the identifier indicating that the data is stored in the third node;

requesting that the third node of the plurality of nodes service the request, in response to identifying that the third node has the read data identifier;

retrieving the response to the request; and

providing the data to the first node.

2. The method of claim 1, further comprising checking a cache storage associated with each respective node in the plurality of nodes before checking other storage allocations of the respective node.

3. The method of claim 2, further comprising managing one or more read operations by a reader daemon associated with each respective node in the plurality of nodes.

4. The method of claim 2, wherein the cache storage is one of a group consisting of RAM, a server storage, Flash storage on a solid-state drive and Flash on a peripheral component interconnect express card.

5. The method of claim 2, wherein the cache is managed by a cache daemon.

6. The method of claim 1, further comprising grouping the plurality of nodes into a plurality of groups based on a physical location of each of the plurality of nodes, where each of the plurality of storage components to which data is replicated is located within a unique group of the plurality of groups.

7. The method of claim 1, further comprising combining the plurality of groups into a plurality of zones based on one or more physical components that serve the plurality of zones, such that a subset of the plurality of zones that are served by a predetermined physical component are combined into a predetermined zone;

wherein each of the plurality of storage components to which data is replicated is located within a unique zone of the plurality of zones.

8. The method of claim 1, further comprising:

identifying a predetermined node of the plurality of nodes that includes a storage component to which the data is replicated;

determining that requests for the data are to be sent to the predetermined node first before other nodes of the plurality of nodes that include a storage component to which the data is replicated; and

storing the data in a random access memory (RAM) cache of the predetermined node, in response to the determining.

9. A system comprising:

a swarm database system having a plurality of nodes each comprising one or more processors and memory, the memory of the plurality of nodes storing one or more programs to be executed by the one or more processors, where each of the plurality of nodes include a plurality of partitions mapped by a table stored within the node, the plurality of partitions creating a plurality of storage components within each of the nodes, where the swarm database system is configured to;

identify data received or generated by one of the plurality of nodes;

replicate the data to a plurality of storage components, where each of the plurality of storage components is located within a unique node of the plurality of nodes;

receive, at a first node of the plurality of nodes, a request for the data;

determine that a cache of the first node does not include the data;

identify a second node of the plurality of nodes that has an identifier indicating that-the data is stored in the second node, wherein the identifier includes at least one of a key-value pair and a row key;

request that the second node of the plurality of nodes service the request, in response to identifying that the second node has the identifier;

determine that the second node of the plurality of nodes has not responded to the request to service the request for the data within a predetermined time period;

identify a third node of the plurality of nodes that has the identifier indicating that the data is stored in the third node;

request that the third node of the plurality of nodes service the request, in response to identifying that the third node has the read data identifier;

retrieve the response to the request; and

provide the data to the first node.

10. The system of claim 9, wherein the one or more nodes are further configured to,

check a cache storage associated with each respective node in the plurality of nodes before checking other storage allocations of the respective node.

11. The system of claim 10, further comprising, a reader daemon, associated with each respective node in the plurality of nodes, configured to manage one or more read operations.

12. The system of claim 10 wherein the cache storage is one of a group consisting of, RAM, a server storage, Flash storage on a solid-state drive and Flash on a peripheral component interconnect express card.

13. The system of claim 10 further comprising, a cache daemon, configured to manage the cache.

14. The system of claim 9 further comprising grouping the plurality of nodes into a plurality of groups based on a physical location of each of the plurality of nodes, where each of the plurality of storage components to which data is replicated is located within a unique group of the plurality of groups.

15. The system of claim 9, further comprising combining the plurality of groups into a plurality of zones based on one or more physical components that serve the plurality of zones, such that a subset of the plurality of zones that are served by a predetermined physical component are combined into a predetermined zone;

wherein each of the plurality of storage components to which data is replicated is located within a unique zone of the plurality of zones.

16. The system of claim 9, further comprising:

identifying a predetermined node of the plurality of nodes that includes a storage component to which the data is replicated;

determining that requests for the data are to be sent to the predetermined node first before other nodes of the plurality of nodes that include a storage component to which the data is replicated; and

storing the data in a random access memory (RAM) cache of the predetermined node, in response to the determining.