Systems and methods for providing distributed, decentralized data storage and retrieval

• US 8,266,237 B2
• Filed: 04/20/2005
• Issued: 09/11/2012
• Est. Priority Date: 04/20/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Moore, George M., Cseri, Istvan
• Primary Examiner(s): Gorney, Boris
• Application Number:
  US11/110,128
• Publication Number:
  US 20060242155A1
• Time in Patent Office: 2,701 Days
• Field of Search: 709/213, 709/219, 709/225, 709/231, 709/232, 709/226, 711/112
• US Class Current: 709/213
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods for distributed, decentralized storage and retrieval of data in an extensible SOAP environment are disclosed. Such systems and methods decentralize not only the bandwidth required for data storage and retrieval, but also the computational requirements. Accordingly, such systems and methods alleviate the need for one node to do all the storage and retrieval processing, and no single node is required to send or receive all the data.

Claims

1. A system comprising:

a plurality of processor nodes, each one of the plurality of processing nodes comprising storage;

a distributed hash table comprising a plurality of local routing tables maintained in the plurality of processor nodes, each of the plurality of local routing tables containing hash values corresponding to node addresses of neighboring network nodes;

a storage service configured to receive a storage request message comprising a bit stream; and

a routing service configured to receive the bit stream from the storage service, divide the bit stream into a plurality of chunks, compute hash values for each of the plurality of chunks, each hash value based on a concatenation of a bit stream identifier and a user identifier, and transmit into a network a plurality of chunk store messages, the transmitting comprising consulting a local routing table of a first processor node to determine if an address of a recipient processor node corresponds to a neighboring network node of the first processor node, the recipient processor node comprising a chunk store service that divides a received chunk into a plurality of fragments, encodes a fragment into a check block, and transmits the check block to a fragment store service.

2. The system of claim 1, the recipient processor node further comprising:

a fragment store service that is configured to receive from the chunk store service at least one of the plurality of fragments and store the at least one of the plurality of fragments.

3. The system of claim 1, further comprising a retrieval system configured to retrieve a portion of the stored bit stream by using an offset value associated with a chunk.

4. The system of claim 1, wherein the chunk store service starts a timer when the check block is sent out for storage and, if the chunk store service does not receive a confirmation from the fragment store service that the check block has been stored before the timer expires, the chunk store service attempts to store the fragment elsewhere.

5. The system of claim 1, wherein each of the plurality of processor nodes has a time manager that provides alterable time relative to an absolute system time.

6. The system of claim 1, the routing service further configured to:

a) specifically direct a first chunk store message to the neighboring network node when a first hash value corresponding to the neighboring network node is present in the local routing table; and

b) transmit the first chunk store message into the network for hopping through an intermediary network node when the first hash value is not present in the local routing table.

7. The system of claim 1, the storage request message comprising a SOAP message.