Systems and methods for providing distributed, decentralized data storage and retrieval

• US 20060242155A1
• Filed: 04/20/2005
• Published: 10/26/2006
• Est. Priority Date: 04/20/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cseri, Istvan, Moore, George M.
• Granted Patent: US 8,266,237 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Systems and methods for distributed, decentralized storage and retrieval of data in an extensible SOAP environment are disclosed. Such systems and methods decentralize not only the bandwidth required for data storage and retrieval, but also the computational requirements. Accordingly, such systems and methods alleviate the need for one node to do all the storage and retrieval processing, and no single node is required to send or receive all the data.

Claims

1. A network-based system for decentralized, distributed storage of data, the system comprising:

a local cache and transport layer that receives a request to store a bit stream, and routes onto the network a stream store message containing the bit stream; and

a stream store layer that receives the stream store message, divides the bit stream into a plurality of chunks, and sends onto the network a plurality of chunk store messages, each said chunk store message containing a respective one of the plurality of chunks;

wherein each of the chunks is stored on a respective one of a first plurality of network nodes.

2. The system of claim 1, further comprising:

a chunk store layer that receives the chunk store messages, divides each chunk into a respective plurality of fragments, encodes the fragments into check blocks and sends onto the network a plurality of fragment store messages, each said fragment store message containing a respective one of the plurality of fragments,

wherein each of the fragments is stored on a respective one of a second plurality of network nodes.

3. The system of claim 2, further comprising:

a fragment store layer that receives the fragment store messages, and causes each of the check blocks to be stored across a third plurality of network nodes.

4. The system of claim 3, wherein the bit stream is stored on a network node associated with the stream store layer.

5. The system of claim 2, wherein each fragment is encoded into a sufficient number of check blocks to ensure retrieval of the fragment to within a predetermined degree of confidence.

6. The system of claim 1, wherein the local cache and transport layer computes a hash value based on the bit stream, and wherein the hash value is used to determine an address associated with a node to which the stream store message is intended to be sent.

7. The system of claim 1, wherein the local cache and transport layer that receives a confirmation from the stream store layer that the bit stream has been stored.

8. The system of claim 7, wherein the local cache and transport layer starts a timer when the bit stream is sent out for storage and, if the local cache and transport layer does not receive a confirmation from the stream store layer that the bit stream has been stored before the timer expires, the local cache and transport layer attempts to store the bit stream on a different node.

9. The system of claim 8, wherein each node has a time manager that keeps time for that node relative to an absolute system time, and wherein the stream store layer notifies the local cache and transport layer not to time out because the bit stream is still in the process of being stored.

10. The system of claim 6, wherein each node maintains a local routing table on behalf of one or more neighboring nodes, at least one of the local routing tables containing the hash value used to determine the address associated with the node to which the stream store message is intended to be sent.

11. The system of claim 1, wherein at least one of the chunk store messages is delivered to an intended message receiver by hopping around the network through one or more intermediary network nodes.

12. A decentralized, peer-to-peer, distributed storage system, the system comprising:

a first collection of services operative to receive a request for storage of a digital advertisement, and to route onto the network a stream store message containing a bit stream associated with the digital advertisement; and

a second collection of services operative to receive the stream store message, divide the bit stream into a plurality of check blocks, and to distribute the check blocks for storage on respective ones of a plurality of network nodes.

13. The system of claim 12, wherein at least one of the network nodes includes a storage device resident in a geographic location local to where the advertisement is to be viewed.

14. The system of claim 12, wherein the advertisement is at least one of a textual ad, a banner ad, and a rich media ad that includes full motion video and stereo audio.

15. The system of claim 12, wherein the advertisement is localized to a specific geographic region or personalized to a specific individual.

16. A decentralized, peer-to-peer, distributed storage system, the system comprising:

a first collection of services operative to receive a request for storage of at least one of a digital photograph and digital video, and to route onto the network a stream store message containing a bit stream associated with the digital photograph or digital video; and

a second collection of services operative to receive the stream store message, divide the bit stream into a plurality of check blocks, and to distribute the check blocks for storage on respective ones of a plurality of network nodes.

17. The system of claim 16, wherein the digital photograph or digital video originated from a phone-based camera.

18. The system of claim 17, wherein the phone-based camera is a wireless-phone-based camera.

19. A decentralized, peer-to-peer, distributed storage system, the system comprising:

a first collection of services operative to receive a request for storage of a video email message, and to route onto the network a stream store message containing a bit stream associated with the video email message; and

a second collection of services operative to receive the stream store message, divide the bit stream into a plurality of check blocks, and to distribute the check blocks for storage on respective ones of a plurality of network nodes.