Peer-to-peer distributed storage

• US 8,051,205 B2
• Filed: 10/13/2008
• Issued: 11/01/2011
• Est. Priority Date: 10/13/2008
• Status: Expired due to Fees

• Current Assignee: MACOM Connectivity Solutions, LLC (MACOM Technology Solutions Holdings, Inc.)
• Original Assignee: Applied Micro Circuits Corporation (MACOM Technology Solutions Holdings, Inc.)
• Inventors: Roy, Subhash Chandra, Gopi, Parameswaran, Ravuri, Vinay
• Primary Examiner(s): Hoang; Hieu
• Application Number:
  US12/250,453
• Publication Number:
  US 20100094921A1
• Time in Patent Office: 1,114 Days
• Field of Search: 709/238, 709/242, 709/243, 709/244
• US Class Current: 709/244
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104

Abstract

A system and method are provided for peer-to-peer distributed file storage in a network of connected clients. The method transmits data file peer-storage information from a first peer node to a network-connected backup manager. The backup manager creates a mapping plan, for mapping data file segments to a peer group, which typically includes a plurality of network-connected peer nodes. The backup manager transmits the mapping plan to the first peer node. The first peer node distributes the data file segments to the peer group in accordance with the mapping plan, for storage in tangible memory media. Typically, the first peer node accepts a data file and encrypts the data file. In one aspect, the backup manager creates a mapping plan for distributing (n+x) erasure-coded segments. After receipt of the mapping plan, the first peer node creates the (n+x) erasure coded file segments in preparation for distribution.

Claims

1. In a peer-to-peer network, a method for peer-to-peer distributed file storage, the method comprising:

transmitting data file peer-storage information for a data file including a plurality of n segments for storage from a first peer node to a network-connected backup manager, the data file peer-storage information including a size of the data file and an identification of the first peer node;

based on the data file peer-storage information, the network-connected backup manager creating a mapping plan that maps (n+x) erasure-coded segments including the plurality of n segments and x extra erasure-coded segments across a plurality of peer nodes forming a peer group, wherein the mapping plan cross-references each segment of the (n+x) erasure-coded segments to a corresponding peer node from the plurality of peer nodes;

the network-connected backup manager transmitting the mapping plan to the first peer node; and

the first peer node distributing the (n+x) erasure-coded segments to the plurality of peer nodes in the peer group in accordance with the mapping plan, wherein the (n+x) erasure-coded segments are stored in tangible memory media at the plurality of peer nodes.

2. The method of claim 1 wherein the first peer node distributing the (n+x) erasure-coded segments includes a file distribution manager at the first peer node generating a distributed hash table (DHT) cross-referencing the (n+x) erasure-coded segments to the plurality of peer nodes storing the (n+x) erasure-coded segments.

3. The method of claim 1 further comprising:

the first peer node encrypting the data file prior to the first peer node distributing the (n+x) erasure-coded segments.

4. The method of claim 1 further comprising:

prior to the first peer node distributing the (n+x) erasure-coded segments, the first peer node receiving a node write-access key from the network-connected backup manager; and

wherein the first peer node distributing the (n+x) erasure-coded segments includes sending the node write-access key with the (n+x) erasure-coded segments, to gain storage access to a node associated with the node write-access key in the peer group.

5. The method of claim 1 further comprising:

the first peer node receiving instructions to recover the data file;

the first peer node sending download requests for downloading the (n+x) erasure-coded segments to the peer group; and

the first peer node reassembling the data file from the downloaded (n+x) erasure-coded segments.

6. The method of claim 5 wherein the first peer node sending download requests includes:

the first peer node reading a stored DHT; and

accessing a list of the (n+x) erasure-coded segments cross-referenced to the peer nodes storing the (n+x) erasure-coded segments.

7. The method of claim 5 further comprising:

prior to the first peer node sending download requests, the first peer node receiving a node read-access key from the network-connected backup manager; and

wherein the first peer node sending download requests includes sending the node read-access key with the download requests, to gain read access to a node associated with the node read-access key in the peer group.

8. The method of claim 5 wherein the first peer node reassembling the data file includes reassembling an encrypted data file;

and

the method further comprising;

decrypting the encrypted data file.

9. The method of claim 1 further comprising:

the network-connected backup manager maintaining a record of file storage activity of the first peer node.

10. In a peer-to-peer network, a system for peer-to-peer distributed file storage, the system comprising:

a first peer node having a coding module with a first interface to receive a data file including a plurality of n segments for storage and a second interface to supply data file peer-storage information for the data file including a size of the data file and an identification of the first peer node, and a transceiver with a third interface connected to the coding module, the transceiver transmitting the data file peer-storage information to a network-connected backup manager;

the network-connected backup manager having a fourth interface to receive the data file peer-storage information and to send a mapping plan created by the network-connected backup manager based on the data file peer-storage information to the first peer node, the mapping plan maps (n+x) erasure-coded segments including the plurality of n segments and x extra erasure-coded segments across a plurality of peer nodes forming a peer group, wherein the mapping plan cross-references each segment of the (n+x) erasure-coded segments to a corresponding peer node from the plurality of peer nodes;

wherein the first peer node coding module creates the (n+x) erasure-coded segments and uses the mapping plan for cross-referencing the (n+x) erasure-coded segments to the plurality of peer nodes in the peer group; and

wherein the first peer node transceiver transmits the (n+x) erasure-coded segments to the plurality of peer nodes in the peer group in accordance with the mapping plan, wherein the (n+x) erasure-coded segments are stored in tangible memory media at the plurality of peer nodes.

11. The system of claim 10 wherein the first peer node includes a database for storing a generated distributed hash table (DHT) cross-referencing the (n+x) erasure-coded segments to the plurality of peer nodes storing the (n+x) erasure-coded segments.

12. The system of claim 10 wherein the first peer node coding module encrypts the data file prior to the first peer node transceiver transmitting the (n+x) erasure-coded segments to the plurality of peer nodes.

13. The system of claim 10 wherein the network-connected backup manager sends a node write-access key to the first peer node transceiver;

and

wherein the first peer node coding module prepares the (n+x) erasure-coded segments for distribution with the node write-access key to gain storage access to a peer node associated with the write-access key in the peer group.

14. The system of claim 10 wherein the first peer node coding module prepares download requests for downloading the (n+x) erasure-coded segments from the peer group, in response to receiving instructions to recover the data file, and reassembles the data file from the downloaded (n+x) erasure-coded segments.

15. The system of claim 14 wherein the first peer node coding module prepares the download requests by reading a stored DHT and accessing a list of the (n+x) erasure-coded segments cross-referenced to the plurality of peer nodes storing the (n+x) erasure-coded segments.

16. The system of claim 14 wherein the first peer node transceiver receives a node read-access key from the network-connected backup manager;

and

wherein the first peer node coding module prepares the download requests with the node read-access key to gain read access to a peer node associated with the read-access key in the peer group.

17. The system of claim 14 wherein the first peer node coding module reassembles an encrypted data file and decrypts the encrypted data file to recover the data file.

18. The system of claim 10 wherein the network-connected backup manager maintains a record of file storage activity of the first peer node.

19. A device for peer-to-peer distributed file storage, the device comprising:

a coding module having a first interface to receive a data file including a plurality of n segments for storage and a second interface to supply data file peer-storage information for the data file including a size of the data file and an identification of the device;

a transceiver having a third interface connected to the coding module, the transceiver transmitting the data file peer-storage information to a backup manager connected to the device over a network and receiving a mapping plan created by the backup manager based on the data file peer-storage information, the mapping plan maps (n+x) erasure-coded segments including the plurality of n segments and x extra erasure-coded segments across a plurality of peer nodes forming a peer group, wherein the mapping plan cross-references each segment of the (n+x) erasure-coded segments to a corresponding peer node from the plurality of peer nodes;

wherein the coding module segments the data file to create the (n+x) erasure-coded segments and uses the mapping plan for cross-referencing each segment of the (n+x) erasure-coded segments to the corresponding peer node; and

wherein the transceiver transmits the (n+x) erasure-coded segments to the plurality of peer nodes in accordance with the mapping plan, wherein the (n+x) erasure-coded segments are stored in tangible memory media at the plurality of peer nodes.