Peer-To-Peer Distributed Storage

• US 20100094921A1
• Filed: 10/13/2008
• Published: 04/15/2010
• Est. Priority Date: 10/13/2008
• Status: Active Grant

• Current Assignee: MACOM Connectivity Solutions, LLC (MACOM Technology Solutions Holdings, Inc.)
• Original Assignee: Applied Micro Circuits Corporation (MACOM Technology Solutions Holdings, Inc.)
• Inventors: Roy, Subhash Chandra, Gopi, Parameswaran, Ravuri, Vinay
• Granted Patent: US 8,051,205 B2
• Time in Patent Office: Days
• US Class Current: 709/201
• CPC Class Codes: G06F 15/16 Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs coordinating program control therefor G06F9/52; in regulating and control system G05B
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104

Abstract

A system and method are provided for peer-to-peer distributed file storage in a network of connected clients. The method transmits data file peer-storage information from a first peer node to a network-connected backup manager. The backup manager creates a mapping plan, for mapping data file segments to a peer group, which typically includes a plurality of network-connected peer nodes. The backup manager transmits the mapping plan to the first peer node. The first peer node distributes the data file segments to the peer group in accordance with the mapping plan, for storage in tangible memory media. Typically, the first peer node accepts a data file and encrypts the data file. In one aspect, the backup manager creates a mapping plan for distributing (n+x) erasure-coded segments. After receipt of the mapping plan, the first peer node creates the (n+x) erasure coded file segments in preparation for distribution.

Claims

1. In a network of connected clients, a method for peer-to-peer distributed file storage, the method comprising:

transmitting data file peer-storage information from a first peer node to a network-connected backup manager;

the backup manager creating a mapping plan, for mapping data file segments to a peer group including a plurality of network-connected peer nodes;

the backup manager transmitting the mapping plan to the first peer node; and

,

the first peer node distributing the data file segments to the peer group in accordance with the mapping plan, for storage in tangible memory media.

2. The method of claim 1 wherein transmitting data file peer-storage information includes transmitting information for a file including n segments;

wherein creating the mapping plan includes the backup manager creating a mapping plan for distributing (n+x) erasure-coded segments; and

,

wherein distributing the data file segments includes the first node creating (n+x) erasure coded file segments prior to distributing file segments.

3. The method of claim 1 wherein distributing the data file segments to the peer group includes a first peer node file distribution manager generating a distributed hash table (DHT) cross-referencing file segments to the peer nodes storing the file segments.

4. The method of claim 1 further comprising:

the first peer node encrypting the data file; and

,

wherein distributing the data file segments to the peer group includes the first node distributing segments of the encrypted data file.

5. The method of claim 1 further comprising:

prior the first peer node distributing data file segments, the first peer node receiving a requested node write-access key from the backup manager; and

,

wherein distributing the data file segments includes sending the node write-access key with the data file segments, to gain storage access to a node in the peer group.

6. The method of claim 1 further comprising:

the first peer node receiving instructions to recover the stored data file;

the first peer node sending download requests for data file segments to the peer group; and

,

the first peer node reassembling the data file from the downloaded data file segments.

7. The method of claim 6 wherein sending download requests for the data file segments includes sending requests for (n+x) erasure-coded segments;

and,

wherein reassembling the data file from the downloaded data file segments includes the first peer node using an erasure code to recover n data file segments from the (n+x) coded segments.

8. The method of claim 6 wherein sending download requests to the first peer group includes:

the first peer node reading a stored DHT; and

,

accessing a list of file segments cross-referenced to the peer nodes storing the file segments.

9. The method of claim 6 further comprising:

prior to sending download requests to the peer group, the first node receiving a requested node read-access key from the backup manager; and

,

wherein sending download requests for data file segments includes sending the node read-access key with the download requests, to gain read access to a node in the peer group.

10. The method of claim 6 wherein reassembling the data file from the downloaded data file segments includes reassembling an encrypted data file;

and,

the method further comprising;

decrypting the encrypted data file.

11. The method of claim 1 further comprising:

the backup manager maintaining a record of the first node file storage activity.

12. In a network of connected clients, a system for peer-to-peer distributed file storage, the system comprising:

a group of network-connected peer devices, including a first node having a coding module with an interface to receive a data file for storage and an interface to supply data file peer-storage information, and a transceiver with an interface connected to the coding module and a network interface, the transceiver transmitting the data file peer-storage information to a backup manager;

a network-connected backup manager having an interface to receive the data file peer-storage information and to send a mapping plan to the first node, for mapping data file segments to the peer group;

wherein the first node coding module segments the data file and uses the mapping plan for cross-referencing data file segments to peer devices in a peer group; and

,

wherein the first node transceiver transmits mapped data file segments to network-connected peers for storage in tangible memory media.

13. The system of claim 12 wherein the first node coding module supplies data file peer-storage information for a data file of n segments;

wherein the backup manager creates a mapping plan for distributing the (n+x) erasure coded segments; and

,

wherein the first node creates (n+x) erasure coded file segments for distribution.

14. The system of claim 12 wherein the first node includes a file distribution manager for generating a distributed hash table (DHT) cross-referencing file segments to the peer nodes storing the file segments.

15. The system of claim 12 wherein the first node coding module encrypts the data file prior to distributing data file segments to the peer group.

16. The system of claim 12 wherein the backup manager sends a requested node write-access key to the first node transceiver;

and,

wherein the first node prepares data file segments for distribution using the node write-access key to gain storage access to a node in the peer group.

17. The system of claim 12 wherein the first node coding module prepares download requests for data file segments to the peer group, in response to receiving instructions to recover the stored data file, and reassembles the data file from the downloaded data file segments.

18. The system of claim 17 wherein the first node transceiver sends download requests for (n+x) erasure-coded segments;

and,

wherein the first node coding module uses an erasure code to recover n data file segments from the (n+x) coded segments.

19. The system of claim 17 wherein the first node coding module prepares download requests to the first peer group by reading a stored DHT, and accesses a list of file segments cross-referenced to the peer nodes storing the file segments.

20. The system of claim 17 wherein the first node transceiver receives a requested node read-access key from the backup manager;

and,

wherein the first node coding module prepares download requests using the node read-access key to gain read access to a node in the peer group.

21. The system of claim 17 wherein the first node coding module decrypts the data file subsequent to reassembling an encrypted data file.

22. The system of claim 12 wherein the backup manager maintains a record of the first node file storage activity.

23. A device for peer-to-peer distributed file storage, the device comprising:

a coding module having an interface to receive a data file and an interface to supply data file peer-storage information;

a transceiver having an interface connected to the coding module and a network interface, the transceiver transmitting the data file peer-storage information to a backup manager and receiving a mapping plan;

wherein the coding module segments the data file and uses the mapping plan for cross-referencing data file segments to peer devices in a peer group; and

,

wherein the transceiver transmits mapped data file segments to network-connected peers for storage in tangible memory media.