Distributing and accessing data in a data processing system

• US 20080071855A1
• Filed: 10/31/2007
• Published: 03/20/2008
• Est. Priority Date: 04/11/1995
• Status: Active Grant

• Current Assignee: PersonalWeb Technologies, L.L.C.
• Original Assignee: Kinetech, Inc. (Brilliant Digital Entertainment Incorporated), Level 3 Communications LLC (Lumen Technologies, Inc.)
• Inventors: Farber, David, Lachman, Ronald
• Granted Patent: US 7,945,539 B2
• Time in Patent Office: Days
• US Class Current: 709/201
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/164 File meta data generation
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: Y10S 707/914 Video
• CPC Class Codes: Y10S 707/915 Image
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/922 Communications
• CPC Class Codes: Y10S 707/959 Network
• CPC Class Codes: Y10S 707/961 Associative
• CPC Class Codes: Y10S 707/966 Distributed
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A method of distributing a data item to a plurality of computers in a network of computers. A data item is divided into a plurality of segments. Each segment is stored on at least one computer in said network. A list of computers onto which each of said segments has been stored is maintained. When a data item is requested, its component segments are obtained from at least two computers in the network. Some of the computers may form a peer-to-peer network.

Claims

1. A method of distributing a data item to a plurality of computers in a network of computers, the method comprising:

(A) dividing the data item into a plurality of segments; and

(B) for each particular segment of said plurality of segments,

(b1) causing each said particular segment to be stored on at least one computer in said network;

(C) maintaining a list of computers onto which each of said segments has been stored.

2. A method as recited in claim 1 wherein each segment of the plurality of segments is distinct from each other segment.

3. A method as recited in claim 1 wherein at least some of said computers in said network make up part of a peer-to-peer network of computers.

4. A method as recited in claim 1 further comprising:

(D) maintaining, for each computer on said list of computers, information relating to a measure of availability of said each computer.

5. A method as recited in claim 1 further comprising:

for each particular segment, determining an identifier, said identifier being determined as a function of all of the data in said particular segment and only the data in said particular segment, where any two identical segments will have identical identifiers, and wherein the step (C) of maintaining further comprises associating each of said segments with its corresponding identifier.

6. A method as recited in claim 1 wherein at least some of said plurality of segments are stored in an encoded form by compressing said at least some segments.

7. A method as recited in claim 6 wherein at least some of said segments are compressed after being sent.

8. A method as recited in claim 1 further comprising:

for each particular segment, determining an identifier, said identifier being determined as a given function of the data comprising said particular segment, where any two identical segments will have identical identifiers, and wherein the step (C) of maintaining further comprises associating each of said segments with its corresponding identifier.

9. A method of distributing a plurality of data items to a plurality of computers in a network of computers, the method comprising:

(A) for each of said plurality of data items,

(a1) dividing the data item into a plurality of segments; and

(a2) for each particular segment of said plurality of segments,

(a21) causing each said particular segment to be stored on at least one computer in said network;

(B) maintaining a list of computers onto which each of said segments for each of said plurality of data items has been stored.

10. A method of obtaining a data item at a computer in a network of computers, said data item comprising a plurality of segments, the method comprising:

(A) for each particular segment of said plurality of segments that comprise said data item, said computer;

(a1) requesting said particular segment from at least one of a plurality of computers in said network of computers; and

(a2) obtaining said particular segment from said at least one of a plurality of computers in said network of computers.

11. A method as recited in claim 10 further comprising:

said computer combining each said particular segment to form said data item.

12. A method as recited in claim 10 wherein at least some of said computers in said network make up part of a peer-to-peer network of computers.

13. A method as recited in claim 10 further comprising, for each said particular segment:

obtaining a list of one or more computers that should have a copy of said particular segment stored thereon, and wherein said step (a1) of requesting comprises requesting said particular segment from at least of said computers that should have a copy of said particular segment stored thereon.

14. A method as recited in claim 11 wherein said segments are stored on computers in said network in an encoded form, and wherein said step (a2) of obtaining, obtains the encoded form of each said particular segment, the method further comprising:

for each particular segment of said plurality of segments that comprise said data item;

(a3) decoding the particular segment.

15. A method as recited in claim 14 wherein the encoded form comprises a compressed form, and wherein step (a3) comprises decompressing the particular segment.

16. A method as recited in claim 10 wherein said particular segments have been stored on said plurality of computers prior to said step (a1) of requesting.

17. A method of obtaining a data item at a computer in a network of computers, said data item comprising a plurality of segments, the method comprising:

(A) for each particular segment of said plurality of segments that comprise said data item;

(a0) obtaining a list of computers in said network of computers that should have said particular segment;

(a1) requesting said particular segment from at least one computer on said list of computers; and

(a2) obtaining of said particular segment from said at least one computer on said list; and

(B) combining each said particular segment to form said data item.

18. A method as recited in claim 17 wherein at least some of said computers in said network make up part of a peer-to-peer network of computers.

19. A method as recited in claim 17 wherein said step (a2) obtains an encoded copy of said particular segment, the method further comprising the step of:

(a3) decoding the encoded copy of the particular segment.

20. A method as recited in claim 19 wherein said encoded copy is encoded by compressing said particular segment, and wherein said step (a3) of decoding comprises decompressing said encoded copy of the particular segment.

21. A method of storing a data item, the method comprising:

(A) dividing the data item into a plurality of parts; and

(B) causing each particular part of said plurality of parts of said data item to be stored on at least one computer a said network of computers;

(C) maintaining a list of computers in said network of computers onto which each of said parts has been stored.

22. A method as recited in claim 21 wherein at least some of said computers in said network make up part of a peer-to-peer network of computers.