Accessing Data In A Content-Addressable Data Processing System

• US 20110225177A1
• Filed: 05/06/2011
• Published: 09/15/2011
• Est. Priority Date: 04/11/1995
• Status: Abandoned Application

• Current Assignee: PersonalWeb Technologies, L.L.C.
• Original Assignee: Kinetech, Inc. (Brilliant Digital Entertainment Incorporated), Level 3 Communications LLC (Lumen Technologies, Inc.)
• Inventors: Lachman, Ronald D., Farber, David A.
• Application Number:
  US13/102,337
• Publication Number:
  US 20110225177A1
• Time in Patent Office: Days
• US Class Current: 707/758
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/164 File meta data generation
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: Y10S 707/914 Video
• CPC Class Codes: Y10S 707/915 Image
• CPC Class Codes: Y10S 707/916 Audio
• CPC Class Codes: Y10S 707/922 Communications
• CPC Class Codes: Y10S 707/959 Network
• CPC Class Codes: Y10S 707/961 Associative
• CPC Class Codes: Y10S 707/966 Distributed
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation

Abstract

A method of managing information in a content-addressable computer system comprising a plurality of computers including at least a first computer and at least two second computers. The method includes creating a first identifier for a data item, at least in part, by hashing the content of the data item; sending a request including the first identifier from the first computer to the at least two second computers to access the data item, wherein at least one of the at least two second computers uses the first identifier to access the data item; and verifying that the content of the data item has not changed since the creation of the first identifier. 2. The method may include using the first identifier to locate a storage location for the data item on the computer system. The first identifier may be based on the entire content of the data item

Claims

1. A method of managing information in a content-addressable computer system comprising a plurality of computers including at least a first computer and at least two second computers, the method comprising:

(A) creating a first identifier for a data item, at least in part, by hashing the content of the data item;

(B) sending a request including the first identifier from the first computer to the at least two second computers to access the data item, wherein at least one of the at least two second computers uses the first identifier to access the data item; and

(C) verifying that the content of the data item has not changed since the creation of the first identifier.

2. The method of claim 1 further comprising:

(D) using the first identifier to locate a storage location for the data item on the computer system.

3. The method of claim 1, wherein the creating in (A) comprises:

creating the first identifier based on the entire content of the data item.

4. The method of claim 1, wherein a data item may comprise a file, a portion of a file, a page in memory, a digital message, a portion of a digital message, a digital image, a portion of a digital image, a video signal, a portion of a video signal, an audio signal, or a portion of an audio signal, a software product, or a portion of a software product, or a sequence of identifiers for components of a compound data item.

5. The method of claim 1, wherein the first identifier is the only identifier the first computer uses to identify the data item in the request to the at least two second computers.

6. The method of claim 1 wherein the hashing the content of the data item uses a hash function selected from the functions:

MD4, MD5, and SHA.

7. The method of claim 1, wherein the creating act (A) comprises generating the first identifier by applying a cryptographic hash function to the entire content of the data item.

8. The method of claim 5, wherein the data item includes a second identifier for at least one digital data item, and wherein the second identifier for the at least one digital data item is based, at least in part, on the content of the digital data item.

9. The method of claim 1, wherein the computer system comprises a plurality of computers in addition to the first computer and the at least two second computers, wherein the plurality of computers are interconnected by at least one network, and wherein each of the plurality of computers manages storage of one or more data items.

10. The method of claim 1, wherein the first computer and the at least two second computers are peer computers coupled by at least one network.

11. The method of claim 1, wherein storage of the data item is managed by at least one of the at least two second computers.

12. The method of claim 11, further comprising an act, performed by the at least one of the at least two second computers, of using the first identifier to locate a storage location for the data item on the computer system.

13. The method of claim 11, wherein the data item is stored on the at least one of the at least two second computers.

14. The method of claim 11, wherein the data item is stored by the at least one of the at least two second computers on a storage device managed by the at least one of the at least two second computers.

15. The method of claim 11, wherein the request comprises request to read the data item.

16. The method of claim 15, wherein the first computer issued the request, and wherein the method further comprises an act of transmitting the data item from the at least one of the second computers to the first computer.

17. The method of claim 1, wherein first identifier for a data item is based, at least in part, on a size or length of the data item.

18. The method of claim 11, wherein the data item comprises digital data item, wherein the request comprises request to delete the digital data item, and wherein the method further comprises an act of:

deleting the digital data item in response to the request.

19. The method of claim 1, wherein act (C) of verifying comprises:

(D) generating a third identifier for the data item based upon the content of the data item; and

(E) comparing the third identifier to the first identifier to verify that the first and third identifiers are identical.

20. The method of claim 19, wherein the act (D) comprises generating the third identifier by applying the cryptographic hash function to the at least a portion of the content of the data item.

21. The method of claim 19, wherein the act (A) comprises generating the first identifier by applying a cryptographic hash function to the entire content of the data item, and wherein the act (D) comprises an act of generating the third identifier by applying the cryptographic hash function to the entire content of the data item.

22. The method of claim 12, wherein the at least one of the at least two second computers comprises a file system to manage data items stored by the at least one of the at least two second computers, wherein the first identifier is used as a name of a file in which the data item is stored by the at least one of the at least two second computers, and wherein the method further comprises an act of using the first identifier to index into the file system to locate the data item.

23. The method of claim 22, wherein the act (A) comprises generating the first identifier by applying a cryptographic hash function to the entire content of the data item.

24. The method of claim 1 further comprising:

obtaining the first identifier at the first computer.

25. The method of claim 11, wherein at least one of the at least two second computers comprises file server.

26. The method of claim 11, wherein at least one of the at least two second computers comprises storage system.

27. The method of claim 1, wherein storage of the data item is managed by the at least one of the at least two second computers, wherein the act (B) is performed by the first computer.

28. The method of claim 1, wherein the request comprises request to have the data item stored by the at least one of the at least two second computers.

29. The method of claim 1, further comprising acts, performed by the at least one of the at least two second computers in response to receipt of the request, of:

storing the data item in at least one storage location managed by the at least one of the at least two second computers; and

associating the storage location with the first identifier.

30. The method of claim 1, further comprising an act of receiving the first identifier at the at least one of the at least two second computers.

31. The method of claim 1, further comprising an act, performed by the at least one of the at least two second computers in response to receipt of the request, of generating the first identifier.

32. The method of claim 30, wherein the act (C) comprises:

generating a third identifier for the data item; and

comparing the third identifier to the first identifier to verify that the first and third identifiers are identical.

33. The method of claim 15, wherein the first identifier in the request comprises first identifier for the digital data item, and wherein the method further comprises acts of:

receiving the digital data item at the first computer;

generating, at the first computer, a third identifier for the data item received at the first computer; and

comparing the third identifier to the first identifier to verify that the first and third identifiers are identical.

34. The method of claim 15, wherein the first computer issued the request, and wherein the method further comprises an act of receiving the data item at the first computer.

35. The method of claim 28, further comprising an act of associating, at the first computer, the data item with the first identifier.

36. The method of claim 28, wherein the act (A) is performed by at least one of the at least two second computers.

37. The method of claim 28, wherein the act (A) is performed by the first computer.

38. At least one non-transitory computer readable medium encoded with instructions that, when executed on a computer system, perform a method of managing information, wherein the computer system comprises content addressable computer system comprising a plurality of computers including at least a first computer and at least two second computers, the method comprising acts of:

(A) creating a first identifier for a data item, at least in part, by hashing the content of the data item;

(B) sending a request including the first identifier from the first computer to the at least two second computers to access the data item, wherein at least one of the at least two second computers uses the first identifier to access the data item; and

(C) verifying that the content of the data item has not changed since the creation of the first identifier.

39. The at least one non-transitory computer readable medium of claim 38, wherein the method further comprises an act of:

(D) using the first identifier to locate a storage location for the data item on the computer system.

40. The at least one non-transitory computer readable medium of claim 38, wherein the act (A) comprises an act of creating the first identifier based on the entire content of the data item.

41. The at least one non-transitory computer readable medium of claim 38, wherein the data item comprises digital data item.

42. The at least one non-transitory computer readable medium of claim 38, wherein the data item may comprise a file, a portion of a file, a page in memory, a digital message, a portion of a digital message, a digital image, a portion of a digital image, a video signal, a portion of a video signal, an audio signal, or a portion of an audio signal, a software product, or a portion of a software product, or a sequence of identifiers for components of a compound data item.

43. The at least one non-transitory computer readable medium of claim 38, wherein the first identifier is the only identifier the first computer uses to identify the data item in the request to the at least two second computers.

44. The at least one non-transitory computer readable medium of claim 38, wherein the act (A) comprises generating the first identifier by applying a cryptographic hash function to the entire content of the data item.

45. The at least one non-transitory computer readable medium of claim 42, wherein, wherein the hashing the content of the data item uses a hash function selected from the functions:

MD4, MD5, and SHA.

46. The at least one non-transitory computer readable medium of claim 38, wherein the computer system comprises a plurality of computers in addition to the first computer and the at least two second computers, wherein the plurality of computers are interconnected by at least one network, and wherein each of the plurality of computers manages storage of one or more data items.

47. The at least one non-transitory computer readable medium of claim 38, wherein the first computer and the at least two second computers are peer computers coupled by at least one network.

48. The at least one non-transitory computer readable medium of claim 38, wherein storage of the data item is managed by at least one of the at least two second computers.

49. The at least one non-transitory computer readable medium of claim 48, wherein the method further comprises an act of using the first identifier to locate a storage location for the data item on the computer system.

50. The at least one non-transitory computer readable medium of claim 48, wherein the data item is stored on at least one of the at least two second computers.

51. The at least one non-transitory computer readable medium of claim 48, wherein the data item is stored by the at least one of the at least two second computers on a storage device managed by the at least one of the at least two second computers.

52. The at least one non-transitory computer readable medium of claim 48, wherein the request comprises request to read the data item.

53. The at least one non-transitory computer readable medium of claim 52, wherein the request identifies the first computer as having issued the request, and wherein the method further comprises an act of transmitting the data item from the at least one of the at least two second computers to the first computer.

54. The at least one non-transitory computer readable medium of claim 48, wherein the request comprises request to delete the data item, and wherein the method further comprises an act of:

deleting the data item in response to the request.

55. The at least one non-transitory computer readable medium of claim 48, wherein the first identifier for a data item is based, at least in part, on a size or length of the data item.

56. The at least one non-transitory computer readable medium of claim 49, wherein the act (C) comprises acts of:

(D) generating a third identifier for the data item based upon the content of the data item; and

(E) comparing the third identifier to the first identifier to verify that the first and third identifiers are identical.

57. The at least one non-transitory computer readable medium of claim 56, wherein the act (D) comprises an act of generating the third identifier by applying the cryptographic hash function to the at least a portion of the content of the data item.

58. The at least one non-transitory computer readable medium of claim 56, wherein the act (A) comprises generating the first identifier by applying a cryptographic hash function to the entire content of the data item, and wherein the act (D) comprises an act of generating the third identifier by applying the cryptographic hash function to the entire content of the data item.

59. The at least one non-transitory computer readable medium of claim 49, wherein at least one of the at least two second computers comprises a file system to manage data items stored by the at least one of the at least two second computers, wherein the first identifier is used as a name of a file in which the data item is stored by the at least one of the at least two second computers, and wherein the method further comprises an act of using the first identifier to index into the file system to locate the data item.

60. The at least one non-transitory computer readable medium of claim 38, wherein the act (A) comprises generating the first identifier by applying a cryptographic hash function to the entire content of the data item.

61. The at least one non-transitory computer readable medium of claim 48, wherein the method further comprises an act of:

obtaining the first identifier at the first computer.

62. The at least one non-transitory computer readable medium of claim 48, wherein at least one of the at least two second computers comprises file server.

63. The at least one non-transitory computer readable medium of claim 48, wherein at least one of the at least two second computers comprises storage system.

64. The at least one non-transitory computer readable medium of claim 38, wherein storage of the data item is managed by at least one of the at least two second computers, wherein the instructions are executed, at least in part, on the first computer.

65. The at least one non-transitory computer readable medium of claim 38, wherein storage of the data item is managed by at least one of the at least two second computers.

66. The at least one non-transitory computer readable medium of claim 65, wherein the request comprises request to have the data item stored by the at least one of the at least two second computers.

67. The at least one non-transitory computer readable medium of claim 65, wherein the method further comprises acts, performed in response to receipt of the request, of:

storing the data item in at least one storage location managed by the at least one of the at least two second computers; and

associating the storage location with the identifier.

68. The at least one non-transitory computer readable medium of claim 65, wherein the method further comprises an act of receiving the first identifier at the at least one of the at least two second computers.

69. The at least one non-transitory computer readable medium of claim 65, wherein the method further comprises an act, performed in response to receipt of the request, of generating the first identifier.

70. The at least one non-transitory computer readable medium of claim 68, wherein the act (C) comprises:

generating a third identifier for the data item; and

comparing the third identifier to the first identifier to verify that the first and third identifiers are identical.

71. The at least one non-transitory computer readable medium of claim 52, wherein the first identifier in the request comprises first identifier for the digital data item, and wherein the method further comprises acts of:

receiving the digital data item at the first computer;

generating, at the first computer, a third identifier for the data item received at the second computer; and

comparing the third identifier to the first identifier to verify that first and third identifiers are identical.

72. The at least one non-transitory computer readable medium of claim 52, wherein the request identifies the first computer as having issued the request, and wherein the method further comprises an act of receiving the data item at the first computer.

73. The at least one non-transitory computer readable medium of claim 66, wherein in the method further comprises an act of associating, at the first computer, the data item with the first identifier.

74. The at least one non-transitory computer readable medium of claim 66, wherein the instructions are executed, at least in part, on at least one of the at least two second computers.

75. The at least one non-transitory computer readable medium of claim 66, wherein the instructions are executed, at least in part, on the first computer.

76. A content addressable computer system for managing information, comprising:

a first computer;

at least two second computers; and

at least one controller comprising hardware and software, in combination with said hardware, that;

creates a first identifier for a data item, at least in part, by hashing the content of the data item;

sends to the at least two second computers a request including the first identifier to access the data item, wherein at least one of the at least two second-computers uses the first identifier to access the data item; and

verifies that the content of the data item has not changed since the creation of the first identifier.

77. The content addressable computer system of claim 76, wherein the at least one controller uses the first identifier to locate a storage location for the data item on the computer system.

78. The content addressable computer system of claim 76, wherein the at least one controller creates the first identifier based on the entire content of the data item.

79. The content addressable computer system of claim 76, wherein the data item comprises digital data item.

80. The content addressable computer system of claim 76, wherein the first identifier is the only identifier the first computer uses to identify the data item in the request to the at least two second computers.

81. The content addressable computer system of claim 76, wherein the data item comprises at least one digital data item.

82. The content addressable computer system of claim 76, wherein the at least one controller generates the first identifier by applying a cryptographic hash function to the entire content of the data item.

83. The content addressable computer system of claim 80, wherein the descriptor file includes the second identifier, and wherein the second identifier is based, at least in part, on the content of the digital data item.

84. The content addressable computer system of claim 76, further comprising a plurality of computers in addition to the first computer and the at least two second computers, and a network that interconnects the plurality of computers, and wherein each of the plurality of computers manages storage of one or more data items.

85. The content addressable computer system of claim 76, wherein the first computer and the at least two second computers are peer computers coupled by at least one network.

86. The content addressable computer system of claim 76, wherein storage of the data item is managed by at least one of the at least two second computers, wherein at least one controller issues a request, from the first computer, to access the data item, and wherein the request includes the first identifier.

87. The content addressable computer system of claim 86, wherein the at least one controller is disposed, at least in part, on the at least one of the at least two second computers, and wherein the at least one controller uses the first identifier to locate a storage location for the data item on the computer system.

88. The content addressable computer system of claim 86, wherein the data item is stored on the at least one of the at least two second computers.

89. The content addressable computer system of claim 86, wherein the computer system further comprises a storage device managed by the at least one of the at least two second computers, and wherein the data item is stored by the at least one of the at least two second computers on the storage device managed by the at least one of the at least two second computers.

90. The content addressable computer system of claim 86, wherein the request comprises request to read the data item.

91. The content addressable computer system of claim 90, wherein the request identifies the first computer as having issued the request, and wherein the at least one controller transmits the data item from the at least one of the at least two second computers to the first computer.

92. The content addressable computer system of claim 86, wherein a data item may comprise a file, a portion of a file, a page in memory, a digital message, a portion of a digital message, a digital image, a portion of a digital image, a video signal, a portion of a video signal, an audio signal, or a portion of an audio signal, a software product, or a portion of a software product, or a sequence of identifiers for components of a compound data item.

93. The content addressable computer system of claim 86, wherein the data item comprises digital data item, wherein the request comprises request to delete the digital data item, and wherein the at least one controller deletes the digital data item in response to receipt of the request.

94. The content addressable computer system of claim 87, wherein the request includes the first identifier for the data item, and wherein the at least one controller:

generates a third identifier for the data item based upon the content of the data item stored by the first computer; and

compares the third identifier to the first identifier to verify that the first and third identifiers are identical.

95. The content addressable computer system of claim 94, wherein the at least one controller generates the third identifier by applying the cryptographic hash function to the at least a portion of the content of the data item.

96. The content addressable computer system of claim 94, wherein the at least one controller generates the first identifier by applying a cryptographic hash function to the entire content of the data item, and generates the third identifier by applying the cryptographic hash function to the entire content of the data item.

97. The content addressable computer system of claim 87, wherein the at least one of the at least two second computers comprises a file system to manage data items stored by the at least one of the at least two second computers, wherein the first identifier is used as the name of a file in which the data item is stored by the at least one of the at least two second computers, and wherein the at least one controller uses the first identifier to index into the file system to locate the data item.

98. The content addressable computer system of claim 97, wherein the at least one controller generates the first identifier by applying a cryptographic hash function to the entire content of the data item.

99. The content addressable computer system of claim 76, wherein the hashing the content of the data item uses a hash function selected from the functions:

MD4, MD5, and SHA.

100. The content addressable computer system of claim 86, wherein at least one of the at least two second computers comprises file server.

101. The content addressable computer system of claim 86, wherein at least one of the at least two second computers comprises storage system.

102. The content addressable computer system of claim 76, wherein storage of the data item is managed by at least one of the at least two second computers, and wherein the at least one controller is disposed, at least in part, on the at least one of the at least two second computers.

103. The content addressable computer system of claim 76, wherein storage of the data item is managed by at least one of the at least two second computers, wherein the at least one controller issues a request, from the first computer, to access the data item.

104. The content addressable computer system of claim 103, wherein the request comprises request to have the data item stored by the at least one of the at least two second computers.

105. The content addressable computer system of claim 103, wherein the at least one controller is disposed, at least in part, on the at least one of the at least two second computers, and wherein the at least one controller, in response to receipt of the request, stores the data item in at least one storage location managed by the at least one of the at least two second computers and

associates the storage location with the first identifier.

106. The content addressable computer system of claim 103, wherein the at least one controller receives the first identifier at the at least one of the at least two second computers.

107. The content addressable computer system of claim 103, wherein the at least one controller, in response to receipt of the request, generates the first identifier.

108. The content addressable computer system of claim 106, wherein the at least one controller:

generates, at the at least one of the at least two second computers, a third identifier for the data item; and

compares the third identifier to the first identifier to verify that the first and third identifiers are identical.

109. The content addressable computer system of claim 90, wherein the at least one controller:

receives the data item at the first computer;

generates, at the first computer, a third identifier for the data item received at the first computer; and

compares the third identifier to the first identifier to verify that first and third identifiers are identical.

110. The content addressable computer system of claim 90, wherein the request identifies the first computer as having issued the request, and wherein the at least one controller receives the data item at the first computer.

111. The content addressable computer system of claim 104, wherein the at least one controller associates, at the first computer, the data item with the first identifier.