Content addressable information encapsulation, representation and transfer

• US 20060080308A1
• Filed: 11/18/2005
• Published: 04/13/2006
• Est. Priority Date: 01/23/1998
• Status: Abandoned Application

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Van Riel, Jan F., Teugels, Tom, Carpentier, Paul R.
• Application Number:
  US11/283,152
• Publication Number:
  US 20060080308A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/065 for group communications cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1068 Discovery involving direct consultation or announcement among potential requesting and potential source peers
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Representing a number of assets on an originating computer begins with selecting the assets to be represented. Cryptographic hash asset identifiers are generated; each of the asset identifiers is computed using the contents of a particular asset. The asset identifier is a content-based or content-addressable asset name for the asset and is location independent. An asset list is generated that includes the asset identifiers computed from the assets. A cryptographic hash asset list identifier is generated that is computed from the asset list. The asset list identifier is stored for later retrieval. The assets selected are also stored for safekeeping either locally or on a computer network. In the event of loss of the files from the originating computer, the asset list identifier is retrieved. Using the asset list identifier, the original asset list is found and retrieved from its safe location. The asset identifiers from the retrieved asset list are used to find and retrieve the individual assets from their backup locations. The assets are verified by recomputing the cryptographic hash asset identifier for each asset retrieved and comparing it to the asset identifier from the asset list. The MD5 algorithm is used for the cryptographic hash function. Assets are retrieved using a multicast protocol. A series of importer programs searches for assets to retrieve in progressively more remote locations. Assets are retrieved whole or in segments.

Claims

1. A method for use in a computer system comprising at least first and second computers, wherein the first computer provides content addressable storage, the method comprising an act of:

(A) transmitting, from the second computer to the first computer, a request to access a first portion of a unit of data stored by the first computer, the request identifying the unit of data via an identifier that is based, at least in part, on the content of the unit of data, the first portion of the unit of data comprising less than the entire unit of data.

2. The method of claim 1, wherein the unit of data comprises at least one digital asset.

3. The method of claim 1, wherein the act of transmitting comprises an act of transmitting a request that identifies the unit of data via an identifier that was generated by applying a cryptographic hash function to at least a portion of the content of the unit of data.

4. The method of claim 1, further comprising an act of:

(B) receiving a second portion of the unit of data at the second computer.

5. The method of claim 4, wherein the first portion of the unit of data comprises one or more individual segments of the unit of data that collectively comprise less than the entire unit of data so that the act (A) comprises an act of transmitting a request to access the one or more individual segments of the unit of data;

and

wherein the second portion equals the first portion so that the act (B) comprises an act of receiving the one or more individual segments of the unit of data.

6. The method of claim 5, wherein the individual segments comprise specific entries for a database or a code patch for a software application.

7. The method of claim 4, wherein the first portion of the unit of data is the same as the second portion of the unit of data.

8. The method of claim 4, wherein the second portion of the unit of data comprises less than the entire unit of data.

9. The method of claim 1, wherein the first computer is a silo.

10. The method of claim 1, wherein the first computer is a file server.

11. The method of claim 1, wherein the first computer is a storage system.

12. The method of claim 1, wherein the computer system comprises a plurality of computers in addition to the first and second computers and wherein the plurality of computers are interconnected by at least one network, and wherein each of the plurality of computers manages storage of one or more units of data.

13. The method of claim 1, wherein the first and second computers are peer computers coupled by at least one network.

14. The method of claim 1, wherein the unit of data is stored on the first computer.

15. The method of claim 1, wherein the unit of data is stored by the first computer on a storage device managed by the first computer.

16. At least one computer readable medium encoded with instructions that, when executed on a computer system, perform a method, the computer system comprising at least first and second computers, wherein the first computer provides content addressable storage, the method comprising an act of:

(A) transmitting, from the second computer to the first computer, a request to access a first portion of a unit of data stored by the first computer, the request identifying the unit of data via an identifier that is based, at least in part, on the content of the unit of data, the first portion of the unit of data comprising less than the entire unit of data.

17. The at least one computer readable medium of claim 16, wherein the method further comprises an act of:

(B) receiving a second portion of the unit of data at the second computer.

18. The at least one computer readable medium of claim 17, wherein the first portion of the unit of data comprises one or more individual segments of the unit of data that collectively comprise less than the entire unit of data so that the act (A) comprises an act of transmitting a request to access the one or more individual segments of the unit of data;

and

wherein the second portion equals the first portion so that the act (B) comprises an act of receiving the one or more individual segments of the unit of data.

19. The at least one computer readable medium of claim 17, wherein the first portion of the unit of data is the same as the second portion of the unit of data.

20. The at least one computer readable medium of claim 17, wherein the second portion of the unit of data comprises less than the entire unit of data.

21. A second computer for use in a computer system comprising at least a first computer and the second computer, wherein the first computer provides content addressable storage, the second computer comprising:

an output for transmitting access requests; and

at least one controller, coupled to the output, that transmits, from the second computer to the first computer, a request to access a first portion of a unit of data stored by the first computer, the request identifying the unit of data via an identifier that is based, at least in part, on the content of the unit of data, the first portion of the unit of data comprising less than the entire unit of data.

22. The second computer of claim 21, wherein the at least one controller receives a second portion of the unit of data at the second computer.

23. The second computer of claim 22, wherein the first portion of the unit of data comprises one or more individual segments of the unit of data that collectively comprise less than the entire unit of data so that the at least one controller transmits a request to access the one or more individual segments of the unit of data;

and

wherein the second portion equals the first portion so that the at least one controller receives the one or more individual segments of the unit of data.

24. The second computer of claim 22, wherein the first portion of the unit of data is the same as the second portion of the unit of data.

25. The second computer of claim 22, wherein the second portion of the unit of data comprises less than the entire unit of data.