Content addressable information encapsulation, representation, and transfer

• US 20060080307A1
• Filed: 11/17/2005
• Published: 04/13/2006
• Est. Priority Date: 01/23/1998
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Van Riel, Jan F., Teugels, Tom, Carpentier, Paul R.
• Granted Patent: US 7,770,228 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/065 for group communications cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1068 Discovery involving direct consultation or announcement among potential requesting and potential source peers
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Representing a number of assets on an originating computer begins with selecting the assets to be represented. Cryptographic hash asset identifiers are generated; each of the asset identifiers is computed using the contents of a particular asset. The asset identifier is a content-based or content-addressable asset name for the asset and is location independent. An asset list is generated that includes the asset identifiers computed from the assets. A cryptographic hash asset list identifier is generated that is computed from the asset list. The asset list identifier is stored for later retrieval. The assets selected are also stored for safekeeping either locally or on a computer network. In the event of loss of the files from the originating computer, the asset list identifier is retrieved. Using the asset list identifier, the original asset list is found and retrieved from its safe location. The asset identifiers from the retrieved asset list are used to find and retrieve the individual assets from their backup locations. The assets are verified by recomputing the cryptographic hash asset. identifier for each asset retrieved and comparing it to the asset identifier from the asset list. The MD5 algorithm is used for the cryptographic hash function. Assets are retrieved using a multicast protocol. A series of importer programs searches for assets to retrieve in progressively more remote locations. Assets are retrieved whole or in segments.

Claims

1. A method for use in a computer system comprising at least first and second computers, wherein the first computer provides content addressable storage, the method comprising acts of:

(A) receiving, at the first computer, a request to access a first portion of a unit of data, the request identifying the unit of data via an identifier that is based, at least in part, on the content of the unit of data; and

(B) using the identifier to locate at least a second portion of the unit of data stored by the first computer.

2. The method of claim 1, wherein the request identifies the second computer as having issued the request, and wherein the method further comprises an act of:

(C) returning the at least a second portion of the unit of data to the second computer.

3. The method of claim 1, wherein the unit of data comprises at least one digital asset

4. The method of claim 1, wherein the identifier is generated by applying a cryptographic hash function to at least a portion of the content of the unit of data.

5. The method of claim 2, wherein the first portion of the unit of data comprises one or more individual segments of the unit of data that collectively comprise less than the entire unit of data so that the act (A) comprises an act of receiving a request to access the one or more individual segments of the unit of data;

and

wherein the second portion is the same as the first portion so that the act (C) comprises an act of returning the one or more individual segments of the entire unit of data.

6. The method of claim 5, wherein the individual segments comprise specific entries for a database or a code patch for a software application.

7. The method of claim 1, wherein the first portion of the unit of data comprises the entire unit of data so that the act (A) comprises an act of receiving a request to access the entire unit of data.

8. The method of claim 1, wherein the first portion of the unit of data comprises less than the entire unit of data so that the act (A) comprises an act of receiving a request to access less than the entire unit of data.

9. The method of claim 2, wherein the first portion of the unit of data comprises the entire unit of data so that the act (A) comprises an act of receiving a request to access the entire unit of data;

and

wherein the second portion of the unit of data comprises less than the entire unit of data so that the act (C) comprises returning less than the entire unit of data.

10. The method of claim 2, wherein the first portion of the unit of data is the same as the second portion of the unit of data.

11. The method of claim 1, wherein the first portion of the unit of data comprises less than the entire unit of data and wherein the second portion of the unit of data comprises less than the entire unit of data.

12. The method of claim 11, wherein the second portion of the unit of data is the same as the first portion of the unit of data.

13. The method of claim 1, wherein the first computer is a silo.

14. The method of claim 1, wherein the first computer is a file server.

15. The method of claim 1, wherein the first computer is a storage system.

16. The method of claim 1, wherein the computer system comprises a plurality of computers in addition to the first and second computers and wherein the plurality of computers are interconnected by at least one network, and wherein each of the plurality of computers manages storage of one or more units of data.

17. The method of claim 1, wherein the first and second computers are peer computers coupled by at least one network.

18. The method of claim 1, wherein the unit of data is stored on the first computer.

19. The method of claim 1, wherein the unit of data is stored by the first computer on a storage device managed by the first computer.

20. At least one computer readable medium encoded with instructions that, when executed on a computer system, perform a method, the computer system comprising at least first and second computers, wherein the first computer provides content addressable storage, the method comprising acts of:

(A) receiving, at the first computer, a request to access a first portion of a unit of data, the request identifying the unit of data via an identifier that is based, at least in part, on the content of the unit of data; and

(B) using the identifier to locate at least a second portion of the unit of data stored by the first computer.

21. The at least one computer readable medium of claim 20, wherein the request identifies the second computer as having issued the request, and wherein the method further comprises an act of:

(C) returning the at least a second portion of the unit of data to the second computer.

22. The at least one computer readable medium of claim 21, wherein the first portion of the unit of data comprises one or more individual segments of the unit of data that collectively comprise less than the entire unit of data so that the act (A) comprises an act of receiving a request to access the one or more individual segments of the unit of data;

and

wherein the second portion is the same as the first portion so that the act (C) comprises an act of returning the one or more individual segments of the entire unit of data.

23. The at least one computer readable medium of claim 20, wherein the first portion of the unit of data comprises less than the entire unit of data so that the act (A) comprises an act of receiving a request to access less than the entire unit of data.

24. The at least one computer readable medium of claim 21, wherein the first portion of the unit of data is the same as the second portion of the unit of data.

25. The at least one computer readable medium of claim 20, wherein the first portion of the unit of data comprises less than the entire unit of data and wherein the second portion of the unit of data comprises less than the entire unit of data.

26. The at least one computer readable medium of claim 25, wherein the second portion of the unit of data is the same as the first portion of the unit of data.

27. The at least one computer readable medium of claim 20, wherein the first computer is a storage system.

28. A first computer for use in a computer system comprising at least the first computer and a second computer, wherein the first computer provides content addressable storage, the first computer comprising:

an input for receiving access requests; and

at least one controller, coupled to the input, that;

receives, via the input, a request to access a first portion of a unit of data, the request identifying the unit of data via an identifier that is based, at least in part, on the content of the unit of data; and

uses the identifier to locate at least a second portion of the unit of data stored by the first computer.

29. The first computer of claim 28, wherein the request identifies the second computer as having issued the request, and wherein the at least one controller returns the at least a second portion of the unit of data to the second computer.

30. The first computer of claim 29, wherein the first portion of the unit of data comprises one or more individual segments of the unit of data that collectively comprise less than the entire unit of data so that the at least one controller receives a request to access the one or more individual segments of the unit of data;

and

wherein the second portion is the same as the first portion so that the at least one controller returns the one or more individual segments of the entire unit of data.

31. The first computer of claim 28, wherein the first portion of the unit of data comprises less than the entire unit of data so that the at least one controller receives a request to access less than the entire unit of data.

32. The first computer of claim 29, wherein the first portion of the unit of data is the same as the second portion of the unit of data.

33. The first computer of claim 28, wherein the first portion of the unit of data comprises less than the entire unit of data and wherein the second portion of the unit of data comprises less than the entire unit of data.

34. The first computer of claim 33, wherein the second portion of the unit of data is the same as the first portion of the unit of data.

35. The first computer of claim 28, wherein the first computer is a storage system.