Access to content addressable data over a network

• US 7,487,551 B2
• Filed: 12/07/2005
• Issued: 02/03/2009
• Est. Priority Date: 01/23/1998
• Status: Expired due to Term

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Van Riel, Jan F., Teugels, Tom, Carpentier, Paul
• Primary Examiner(s): Smithers; Matthew
• Assistant Examiner(s): CALLAHAN, PAUL E
• Application Number:
  US11/296,674
• Publication Number:
  US 20060129576A1
• Time in Patent Office: 1,154 Days
• Field of Search: 726/30, 705/2, 705/51, 713/165, 713/190
• US Class Current: 726/30
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/065 for group communications cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1068 Discovery involving direct consultation or announcement among potential requesting and potential source peers
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Access to content addressable data on a network is facilitated using digital information storing devices or data repositories (“silos”) that monitor broadcast data requests over the network. A number of silos automatically monitor both data requests and data itself that are broadcast over a network. The silos selectively store data. Each silo responds to data requests broadcast over the network with data the silo has previously intercepted. A content addressable file scheme is used to enable the data repositories to reliably identify data being requested. When a data request is received, each silo evaluates whether it has all or a portion of the data being requested and responds to requests when it has the data. Requests for data are implemented by broadcasting a cryptographic hash data identifier of the data file needed. The data identifier is used by a silo to determine which data to receive and store.

Claims

1. A method of storing at least one digital asset in a computer environment comprising a plurality of nodes coupled via at least one communication medium, the computer environment making the at least one digital asset accessible to at least one client, the method comprising an act of:

(A) storing the at least one digital asset on two or more of the plurality of nodes, the at least one digital asset comprising content and being identified by the at least one client via an identifier that is based, at least in part, on at least some of the content of the at least one digital asset, and is the only identifier that can be used to identify the at least one digital asset in any communication between the first and second nodes.

2. The method of claim 1, wherein the at least one communication medium comprises at least one network.

3. The method of claim 1, wherein at least one of the plurality of nodes comprises a silo that manages storage of digital assets.

4. The method of claim 1, wherein each of the two or more of the plurality of nodes comprises a content addressable storage system.

5. The method of claim 1, wherein the plurality of nodes are peer computers that are coupled by the at least one communication medium.

6. The method of claim 1, wherein the act (A) comprises storing the at least one digital asset on storages devices managed by the two or more of the plurality of nodes.

7. The method of claim 1, wherein the act (A) comprises storing the at least one digital asset so that it is available to the at least one client from any of the two or more of the plurality of nodes.

8. The method of claim 1, wherein the identifier is based, at least in part, on a cryptographic hash of at least some of the content of the at least one digital asset.

9. The method of claim 1, wherein each of the two or more of the plurality of nodes comprises a data repository.

10. The method of claim 1, wherein the act (A) comprises:

(A1) storing the at least one digital asset on a first of the plurality of nodes; and

(A2) subsequent to the act (A1), copying the at least one digital asset from the first of the plurality of nodes to a second of the plurality of nodes.

11. The method of claim 10, wherein the act (A2) is performed in a manner that is transparent to the at least one client.

12. The method of claim 10, wherein the act (A2) is initiated by the first of the plurality of nodes.

13. The method of claim 1, wherein the at least one communication medium that couples the plurality of nodes is a network that is external to each of the plurality of nodes, wherein each of the two or mode nodes is a computer system comprising a plurality of components interconnected via an internal communication medium reserved for internal communications among the plurality of components, and wherein the at least one communication medium is separate from the internal communication medium of any of the two or more of the plurality of nodes.

14. The method of claim 13, wherein the at least one client has access to each of the two or more nodes via the network.

15. The method of claim 1, wherein the at least one network comprises a public network.

16. The method of claim 1, wherein the at least one network comprises the Internet.

17. At least one computer readable medium encoded with a plurality of instructions for execution on a computer environment comprising at least a first node, a second node and at least one communication medium that couples the first and second nodes, the computer environment making at least one digital asset accessible to at least one client that, when executed on the first node, perform a method comprising an act of:

(A) after the at least one digital asset has been stored on the first node, copying the at least one digital asset from the first node to the second node via the at least one communication medium, the at least one digital asset comprising content and being identified by the at least one client and the first node via an identifier that is based, at least in part, on at least some of the content of the at least one digital asset and is the only identifier that can be used to identify the at least one digital asset in any communication between the first and second nodes.

18. The at least one computer readable medium of claim 17, wherein the at least one communication medium comprises at least one network.

19. The at least one computer readable medium of claim 17, wherein the first node and the second node each comprises a content addressable storage system.

20. The at least one computer readable medium of claim 17, wherein the identifier is based, at least in part, on a cryptographic hash of at least some of the content of the at least one digital asset.

21. The at least one computer readable medium of claim 17, wherein the act (A) is performed in a manner that is transparent to the at least one client.

22. The at least one computer readable medium of claim 17, wherein the at least one communication medium comprises a network that is external to first node and the second node, wherein each of the first and second nodes is a computer system comprising a plurality of components interconnected via an internal communication medium reserved for internal communications among the plurality of components, and wherein the at least one communication medium is separate from the internal communication media of the first and second nodes.

23. A first node for use in a computer environment that comprises the first node, a second node and at least one communication medium that couples the first and second nodes, the first node for storing a plurality of digital assets and making the plurality of digital assets accessible to at least one client, each of the plurality of digital assets comprising content, the first node comprising:

at least one storage device to store the plurality of digital assets; and

at least one processor programmed to;

provide a content addressable interface for the first node so that the plurality of digital assets each is identified by the first node and the at least one client via a first identifier that is based, at least in part, on at least some of the content of the digital asset, the first node further storing at least one unit of data that comprises the first identifier and metadata relating to the digital asset, the unit of data being identified via a second identifier that is based, at least in part, on the content of the unit of data; and

after at least one of the plurality of digital assets has been stored on the at least one storage device, copy the at least one of the plurality of digital assets from the first node to the second node via the at least one communication medium.

24. The first node of claim 23, wherein the at least one communication medium comprises at least one network.

25. The first node of claim 23, wherein the first node comprises a content addressable storage system.

26. The first node of claim 23, wherein the first identifier is based, at least in part, on a cryptographic hash of at least some of the content of the at least one digital asset.

27. The first node of claim 23, wherein the at least one processor is programmed to initiate the copy of the at least one of the plurality of digital assets without prompting from the at least one client.

28. The first node of claim 23, wherein the at least one communication medium comprises a network that is external to first node and the second node, wherein the first node is a computer system comprising a plurality of components interconnected via an internal communication medium reserved for internal communications among the plurality of components, and wherein the at least one communication medium is separate from the internal communication medium of the first node.

29. The first node of claim 23, wherein the first node is a silo that manages storage of the plurality of digital assets.

30. The first node of claim 23, wherein the first node is a data repository.