Access to content addressable data over a network

• US 8,074,289 B1
• Filed: 05/29/2008
• Issued: 12/06/2011
• Est. Priority Date: 01/23/1998
• Status: Expired due to Fees

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Carpentier, Paul, Van Riel, Jan F., Teugels, Tom
• Primary Examiner(s): Shiferaw, Eleni
• Assistant Examiner(s): CALLAHAN, PAUL E
• Application Number:
  US12/129,605
• Time in Patent Office: 1,286 Days
• Field of Search: 705/2, 705/51, 713/165, 713/190, 711/108, 711/216, 711/106, 726/30
• US Class Current: 726/30
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/065 for group communications cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1068 Discovery involving direct consultation or announcement among potential requesting and potential source peers
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Access to content addressable data on a network is facilitated using digital information storing devices or data repositories (“silos”) that monitor broadcast data requests over the network. A number of silos automatically monitor both data requests and data itself that are broadcast over a network. The silos selectively store data. Each silo responds to data requests broadcast over the network with data the silo has previously intercepted. A content addressable file scheme is used to enable the data repositories to reliably identify data being requested. When a data request is received, each silo evaluates whether it has all or a portion of the data being requested and responds to requests when it has the data. Requests for data are implemented by broadcasting a cryptographic hash data identifier of the data file needed. The data identifier is used by a silo to determine which data to receive and store. A silo includes a network interface, a digital asset collector, an asset request list, asset storage, an asset identifier processor and an asset supplier. The asset identifier processor computes a cryptographic hash asset identifier for a received asset and compares it to an asset identifier on its asset request list to verify it has the correct asset. A hash of a list of assets is also computed and broadcast over the network. When the hash of the list of assets is received by a silo, it places all the assets in its asset request list.

Claims

1. A method of deleting a digital asset in a data repository that includes an asset list that identifies the digital asset as being stored in the data repository, the method comprising:

with at least one processor;

receiving from a broadcast network transmission, at the data repository, a broadcast cryptographic hash asset identifier;

adding the broadcast cryptographic hash asset identifier to a list of files to be deleted;

comparing the broadcast cryptographic hash asset identifier to a generated cryptographic hash asset identifier for the digital asset; and

deleting the digital asset from the asset list that identifies the digital asset as being stored in the data repository when the broadcast cryptographic hash asset identifier matches the generated cryptographic hash asset identifier.

2. The method of claim 1, wherein the broadcast cryptographic hash asset identifier is received in a list of cryptographic hash asset identifiers.

3. The method of claim 2, wherein the list of cryptographic hash asset identifiers is included in a descriptor file.

4. The method of claim 1, wherein the data repository is a client computer.

5. The method of claim 1, wherein the data repository is a silo.

6. The method of claim 5, further comprising:

monitoring, at the silo, a plurality of broadcast asset requests, wherein each of the broadcast asset requests comprises a requested asset identifier; and

maintaining, at the silo, a silo request list comprising a list of asset identifiers from one or more of the plurality of broadcast asset requests, each of the asset identifiers of the silo request list associated with assets that are not stored in the silo.

7. The method of claim 1, wherein the generated cryptographic hash asset identifier for the digital asset is the only identifier that can be used to identify the digital asset in the data repository.

8. At least one computer readable storage device encoded with instructions that, when executed, perform a method of deleting a digital asset in a data repository that includes an asset list that identifies the digital asset as being stored in the data repository, the method comprising:

receiving from a broadcast network transmission, at the data repository, a broadcast cryptographic hash asset identifier;

adding the broadcast cryptographic hash asset identifier to a list of files to be deleted;

comparing the broadcast cryptographic hash asset identifier to a generated cryptographic hash asset identifier for the digital asset; and

deleting the digital asset from the asset list that identifies the digital asset as being stored in the data repository when the broadcast cryptographic hash asset identifier matches the generated cryptographic hash asset identifier.

9. The at least one computer readable storage device of claim 8, wherein the broadcast cryptographic hash asset identifier is received in a list of cryptographic hash asset identifiers.

10. The at least one computer readable storage device of claim 9, wherein the list of cryptographic hash asset identifiers is included in a descriptor file.

11. The at least one computer readable storage device of claim 8, wherein the data repository is a client computer.

12. The at least one computer readable storage device of claim 8, wherein the data repository is a silo.

13. The at least one computer readable storage device of claim 8, wherein the generated cryptographic hash asset identifier for the digital asset is the only identifier that can be used to identify the digital asset in the data repository.

14. A data repository that includes an asset list that identifies a digital asset as being stored in the data repository, the data repository comprising:

at least one physical storage device that stores the asset list and the digital asset; and

at least one controller, coupled to the at least one physical storage device, that;

receives, from a broadcast network transmission, a broadcast cryptographic hash asset identifier;

adds the broadcast cryptographic hash asset identifier to a list of files to be deleted;

compares the broadcast cryptographic hash asset identifier to a generated cryptographic hash asset identifier for the digital asset; and

deletes the digital asset from the asset list that identifies the digital asset as being stored in the data repository when the broadcast cryptographic hash asset identifier matches the generated cryptographic hash asset identifier.

15. The data repository of claim 14, wherein the at lest one controller receives the broadcast cryptographic hash asset identifier in a list of cryptographic hash asset identifiers.

16. The data repository of claim 15, wherein the list of cryptographic hash asset identifiers is included in a descriptor file.

17. The data repository of claim 14, wherein the data repository is a client computer.

18. The data repository of claim 14, wherein the data repository is a silo.

19. The data repository of claim 18, wherein:

the at least one controller monitors a plurality of broadcast asset requests, wherein each of the broadcast asset requests comprises a requested asset identifier; and

the at least one physical storage device stores a silo request list comprising a list of asset identifiers from one or more of the plurality of broadcast asset requests, each of the asset identifiers of the silo request list associated with assets that are not stored in the silo.

20. The data repository of claim 14, wherein the generated cryptographic hash asset identifier for the digital asset is the only identifier that can be used to identify the digital asset in the data repository.