Access to content addressable data over a network

• US 7,503,076 B2
• Filed: 09/26/2003
• Issued: 03/10/2009
• Est. Priority Date: 01/23/1998
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Van Riel, Jan F., Teugels, Tom, Carpentier, Paul R.
• Primary Examiner(s): Moise; Emmanuel L
• Assistant Examiner(s): CALLAHAN, PAUL E
• Application Number:
  US10/673,356
• Publication Number:
  US 20040068652A1
• Time in Patent Office: 1,992 Days
• Field of Search: 705/2, 705/51, 713/165, 713/190, 369/27.01, 711/108, 711/216, 726/30
• US Class Current: 726/30
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: H04L 12/1854 with non-centralised forwarding system, e.g. chaincast
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/065 for group communications cryptographic mechanisms or cryptographic arrangements for key management involving conference or group key H04L9/0833
• CPC Class Codes: H04L 63/12 Applying verification of the received information cryptographic mechanisms or cryptographic arrangements for data integrity or data verification H04L9/32
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1068 Discovery involving direct consultation or announcement among potential requesting and potential source peers
• CPC Class Codes: H04L 67/108 characterised by resources being split in blocks or fragments
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Access to content addressable data on a network is facilitated using digital information storing devices or data repositories (“silos”) that monitor broadcast data requests over the network. A number of silos automatically monitor both data requests and data itself that are broadcast over a network. The silos selectively store data. Each silo responds to data requests broadcast over the network with data the silo has previously intercepted. A content addressable file scheme is used to enable the data repositories to reliably identify data being requested. When a data request is received, each silo evaluates whether it has all or a portion of the data being requested and responds to requests when it has the data. Requests for data are implemented by broadcasting a cryptographic has data identifier of the data file needed. The data identifier is used by a silo to determine which data to receive and store.

Claims

1. A method of storing a digital asset in a data repository coupled to a network, said method comprising:

receiving, at the data repository, a broadcast cryptographic hash descriptor file identifier that identifies the digital asset;

determining whether the broadcast cryptographic hash descriptor file identifier is a cryptographic hash descriptor file identifier known to the data repository;

when the broadcast cryptographic hash descriptor file is not known to the data repository, adding the broadcast cryptographic hash descriptor file identifier to a list of desired broadcast cryptographic hash descriptor file identifiers;

receiving, at the data repository, the digital asset identified by the broadcast cryptographic hash descriptor file identifier;

generating a second cryptographic hash descriptor file identifier from the received digital asset; and

verifying that the second cryptographic hash descriptor file identifier for the received digital asset matches the broadcast cryptographic hash descriptor file identifier, wherein adding the broadcast cryptographic hash descriptor file identifier to the list includes;

determining the number of times the broadcast cryptographic hash descriptor file identifier has been received at the data repository; and

determining whether to add the cryptographic hash descriptor file identifier to said list based upon said number of times.

2. A method of storing a digital asset in a data repository coupled to a network, said method comprising:

receiving, at the data repository, a broadcast cryptographic hash descriptor file identifier that identifies the digital asset;

determining whether the broadcast cryptographic hash descriptor file identifier is a cryptographic hash descriptor file identifier known to the data repository;

when the broadcast cryptographic hash descriptor file is not known to the data repository, adding the broadcast cryptographic hash descriptor file identifier to a list of desired broadcast cryptographic hash descriptor file identifiers;

receiving, at the data repository, the digital asset identified by the broadcast cryptographic hash descriptor file identifier;

generating a second cryptographic hash descriptor file identifier from the received digital asset; and

verifying that the second cryptographic hash descriptor file identifier for the received digital asset matches the broadcast cryptographic hash descriptor file identifier, wherein receiving the digital asset identified by the transmitted cryptographic hash descriptor file identifier includes;

receiving portions of said asset identified by the transmitted cryptographic hash descriptor file identifier at different times; and

assembling the portions of the asset into the complete asset.

3. A method as recited in claim 2, further comprising issuing a broadcast request for the digital asset corresponding to the broadcast cryptographic hash descriptor file identifier, and wherein the method further comprises:

ending the broadcast request for portions of the digital asset that have not been obtained.

4. A method as recited in claim 3, further comprising:

determining an amount of broadcast traffic on at least a portion of the network; and

determining whether to send the broadcast request based on the amount of broadcast traffic on the at least a portion of the network.

5. A method of storing a digital asset in a data repository coupled to a network, said method comprising:

receiving, at the data repository, a broadcast cryptographic hash descriptor file identifier that identifies the digital asset;

determining whether the broadcast cryptographic hash descriptor file identifier is a cryptographic hash descriptor file identifier known to the data repository;

when the broadcast cryptographic hash descriptor file is not known to the data repository, adding the broadcast cryptographic hash descriptor file identifier to a list of desired broadcast cryptographic hash descriptor file identifiers;

receiving, at the data repository, the digital asset identified by the broadcast cryptographic hash descriptor file identifier;

generating a second cryptographic hash descriptor file identifier from the received digital asset;

verifying that the second cryptographic hash descriptor file identifier for the received digital asset matches the broadcast cryptographic hash descriptor file identifier; and

quarantining the asset while verifying that the generated cryptographic hash descriptor file identifier matches the broadcast cryptographic hash descriptor file identifier.

6. A method of storing a digital asset in a data repository coupled to a network, said method comprising:

receiving, at the data repository, a broadcast cryptographic hash descriptor file identifier that identifies the digital asset;

determining whether the broadcast cryptographic hash descriptor file identifier is a cryptographic hash descriptor file identifier known to the data repository;

when the broadcast cryptographic hash descriptor file is not known to the data repository, adding the broadcast cryptographic hash descriptor file identifier to a list of desired broadcast cryptographic hash descriptor file identifiers;

receiving, at the data repository, the digital asset identified by the broadcast cryptographic hash descriptor file identifier;

generating a second cryptographic hash descriptor file identifier from the received digital asset;

verifying that the second cryptographic hash descriptor file identifier for the received digital asset matches the broadcast cryptographic hash descriptor file identifier; and

responding to a request, received over the network from another device, for a digital asset stored in the data repository by broadcasting portions of the stored asset; and

broadcasting the portions of the stored asset before the entire asset is received at the data repository.