Storage and communication de-duplication
First Claim
Patent Images
1. A method implemented by a sending device, the method comprising:
- prior to receiving a request to communicate a content item via a network, identifying one or more duplicate chunks of data from the content item by comparing data in the content item to previously stored data, wherein the one or more duplicate chunks of data from the content item are one or more chunks of data from the content item that match data that has been previously stored;
generating one or more identifiers for the one or more duplicate chunks of data, wherein each of the one or more identifiers are associated with respective ones of the one or more duplicate chunks of data;
storing the one or more identifiers;
storing the content item such that each of the stored one or more identifiers replace the respective identified ones of the one or more duplicate chunks of data, whereby each duplicate chunk of data is stored only once at the sending device; and
responsive to the request to communicate the data via a network, forming a communication to communicate via the network, the communication having the stored content item with the one or more identifiers associated with the one or more duplicate chunks of data, the one or more identifiers configured to be usable to identify whether the one or more duplicate chunks are available to a receiving device at least one of the group consisting of;
a local cache and a peer-to-peer network.
2 Assignments
0 Petitions
Accused Products
Abstract
Storage and communication de-duplication are described. In one or more implementations, a system comprises one or more modules that are implemented at least partially in hardware, the one or more modules configured to utilize one or more algorithms to calculate hashes of chunks of data, the hashes used to replace the chunks in the data for storage locally in the system as well as to communicate the hashes in response to a request received via a network for the data to avoid communicating at least one of the chunks of the data via the network.
-
Citations
20 Claims
-
1. A method implemented by a sending device, the method comprising:
-
prior to receiving a request to communicate a content item via a network, identifying one or more duplicate chunks of data from the content item by comparing data in the content item to previously stored data, wherein the one or more duplicate chunks of data from the content item are one or more chunks of data from the content item that match data that has been previously stored; generating one or more identifiers for the one or more duplicate chunks of data, wherein each of the one or more identifiers are associated with respective ones of the one or more duplicate chunks of data; storing the one or more identifiers; storing the content item such that each of the stored one or more identifiers replace the respective identified ones of the one or more duplicate chunks of data, whereby each duplicate chunk of data is stored only once at the sending device; and responsive to the request to communicate the data via a network, forming a communication to communicate via the network, the communication having the stored content item with the one or more identifiers associated with the one or more duplicate chunks of data, the one or more identifiers configured to be usable to identify whether the one or more duplicate chunks are available to a receiving device at least one of the group consisting of;
a local cache and a peer-to-peer network. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method implemented by a receiving device, the method comprising:
-
receiving a communication via a network, the communication having a content item, stored at sender, with one or more identifiers associated with one or more chunks taken from data; comparing the one or more identifiers from the communication to one or more identifiers of chunks that are available to the receiving device at a local cache; responsive to a determination that at least one of the identifiers from the communication matches at least one of the identifiers of a corresponding chunk that is available to the receiving device at the local cache, replacing the at least one identifier with the corresponding chunk from the local cache; responsive to a determination that at least one of the identifiers from the communication does not match at least one of the identifiers of a corresponding chunk that is available to the receiving device at the local cache, comparing the one or more identifiers from the communication to one or more identifiers of chunks that are available to the receiving device via a peer-to-peer network; responsive to a determination that at least one of the identifiers from the communication matches at least one of the identifiers of a corresponding chunk that is available to the receiving device via a peer-to-peer network, replacing the at least one identifier with the corresponding chunk from the peer-to-peer network; and responsive to a determination that at least one of the identifiers from the communication does not match at least one of the identifiers of a corresponding chunk that is available to the receiving device at the local cache or the peer-to-peer network, forming a communication to be communicated via the network to a sender of the communication to cause the sender to communicate the corresponding chunk via the network for receipt by the receiving device. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. A sending device comprising:
-
a memory; and at least one processor coupled to the memory, the at least one processor configured to; prior to receiving a request to communicate a content item via a network, identify one or more duplicate chunks of data from the content item by comparing data in the content item to previously stored data, wherein the one or more duplicate chunks of data from the content item are one or more chunks of data from the content item that match data that has been previously stored; generate one or more identifiers for the one or more duplicate chunks of data, wherein each of the one or more identifiers are associated with respective ones of the one or more duplicate chunks of data; store the one or more identifiers; store the content item and the one or more identifiers such that each of the one or more identifiers replace the respective ones of the one or more duplicate chunks of data, whereby each duplicate chunk of data is stored only once at the sending device; and form a communication to communicate via a network, the communication having the stored content item with the one or more identifiers associated with the one or more duplicate chunks of data, the one or more identifiers configured to be usable to identify whether the one or more duplicated chunks are available to a receiving device at at least one of the group consisting of;
a local cache and a peer-to-peer network. - View Dependent Claims (18, 19, 20)
-
Specification