Distributed storage and communication

• US 9,026,844 B2
• Filed: 09/01/2009
• Issued: 05/05/2015
• Est. Priority Date: 09/02/2008
• Status: Active Grant

• Current Assignee: Qando Service Incorporated
• Original Assignee: Qando Service Incorporated
• Inventors: Syrgabekov, Iskender, Zadauly, Yerkin, Laumulin, Chokan
• Primary Examiner(s): Riad, Amine
• Application Number:
  US13/059,348
• Publication Number:
  US 20110145638A1
• Time in Patent Office: 2,072 Days
• Field of Search: 714/6.22
• US Class Current: 714/6.22
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: H04L 1/0041 Arrangements at the transmitter end
• CPC Class Codes: H04L 1/04 using frequency diversity
• CPC Class Codes: H04L 45/24 Multipath

Abstract

Storing, retrieving, transmitting and receiving data (20) by a) separating the data into a plurality of data subsets (A, B); b) generating parity data (P) from the plurality of data subsets (A, B) such that any one or more of the plurality of data subsets may be recreated from the remaining data subsets and the parity data (P). Steps a and b may be repeated on any one or more each of the plurality of data subsets and parity data providing further data subsets and further parity data; and d) storing each of the further data subsets and further parity data in separate storage locations (380) or transmitting the further data subsets and further parity data.

Claims

1. A method of storing data, the method comprising:

provisioning a first level cluster by separating the data into a first plurality of data subsets, and generating parity data from the first plurality of data subsets such that each data subset of the first plurality of data subsets may be recreated from other subsets of the first plurality of data subsets and the data from the first plurality of data subsets;

subsequent to provisioning the first level cluster, provisioning a second level cluster by separating the first plurality of data subsets and the parity data from the first plurality of data subsets into a second plurality of data subsets comprising a greater number of data subsets than the first plurality of data subsets, and generating parity data from the second plurality of data subsets such that each data subset of the second plurality of data subsets may be recreated from other data subsets of the second plurality of data subsets and the parity data from the second plurality of data subsets; and

subsequent to provisioning the second level cluster, storing each data subset of the second plurality of data subsets and the parity data from the second plurality of data subsets in separate storage locations that are remote from each other and accessed over an external network.

2. The method of claim 1, further comprising, subsequent to provisioning the second level cluster and before storing each data subset of the second plurality of data subsets and the parity data from the second plurality of data subsets:

separating the second plurality of data subsets and the parity data from the second plurality of data subsets into a third plurality of data subsets; and

generating parity data from the third plurality of data subsets such that a data subset of the third plurality of data subsets may be recreated from other data subsets of the third plurality of data subsets and the parity data from the third plurality of data subsets.

3. The method of claim 1, further comprising:

providing additional storage locations that are physically separate and remote from each other, and accessed over an external network; and

as the additional storage locations are provided;

separating one or more data subsets of the second plurality of data subsets into a plurality of additional data subsets;

generating parity data from the plurality of additional data subsets such that a data subset of the plurality of additional data subsets may be recreated from other data subsets of the plurality of additional data subsets and the parity data from the plurality of additional data subsets; and

storing each data subset of the plurality of additional data subsets and the parity data from the plurality of additional data subsets in the additional storage locations.

4. The method of claim 1, further comprising encrypting the data.

5. The method of claim 1, wherein the data comprises web pages.

6. The method of claim 1, further comprising:

applying a function to at least one of one or more data subsets of the second plurality of data subsets, or the parity data from the second plurality of data subsets, to generate one or more associated authentication codes.

7. The method of claim 6, wherein the function is a hash function.

8. The method of claim 6, further comprising storing an authentication code of the one or more associated authentication codes with at least one of a data subset of the second plurality of data subsets, or the parity data from the second plurality of data subsets, the authentication code of the one or more associated authentication codes being associated with the at least one of the data subset of the second plurality of data subsets, or the parity data from the second plurality of data subsets.

9. The method of claim 1, wherein generating the parity data from the first plurality of data subsets comprises performing an exclusive OR logical function on the first plurality of data subsets, or wherein generating the parity data from the second plurality of data subsets comprises performing an exclusive OR logical function on the second plurality of data subsets.

10. An apparatus for storing data, the apparatus comprising at least one processor configured to:

provision a first level cluster by separating the data into a first plurality of data subsets, and generating parity data from the first plurality of data subsets such that each data subset of the first plurality of data subsets may be recreated from other data subsets of the first plurality of data subsets and the parity data from the first plurality of data subsets;

subsequent to provisioning the first level cluster, provision a second level cluster by separating the first plurality of data subsets and the parity data from the first plurality of data subsets into a second plurality of data subsets comprising a greater number of data subsets than the first plurality of data subsets, and generating parity data from the second plurality of data subsets such that each data subset of the second plurality of data subsets may be recreated from other data subsets of the second plurality of data subsets and the parity data from the second plurality of data subsets; and

subsequent to provisioning the second level cluster, store each data subset of the second plurality of data subsets and the parity data from the second plurality of data subsets in separate storage locations that are remote from each other and accessed over an external network.

11. The apparatus of claim 10, wherein the at least one processor is further configured to:

apply a function to at least one of one or more data subsets of the second plurality of data subsets, or the parity data from the second plurality of data subsets, to generate one or more associated authentication codes; and

store the one or more data subsets of the second plurality of data subsets, or the parity data from the second plurality of data subsets, with their associated authentication codes.