System, methods, and apparatus for subdividing data for storage in a dispersed data storage grid

• US 8,468,311 B2
• Filed: 06/05/2012
• Issued: 06/18/2013
• Est. Priority Date: 09/30/2005
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gladwin, S. Christopher, England, Matthew M.
• Primary Examiner(s): CHOE, YONG J
• Application Number:
  US13/488,650
• Publication Number:
  US 20120246421A1
• Time in Patent Office: 378 Days
• Field of Search: 711/154, 711/161, 711/170, 711/E12.001, 709/219
• US Class Current: 711/154
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries

Abstract

An efficient method for breaking source data into smaller data subsets and storing those subsets along with coded information about some of the other data subsets on different storage nodes such that the original data can be recreated from a portion of those data subsets in an efficient manner.

Claims

1. A method comprises:

dividing a string of data into a plurality of data subsets;

generating a plurality of data slice subsets from the plurality of data subsets;

combining two or more of the plurality of data slice subsets into a first data slice by at least one of;

appending a first one of the two or more of the plurality of data slice subsets to a second one of the two or more of the plurality of data slice subsets; and

forming the first data slice from the two or more of the plurality of data slice subsets; and

combining another two or more of the plurality of data slice subsets into a second data slice.

2. The method of claim 1 further comprises:

sending the first data slice to a first storage node of a dispersed storage system; and

sending the second data slice to a second storage node of the dispersed storage system.

3. The method of claim 1, wherein the generating the plurality of data slice subsets from the plurality of data subsets comprises:

performing a coding function on the plurality of data subsets to produce coded data subsets, wherein the plurality of data slice subsets includes the coded data subsets and the plurality of data subsets.

4. The method of claim 1, wherein the generating the plurality of data slice subsets from the plurality of data subsets comprises:

performing an information dispersal algorithm on the plurality of data subsets to produce the plurality of data slice subsets.

5. The method of claim 1 further comprises:

the string of data is one of a plurality of strings of a data, wherein the plurality of strings of data represent a file.

6. The method of claim 5 further comprises:

dividing a second string of data of the plurality of strings of data into a second plurality of data subsets;

generating a second plurality of data slice subsets from the second plurality of data subsets;

combining two or more of the second plurality of data slice subsets into a third data slice;

combining another two or more of the second plurality of data slice subsets into a fourth data slice;

sending the third data slice to the first storage node of a dispersed storage system; and

sending the fourth data slice to the second storage node of the dispersed storage system.

7. A computer comprises:

a communication network interface;

a computer processing module operably coupled to;

divide a string of data into a plurality of data subsets;

generate a plurality of data slice subsets from the plurality of data subsets;

combine two or more of the plurality of data slice subsets into a first data slice by at least one of;

appending a first one of the two or more of the plurality of data slice subsets to a second one of the two or more of the plurality of data slice subsets; and

forming the first data slice from the two or more of the plurality of data slice subsets; and

combine another two or more of the plurality of data slice subsets into a second data slice;

send, via the communication network interface, the first data slice to a first storage node of a dispersed storage system; and

send, via the communication network interface, the second data slice to a second storage node of the dispersed storage system.

8. The computer of claim 7, wherein the computer processing module is further operable to generate the plurality of data slice subsets from the plurality of data subsets by:

performing a coding function on the plurality of data subsets to produce coded data subsets, wherein the plurality of data slice subsets includes the coded data subsets and the plurality of data subsets.

9. The computer of claim 7, wherein the computer processing module is further operable to generate the plurality of data slice subsets from the plurality of data subsets by:

performing an information dispersal algorithm on the plurality of data subsets to produce the plurality of data slice subsets.

10. The computer of claim 7 further comprises:

the string of data is one of a plurality of strings of a data, wherein the plurality of strings of data represent a file.

11. The computer of claim 10, wherein the computer processing module is further operable to:

divide a second string of data of the plurality of strings of data into a second plurality of data subsets;

generate a second plurality of data slice subsets from the second plurality of data subsets;

combine two or more of the second plurality of data slice subsets into a third data slice;

combine another two or more of the second plurality of data slice subsets into a fourth data slice;

send, via the communication network interface, the third data slice to the first storage node of a dispersed storage system; and

send, via the communication network interface, the fourth data slice to the second storage node of the dispersed storage system.