Retrieving data from a distributed storage network

• US 9,304,857 B2
• Filed: 12/06/2012
• Issued: 04/05/2016
• Est. Priority Date: 12/12/2011
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Leggette, Wesley, Resch, Jason K., Baptist, Andrew
• Primary Examiner(s): JACOBS, LASHONDA T
• Application Number:
  US13/707,490
• Publication Number:
  US 20130151670A1
• Time in Patent Office: 1,216 Days
• Field of Search: 709/203, 709/217, 709/219, 709/223, 709/224
• US Class Current: 1/1
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/05 using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits H03M13/2906 takes precedence
• CPC Class Codes: H03M 13/611 Specific encoding aspects, e.g. encoding by means of decoding
• CPC Class Codes: H04L 61/00 Network arrangements, protocols or services for addressing or naming
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by a dispersed storage (DS) processing module receiving a request to retrieve a portion of a data object that is stored in a distributed storage network (DSN). The method continues with the DS processing module retrieving data storage mapping information and accessing indexing information, where the indexing information identifies a categorization of the data object into a plurality of categorical data portions. The method continues with the DS processing module identifying a specific categorical data portion of the plurality of categorical data portions and equating the specific categorical data portion to specific storage information of the data storage mapping information to identify at least one data segment of data segments of at least one storage region of storage regions. The method continues with the DS processing module retrieving the at least one data segment from the DSN in accordance with the specific storage information.

Claims

1. A method for execution by a dispersed storage processing module, the method comprises:

receiving a request to retrieve a portion of a data object that is stored in a distributed storage network (DSN), wherein the request includes a DSN address for data storage mapping information regarding the storage of the data object and a data interpretation parameter;

retrieving, based on the DSN address, the data storage mapping information from DSN memory, wherein the data storage mapping information maps storage of the data object as data segments in data storage regions of the DSN in accordance with data storage optimization parameters;

accessing, based on the data interpretation parameter, indexing information regarding the data object, wherein the indexing information identifies a categorization of the data object into a plurality of categorical data portions;

identifying, for the portion of the data object, a specific categorical data portion of the plurality of categorical data portions in accordance with the indexing information;

equating the specific categorical data portion to specific storage information of the data storage mapping information to identify at least one data segment of the data segments of at least one data storage region of the data storage regions; and

retrieving the at least one data segment of the at least one data storage region from the DSN memory in accordance with the specific storage information.

2. The method of claim 1, wherein the data storage mapping information comprises at least one of:

a mapping of a set of data partitions to the data storage regions, wherein the data object is partitioned into the set of data partitions;

information regarding the data segments as a plurality of data segments for each data partition of the set of data partitions; and

information regarding a first data segment of the plurality of data segments for each data partition of the set of data partitions.

3. The method of claim 1, wherein the retrieving the data storage mapping information comprises:

retrieving at least a decode threshold number of at least one set of encoded mapping information slices from a set of dispersed storage units of the DSN based on the DSN address; and

decoding, in accordance with a dispersed storage error coding function, the at least a decode threshold number of the at least one set of encoded mapping information slices to produce the data storage mapping information.

4. The method of claim 1 further comprises:

the data storage optimization parameters includes one or more of;

storage efficiency, storage reliability, DSN performance, data security, and read/write probabilities of the data object; and

the data interpretation parameter includes one or more of;

interpretation-based separators for data filtering, interpretation-based separators data access, chapters of the data object, pages of the data object, sub-chapters of the data object, markers within the data object, time codes associated with the data object, and run-time divisions for playback of the data object.

5. The method of claim 1, wherein the equating the specific categorical data portion to the specific storage information comprises:

aligning the indexing information with the data storage mapping information to establish a common reference point;

identifying a target reference point with respect to the common reference point for the specific categorical data portion; and

utilizing the target reference point to identify the specific storage information of the data storage mapping information with respect to the common reference point.

6. The method of claim 1, wherein the retrieving the at least one data segment comprises:

for each data segment of the at least one data segment of the at least one data storage region;

retrieving at least a decode threshold number of encoded data slices from the DSN; and

decoding, in accordance with a dispersed storage error coding function, the at least a decode threshold number of encoded data slices to reproduce the data segment.

7. The method of claim 1, wherein the accessing the indexing information regarding the data object comprises:

performing a distributed computing function on the data object stored in the DSN to generate the indexing information.

8. A dispersed storage (DS) module comprises:

a first module, when operable within a computing device, causes the computing device to;

receive a request to retrieve a portion of a data object that is stored in a distributed storage network (DSN), wherein the request includes a DSN address for data storage mapping information regarding the storage of the data object and a data interpretation parameter;

a second module, when operable within the computing device, causes the computing device to;

retrieve, based on the DSN address, the data storage mapping information from DSN memory, wherein the data storage mapping information maps storage of the data object as data segments in data storage regions of the DSN in accordance with data storage optimization parameters; and

access, based on the data interpretation parameter, indexing information regarding the data object, wherein the indexing information identifies a categorization of the data object into a plurality of categorical data portions in accordance with data interpretation parameters;

a third module, when operable within the computing device, causes the computing device to;

identify, for the portion of the data object, a specific categorical data portion of the plurality of categorical data portions in accordance with the indexing information; and

equate the specific categorical data portion to specific storage information of the data storage mapping information to identify at least one data segment of the data segments of at least one data storage region of the data storage regions; and

a fourth module, when operable within the computing device, causes the computing device to;

retrieve the at least one data segment of the at least one data storage region from the DSN memory in accordance with the specific storage information.

9. The DS module of claim 8, wherein the data storage mapping information comprises at least one of:

a mapping of a set of data partitions to the data storage regions, wherein the data object is partitioned into the set of data partitions;

information regarding the data segments as a plurality of data segments for each data partition of the set of data partitions; and

information regarding a first data segment of the plurality of data segments for each data partition of the set of data partitions.

10. The DS module of claim 8, wherein the second module functions to retrieve the data storage mapping information by:

retrieving at least a decode threshold number of at least one set of encoded mapping information slices from a set of dispersed storage units of the DSN based on the DSN address; and

decoding, in accordance with a dispersed storage error coding function, the at least a decode threshold number of the at least one set of encoded mapping information slices to produce the data storage mapping information.

11. The DS module of claim 8 further comprises:

the data storage optimization parameters includes one or more of;

storage efficiency, storage reliability, DSN performance, data security, and read/write probabilities of the data object; and

the data interpretation parameter includes one or more of;

interpretation-based separators for data filtering, interpretation-based separators data access, chapters of the data object, pages of the data object, sub-chapters of the data object, markers within the data object, time codes associated with the data object, and run-time divisions for playback of the data object.

12. The DS module of claim 8, wherein the third module functions to equate the specific categorical data portion to the specific storage information by:

aligning the indexing information with the data storage mapping information to establish a common reference point;

identifying a target reference point with respect to the common reference point for the specific categorical data portion; and

utilizing the target reference point to identify the specific storage information of the data storage mapping information with respect to the common reference point.

13. The DS module of claim 8, wherein the fourth module functions to retrieve the at least one data segment by:

for each data segment of the at least one data segment of the at least one data storage region;

retrieving at least a decode threshold number of encoded data slices from the DSN; and

decoding, in accordance with a dispersed storage error coding function, the at least a decode threshold number of encoded data slices to reproduce the data segment.

14. The DS module of claim 8, wherein the second module functions to access the indexing information regarding the data object by:

performing a distributed computing function on the data object stored in the DSN to generate the indexing information.