Distributed storage network for modification of a data object

• US 8,479,078 B2
• Filed: 07/19/2010
• Issued: 07/02/2013
• Est. Priority Date: 10/30/2009
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Cilfone, Bart
• Primary Examiner(s): Merant, Guerrier
• Application Number:
  US12/839,197
• Publication Number:
  US 20110107165A1
• Time in Patent Office: 1,079 Days
• Field of Search: None
• US Class Current: 714/763
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/20 Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
• CPC Class Codes: H04N 21/218 Source of audio or video content , e.g. local disk arrays details of retrieval in video databases G06F16/739

Abstract

A distributed storage network generates a plurality of data segments from a data object and stores each of the plurality of data segments as a plurality of encoded data slices generated from an error encoding dispersal function. When the distributed storage network receives a modification request for the data object, it determines a size of the plurality of data segments of the data object from a segment size field and identifies one of the plurality of data segments requiring modification. The identified data segment is reconstructed from the plurality of encoded data slices and modified in accordance with the modification request.

Claims

1. A method for modifying a data object in a distributed storage network, comprises:

receiving a modification request to modify the data object, wherein the modification request includes requested modifications to the data object and a position indicator that indicates a portion of the data object for modification;

identifying one of a plurality of data segments of the data object including the portion of the data object for modification to produce an identified data segment, wherein the identified data segment is stored as a plurality of encoded data slices that are generated from the identified data segment based on an error encoding dispersal function;

reconstructing the identified data segment from at least a number T of the plurality of encoded data slices to produce a reconstructed data segment, wherein the number T corresponds to a number of the plurality of encoded data slices needed to reconstruct the identified data segment based on the error encoding dispersal function; and

modifying the reconstructed data segment in response to the requested modifications in the modification request to generate a modified data segment.

2. The method of claim 1, wherein the position indicator in the modification request indicates one or more bytes of the data object for modification and wherein identifying one of the plurality of data segments comprises:

determining a size of the plurality of data segments of the data object from a segment size field; and

identifying one of the plurality of data segments requiring modification in response to the position indicator and the size of the plurality of data segments of the data object.

3. The method of claim 1, further comprising:

generating a plurality of rebuilt encoded data slices from the modified data segment using the error encoding dispersal function;

transmitting the plurality of rebuilt encoded data slices for storage in a plurality of storage units; and

modifying a revision field for the plurality of rebuilt encoded data slices.

4. A distributed storage processing module in a distributed storage network, comprises:

a network interface for interfacing with a plurality of storage units in the dispersed storage network; and

a processing module operably coupled to;

receive a modification request to modify a data object, wherein the data object is stored as a plurality of data segments;

identify one of the plurality of data segments requiring modification in response to the modification request to produce an identified data segment, wherein the identified data segment is stored as a plurality of encoded data slices in the plurality of storage units;

retrieve at least a number T of the plurality of encoded data slices from the plurality of storage units, wherein the number T corresponds to a number of the plurality of encoded data slices needed to reconstruct the identified data segment based on an error encoding dispersal function;

reconstruct the identified data segment from at least the number T of the plurality of encoded data slices to produce a reconstructed data segment;

modify the reconstructed data segment in accordance with the modification request to generate a modified data segment; and

generate a plurality of rebuilt encoded data slices from the modified data segment using the error encoding dispersal function and transmit the plurality of rebuilt encoded data slices for storage in the plurality of storage units.

5. The distributed storage processing module of claim 4, wherein the modification request includes requested modifications to the data object and a position indicator that points to one or more bytes of the data object for modification.

6. The distributed storage processing module of claim 5, wherein the processing module is operably coupled to identify one of the plurality of data segments requiring modification in response to the position indicator and size of the plurality of data segments of the data object.

7. The distributed storage processing module of claim 6, wherein the processing module is operably coupled to determine the size of the plurality of data segments of the data object from a segment size field stored in a user vault in a segment and slice directory.

8. The distributed storage processing module of claim 7, wherein the modification request further includes a user identification and object name.

9. The distributed storage processing module of claim 8, wherein the processing module is further operably coupled to:

determine a source name for the data object from the user name and object name, wherein the source name is associated with each of the plurality of data segments of the data object; and

determine a slice name for each of the number T of plurality of encoded data slices of the identified data segment from source name and the segment and slice directory.

10. The distributed storage processing module of claim 9, wherein the processing module is further operably coupled to:

determine a location of the number T of plurality of encoded data slices from the slice name and a virtual DSN address to physical location table.

11. The distributed storage processing module of claim 10, wherein the processing module is further operably coupled to:

modify a slice revision field for the rebuilt encoded data slices.

12. The distributed storage processing module of claim 11, wherein the processing module is further operably coupled to:

update the segment size field in the user vault in the segment and slice directory when the size of the modified data segment varies from the identified data segment; and

update a segment revision field for the modified data segment to indicate a revised number of data object bytes.

13. A method for modifying a data object in a distributed storage network, comprises:

receiving a modification request to modify the data object, wherein the data object is stored as a plurality of data segments;

identifying one of the plurality of data segments requiring modification in response to the modification request to produce an identified data segment, wherein the identified data segment is stored as a plurality of encoded data slices that are generated from the identified data segment based on an error encoding dispersal function;

reconstructing the identified data segment from at least a number T of the plurality of encoded data slices to produce a reconstructed data segment, wherein the number T corresponds to a number of the plurality of encoded data slices needed to reconstruct the identified data segment based on the error encoding dispersal function; and

modifying the reconstructed data segment in accordance with the modification request to generate a modified data segment.

14. The method of claim 13, wherein the modification request includes requested modifications to the data object and a position indicator that points to one or more bytes of the data object for modification.

15. The method of claim 14, further comprising:

determining a size of the plurality of data segments of the data object from a segment size field stored in a user vault in a segment and slice directory; and

identifying one of the plurality of data segments requiring modification in response to the position indicator and the size of the plurality of data segments of the data object.

16. The method of claim 15, further comprising:

updating the segment size field in the user vault in the segment and slice directory when the size of the modified data segment varies from the identified data segment; and

updating a segment revision field of the modified data segment to indicate a revised number of data object bytes.

17. The method of claim 13, further comprising:

generating a plurality of rebuilt encoded data slices from the modified data segment using the error encoding dispersal function; and

transmitting the plurality of rebuilt encoded data slices for storage in a plurality of storage units.

18. The method of claim 17, further comprising:

modifying a slice field for the plurality of rebuilt encoded data slices.

19. The method of claim 13, wherein modifying the reconstructed data segment comprises:

revising the reconstructed data segment in accordance with the modification request to generate the modified data segment, wherein a number of bytes of the modified data segment exceeds a fixed segment size; and

generating at least two data segments from the modified data segment to generate a plurality of divided data segments, wherein each of the plurality of divided data segments have the fixed segment size.

20. A method for modifying a data object in a distributed storage network, comprises:

identifying one of a plurality of data segments of the data object requiring modification in response to a modification request of the data object to produce an identified data segment, wherein the identified data segment is stored as a plurality of encoded data slices in a DSN memory;

determining to replace the identified data segment;

generating a replacement data segment based on one or more modifications in the modification request;

generating a plurality of replacement encoded data slices from the replacement data segment based on an error encoding dispersal function; and

transmitting the plurality of replacement encoded data slices to replace the plurality of encoded data slices of the identified data segment in the DSN memory.