System and method for accessing a data object stored in a distributed storage network

• US 9,378,091 B2
• Filed: 12/27/2013
• Issued: 06/28/2016
• Est. Priority Date: 07/16/2008
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Dhuse, Greg
• Primary Examiner(s): Richardson, James E
• Application Number:
  US14/142,206
• Publication Number:
  US 20140122970A1
• Time in Patent Office: 914 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04N 21/4331 Caching operations, e.g. of an advertisement for later insertion during playback
• CPC Class Codes: H04N 21/632 using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices broadcast-related systems characterised by transmission among terminal devices H04H60/80; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/8456 by decomposing the content in the time domain, e.g. in time segments
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

A user device includes a browser module, a DSN interface to a local or external DSN memory and a DS processing module coupled to the DSN interface for storing and retrieving the data object from the DSN memory, wherein the data object is divided into a plurality of data segments and wherein each of the plurality of data segments is stored in the DSN memory as a plurality of encoded data slices that are generated based on an error encoding dispersal function. The browser module is operable to interpret a user input as a request to display a data object, determine the data object is stored in the DSN memory, request the DS processing module to retrieve the data object from the DSN memory and request an application program to open the data object for display.

Claims

1. A non-transitory memory device storing operational instructions that, when executed by a processing module, causes the processing module to:

interpret a file command, wherein the file command includes a file name for a data object;

determine that the data object is stored in stored in a distributed storage network (DSN) memory, wherein the data object is encoded using a dispersed storage error encoding function to produce a set of encoded data segments and interleaving an encoded data segment of the set of encoded data segments into a number X of a plurality of encoded data slices wherein the number X of the plurality of encoded data slices is stored in the DSN memory and wherein the data object is encoded using the error encoding dispersal storage function by;

arranging data blocks of the data object into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix into the set of encoded data segments;

retrieve from the DSN memory at least a number T of the plurality of encoded data slices, wherein the number T of the plurality of encoded data slices is needed to reconstruct the data object based on the error encoding dispersal storage function and wherein the number T equals less than the number X; and

reconstruct the data object from at least the number T of the plurality of encoded data slices to produce a reconstructed data object.

2. The non-transitory memory device of claim 1, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

determine that the data object is stored in the DSN memory based on the file name for the data object, wherein the file name for the data object includes a domain name associated with the DSN memory.

3. The non-transitory memory device of claim 1, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

determine that the data object is stored in the DSN memory based on the file name for the data object, wherein the file name for the data object includes an identifier of a local drive mapped to the DSN memory.

4. The non-transitory memory device of claim 1, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

determine that the data object is stored in the DSN memory based on the file name for the data object, wherein the file name for the data object includes a virtual DSN address of the data object.

5. The non-transitory memory device of claim 4, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

determine a physical location for the data object by accessing a virtual DSN address to physical location table, wherein the virtual DSN address to physical location table includes a mapping of the virtual DSN address of the data object to one or more physical locations of the plurality of encoded data slices of the data object.

6. The non-transitory memory device of claim 1, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

determine a virtual DSN address for the data object by accessing a virtual DSN address space, wherein the virtual DSN address space includes a mapping of the virtual DSN address to the file name of the data object.

7. The non-transitory memory device of claim 1, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

access a user vault in a virtual DSN address space based on a user vault identifier in the file name for the data object;

determine the virtual DSN address for the data object using the virtual DSN address space and the file name for the data object; and

determine a physical location for the data object by accessing a virtual DSN address to physical location table, wherein the virtual DSN address to physical location table includes a mapping of the virtual DSN address of the data object to one or more physical locations of the plurality of encoded data slices of the data object.

8. A non-transitory memory device storing operational instructions that, when executed by a processing module, causes the processing module to:

process a file command to display multimedia content from a data object, wherein the file command includes a filename for the data object;

determine the data object is stored in a distributed storage network (DSN) memory, wherein the data object is divided into a plurality of data segments and wherein each of the plurality of data segments is encoded using an error encoding dispersal function to generate a plurality of encoded data segments and wherein each of the plurality of encoded data segments is interleaved into an X number of plurality of encoded data slices for storage in the DSN memory and wherein the data object is encoded using the error encoding dispersal storage function by;

arranging data blocks of the data object into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix into the X number of plurality of encoded data slices;

identify a first data segment of the plurality of data segments of the data object to generate an identified data segment;

retrieve at least a number T of a plurality of encoded data slices for the identified data segment from the DSN memory, wherein the number T is less than the number X and corresponds to a number of the plurality of encoded data slices needed to reconstruct the identified data segment based on the error encoding dispersal function;

reconstruct the identified data segment from at least the number T of the plurality of encoded data slices to produce a reconstructed data segment;

process the reconstructed data segment to generate the multimedia content; and

transmit the multimedia content for display in a graphical user interface.

9. The non-transitory memory device of claim 8, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

sequentially stream the plurality of data segments of the data object by;

identifying a next sequential data segment of the data object to generate an identified next sequential data segment;

retrieving at least a number T of the plurality of encoded data slices for the identified next sequential data segment from the DSN memory, wherein the number T corresponds to a number of the plurality of encoded data slices needed to reconstruct the identified next sequential data segment based on the error encoding dispersal function; and

reconstructing the identified next sequential data segment from at least the number T of the plurality of encoded data slices to produce a next reconstructed data segment.

10. The non-transitory memory device of claim 8, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

determine that the data object is stored in the DSN memory based on a file name for the data object.

11. The non-transitory memory device of claim 10, wherein the non-transitory memory device stores further operational instructions that, when executed by the processing module, causes the processing module to:

access a user vault in a virtual DSN address space;

determine the virtual DSN address for the data object using the virtual DSN address space and the file name for the data object; and

determine a physical location for the data object by accessing a virtual DSN address to physical location table, wherein the virtual DSN address to physical location table includes a mapping of the virtual DSN address of the data object to one or more physical locations of the plurality of encoded data slices of the data object.

12. A device comprising:

memory;

a processing module operably coupled to the memory, wherein the processing module is operable to implement a browsing module and a DS processing module wherein the browsing module is operable to;

process a request to display a data object;

determine the data object is stored in a distributed storage network (DSN) memory, wherein the data object is stored in the DSN memory as a plurality of sets of encoded data slices, wherein the plurality of sets of encoded data is created by dividing the data object into a plurality of data segments and wherein each of the plurality of data segments is encoded using an error encoding dispersal function to generate a respective set of encoded data slices of the plurality of sets of encoded data slices by;

arranging data blocks of a data segment of the data object into a data matrix;

generating an encoded data matrix based on the data matrix and an encoding matrix; and

arranging data blocks of the encoded data matrix into the set of encoded data slices;

request the DS processing module to retrieve the data object stored in the DSN memory, wherein the DS processing module is operable to retrieve at least a number T of each set of the plurality of sets of encoded data slices, wherein the number T equals a number of encoded data slices per set of encoded data slices needed to reconstruct a data segment of the data object based on the error encoding dispersal storage function;

reconstruct the data object from the encoded data slices retrieved by the DS processing module; and

process the reconstructed data object for display.

13. The device of claim 12, wherein the browsing module includes a DS processing module plug in operable to:

request the DS processing module to retrieve the data object stored in the DSN memory; and

receive the reconstructed data object from the DS processing module.

14. The device of claim 12, wherein the browsing module is further operable to:

launch an application program to process the reconstructed data object for display, wherein the reconstructed data object is displayed using at least one of;

a graphical user interface generated by the application program and a graphical user interface (GUI) generated by the browsing module.

15. The device of claim 12, wherein the browsing module is operable to process a request to display a data object by:

receiving a file command, wherein the file command includes a file name of the data object.

16. The device of claim 15, wherein the browsing module is operable to determine the data object is stored in the DSN memory using the file name of the data object.

17. The device of claim 12, wherein the browsing module is operable to launch the DS processing module in response to determining the data object is stored in the DSN memory.

18. The device of claim 12, wherein the browsing module is further operable to:

display a formatted page that includes at least one link to an embedded file command;

receive a selection of the at least one link to the embedded file command; and

determine the embedded file command is the request to display the data object, wherein the embedded file command includes a file name of the data object.

19. The device of claim 12, wherein the browsing module is further operable to:

receive a text string input, wherein the text string input includes a file name of a data object; and

determine the text string input is the request to display the data object.