Compute architecture in a memory device of distributed computing system

• US 10,372,506 B2
• Filed: 11/09/2016
• Issued: 08/06/2019
• Est. Priority Date: 12/12/2011
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Baptist, Andrew D., Dhuse, Greg R., Gladwin, S. Christopher, Grube, Gary W., Motwani, Manish, Leggette, Wesley B., Resch, Jason K., Shirley, Jr., Thomas F., Volvovski, Ilya
• Primary Examiner(s): Kanaan, Simon P
• Application Number:
  US15/346,948
• Publication Number:
  US 20170052858A1
• Time in Patent Office: 1,000 Days
• CPC Class Codes: G06F 11/1008 in individual solid state devices G06F11/1004 takes precedence
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5077 Logical partitioning of resources; Management or configuration of virtualized resources specific details on emulation or internal functioning of virtual machines G06F9/455
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system
• CPC Class Codes: H03M 13/09 Error detection only, e.g. using cyclic redundancy check [CRC] codes or single parity bit
• CPC Class Codes: H03M 13/1515 Reed-Solomon codes
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/06 for supporting key management in a packet data network cryptographic mechanisms or cryptographic arrangements for key management H04L9/08
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 67/10 : in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/1017 : based on a round robin mechanism
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method performed by a processing module embedded in a solid state memory device begins by receiving at least one partial task related to a group of slices of contiguous data, and slices of the group of slices of contiguous data to produce received slices. The received slices are random access stored in the solid state memory device, and the processing module decides whether to execute the at least one partial task. In response to a positive determination, a portion of the received slices are random access retrieved, and the at least one partial task is executed using the portion of the received slices to generate a partial result. The partial result is random access stored in the solid state memory device; and the processing module facilitates dispersed storage of the partial result in a distributed storage task network (DSTN).

Claims

1. A method performed by a processing module embedded in a solid state memory device, the method comprising:

receiving, at the processing module, at least one partial task related to a group of slices of contiguous data;

receiving, at the solid state memory device, slices of the group of slices of contiguous data to produce received slices;

random access storing received slices in the solid state memory device, using the processing module;

determining, by the processing module, whether to execute the at least one partial task;

in response to a positive determination;

random access retrieving, by the processing module, a portion of the-received slices from the solid state memory device;

executing, by the processing module, the at least one partial task using the portion of the received slices to generate a partial result;

random access storing, by the processing module, the partial result in the same solid state memory device used to store the received slices;

dispersed storage error encoding the partial result to produce a plurality of sets of dispersed error encoded slices; and

facilitating, by the processing module, dispersed storage of the plurality of sets of dispersed error encoded slices in a distributed storage task network (DSTN).

2. The method of claim 1, wherein the random access storing includes:

updating a slice location table.

3. The method of claim 2, wherein the random access retrieving includes:

accessing the slice location table to retrieve random access addresses within the solid state memory device.

4. The method of claim 1, further comprising:

determining executions steps and a schedule associated with the at least one partial task; and

executing the at least one partial task in accordance with the execution steps and the schedule.

5. The method of claim 1, further comprising:

identifying a particular portion of the received slices on which the at least one partial task is to be executed;

random access retrieving the particular portion of the received slices; and

executing the at least one partial task using the particular portion of the received slices to generate the partial result.

6. The method of claim 1, further comprising:

in response to an unfavorable determination, determining whether more slices are expected to be received.

7. A processing module embedded in a memory device, the processing module configured to execute instructions implementing the following method:

receiving at least one partial task related to a group of slices of contiguous data;

receiving slices of the group of slices of contiguous data to produce received slices;

random access storing received slices in the memory device;

determining whether to execute the at least one partial task;

in response to a positive determination;

random access retrieving a portion of the received slices from the memory device;

executing the at least one partial task using the portion of the received slices to generate a partial result;

random access storing the partial result in the same memory device used to store the received slices;

dispersed storage error encoding the partial result to produce a plurality of sets of dispersed error encoded slices; and

facilitating, by the processing module, dispersed storage of the plurality of sets of dispersed error encoded slices in a distributed storage task network (DSTN).

8. The processing module of claim 7, wherein the random access storing includes:

updating a slice location table.

9. The processing module of claim 8, wherein the random access retrieving includes:

accessing the slice location table to retrieve random access addresses within the memory device.

10. The processing module of claim 7, wherein the processing module is further configured to execute instructions implementing:

determining executions steps and a schedule associated with the at least one partial task; and

executing the at least one partial task in accordance with the execution steps and the schedule.

11. The processing module of claim 7, wherein the processing module is further configured to execute instructions implementing:

identifying a particular portion of the received slices on which the at least one partial task is to be executed;

random access retrieving the particular portion of the received slices; and

executing the at least one partial task using the particular portion of the received slices to generate the partial result.

12. The processing module of claim 7, wherein the processing module is further configured to execute instructions implementing:

in response to an unfavorable determination, determining whether more slices are expected.

13. A solid state memory device comprising:

solid state random access storage circuitry; and

a processor coupled to the solid state random access storage circuitry, the processor configured to;

receive a task related to a group of slices of contiguous data;

receive slices of the group of slices of contiguous data to produce received slices;

store received slices in the solid state random access storage circuitry;

determine whether to execute the task;

in response to a positive determination;

retrieve a portion of the received slices from the solid state random access storage circuitry;

execute the task on the portion of the received slices to generate a partial result;

store the partial result in the same solid state random access storage circuitry used to store the received slices;

dispersed storage error encoding the partial result to produce a plurality of sets of dispersed error encoded slices; and

facilitate dispersed storage of the plurality of sets of dispersed error encoded slices in a distributed storage task network (DSTN) memory, at least partially outside of the random access storage circuitry.

14. The solid state memory device as in claim 13, wherein the processor is further configured to:

update a slice location table.

15. The solid state memory device as in claim 14, wherein the processor is further configured to:

access the slice location table to retrieve random access addresses within the solid state random access storage circuitry.

16. The solid state memory device as in claim 13, wherein the processor is further configured to:

determine executions steps and a schedule associated with the task; and

execute the task in accordance with the execution steps and the schedule.

17. The solid state memory device as in claim 13, wherein the processor is further configured to:

identify a particular portion of the received slices on which the task is to be executed;

retrieve the particular portion of the received slices from the solid state random access storage circuitry; and

execute the task using the particular portion of the received slices to generate the partial result.