Spreading load for highly popular content with asynchronous counted writes

• US 10,203,999 B2
• Filed: 11/04/2016
• Issued: 02/12/2019
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Abhijeet, Kumar, Baptist, Andrew D., Gladwin, S. Christopher, Resch, Jason K.
• Primary Examiner(s): Schell, Joseph O
• Application Number:
  US15/343,658
• Publication Number:
  US 20170054807A1
• Time in Patent Office: 830 Days
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/073 in a memory management context, e.g. virtual memory or cache management memory management G06F12/00; testing of static memory units G11C29/00
• CPC Class Codes: G06F 11/0787 Storage of error reports, e.g. persistent data storage, storage using memory protection
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1088 Reconstruction on already foreseen single or plurality of spare disks
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 11/1612 where the redundant component is persistent storage
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0619 : in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/064 : Management of blocks
• CPC Class Codes: G06F 3/0644 : Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0649 : Lifecycle management
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method begins by receiving a store data object request from the user device. The method continues by initiating storage of N instances of the received data object in the storage set. The method continues by issuing a status response to the user device when detecting that M instances have been successfully stored and sending an instance i of the data to the user device when receiving a read instance i of the data object request from the user device.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving a store data object request including a data object for storage and an instance count N from a requesting entity;

initiating storage of N instances of the data object in a storage set;

issuing a status response to the requesting entity when detecting that M instances have been successfully stored; and

sending a selected instance i of the instance count N of instances of the data object to the requesting entity when the selected instance i is available.

2. The method of claim 1, wherein the initiating storage of N instances includes:

dispersed storage error encoding the data object to produce sets of encoded data slices, and for each instance, generating one or more unique sets of slice names, replicating sets of encoded data slices to produce N replicated sets of encoded data slices, and sequentially storing each of the replicated sets of encoded data slices from instance 1 to instance N.

3. The method of claim 1, wherein the issuing a status response includes:

interpreting received write responses to identify a successfully stored instance, generating and sending the status response when a number of successfully stored instances is at least the same as an instance threshold number.

4. The method of claim 1 further comprises issuing the status response when receiving a status request from the requesting entity.

5. The method of claim 4 wherein the status response includes a number M<

=N indicating a total number of instances that have been stored thus far.

6. The method of claim 4 further comprises, if a request ID of the requesting entity is no longer known, returning a response indicating that the request ID is unknown.

7. The method of claim 6 wherein the request ID is unknown because it is any of:

incorrect or the storage completed.

8. The method of claim 1, wherein the sending an instance i of the data object to the requesting entity includes:

receiving a read instance request, determining whether the instance i has been successfully written, and when successfully written, recovering the instance i and sending the selected instance i to the requesting entity.

9. The method of claim 8, wherein the recovering includes:

retrieving at least a decode threshold number of encoded data slices for each of a plurality of sets of encoded data slices associated with the instance i, and for each set, dispersed storage error decoding the retrieved at least a decode threshold number of encoded data slices to reproduce a data segment.

10. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

receive a store data object request including a data object for storage and an instance count N from a requesting entity;

initiate storage of N instances of the data object in a storage set;

issue a status response to the requesting entity when detecting that M instances have been successfully stored; and

send a selected instance i of the instance count N of instances of the data object to the requesting entity when the selected instance i is available.

11. The computing device of claim 10, wherein the initiating storage of N instances includes:

dispersed storage error encoding the data object to produce sets of encoded data slices, and for each instance, generating one or more unique sets of slice names, replicating sets of encoded data slices to produce N replicated sets of encoded data slices, and sequentially storing each of the replicated sets of encoded data slices from instance 1 to instance N.

12. The computing device of claim 10, wherein the issuing a status response includes:

interpreting received write responses to identify a successfully stored instance, generating and sending the status response when a number of successfully stored instances is at least the same as an instance threshold number.

13. The computing device of claim 10 further comprises the processing module functioning to issue the status response when receiving a status request from the requesting entity.

14. The computing device of claim 13 wherein the status response includes a number M<

=N indicating a total number of instances that have been stored thus far.

15. The computing device of claim 13 further comprises, if a request ID of the requesting entity is no longer known, returning a response indicating that the request ID is unknown.

16. The computing device of claim 15 wherein the request ID is unknown because it is any of:

incorrect or the storage completed.

17. The computing device of claim 10, wherein the sending an instance i of the data object to the requesting entity includes:

receiving a read instance request, determining whether the instance i has been successfully written, and when successfully written, recovering the instance i and sending the selected instance i to the requesting entity.

18. The computing device of claim 17, wherein the recovering includes:

retrieving at least a decode threshold number of encoded data slices for each of a plurality of sets of encoded data slices associated with the instance i, and for each set, dispersed storage error decoding the retrieved at least a decode threshold number of encoded data slices to reproduce a data segment.