Sliding windows for batching index updates

• US 10,402,423 B2
• Filed: 04/25/2018
• Issued: 09/03/2019
• Est. Priority Date: 09/13/2012
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Dhuse, Greg R., Resch, Jason K.
• Primary Examiner(s): Giardino, Jr., Mark A
• Application Number:
  US15/962,923
• Publication Number:
  US 20180239807A1
• Time in Patent Office: 496 Days
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device determines a performance level that is based on updating of an index node that is associated with distributed storage within the DSN of a set of encoded data slices (EDSs) that is associated with a data object. When the computing device determines to modify an index node update time period based on the performance level that is based on the updating of the index node, the computing device further determines whether to shorten or lengthen the index node update time period and selectively/adaptively performs such shorten or lengthen operations based on such determination.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

processing circuitry operably coupled to the interface and to the memory, wherein the processing circuitry is configured to execute the operational instructions to;

determine a performance level that is based on updating of an index node that is associated with distributed storage within the DSN of a set of encoded data slices (EDSs) that is associated with a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs;

determine whether to modify an index node update time period based on the performance level that is based on the updating of the index node;

based on a first determination to modify the index node update time period, determine whether to shorten the index node update time period based on comparison of the performance level to a first threshold; and

based on a second determination to shorten the index node update time period based on a favorable comparison of the performance level to the first threshold, shorten the index node update time period.

2. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

determine the performance level that is based on the updating of the index node including at least one of to detect a frequency level of update conflicts of the index node, to detect a frequency level of update requests for the index node, or to detect an average time to update the index node.

3. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

determine whether to shorten the index node update time period based on a frequency level of update conflicts of the index node being less than a frequency threshold and a frequency level of update requests for the index node is less than an update threshold.

4. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

based on a third determination not to shorten the index node update time period, determine whether to lengthen the index node update time period based on comparison of the performance level to a second threshold; and

based on a fourth determination to lengthen the index node update time period based on a favorable comparison of the performance level to the second threshold, lengthen the index node update time period based on an average time to update the index node is greater than an update time threshold.

5. The computing device of claim 1, wherein the processing circuitry is further configured to execute the operational instructions to:

based on a third determination not to modify the index node update time period, re-determine the performance level that is based on the updating of the index node.

6. The computing device of claim 1, wherein:

a decode threshold number of EDSs are needed to recover the data segment;

a read threshold number of EDSs provides for reconstruction of the data segment;

a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

the set of EDSs is of pillar width and includes a pillar number of EDSs;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

7. The computing device of claim 1, wherein the computing device is located at a first premises that is remotely located from a second premises of at least one storage unit (SU) of a plurality of SUs within the DSN that distributedly store the set of EDSs.

8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

9. A computing device comprising:

an interface configured to interface and communicate with a dispersed or distributed storage network (DSN);

memory that stores operational instructions; and

processing circuitry operably coupled to the interface and to the memory, wherein the processing circuitry is configured to execute the operational instructions to;

determine a performance level that is based on updating of an index node that is associated with distributed storage within the DSN of a set of encoded data slices (EDSs) that is associated with a data object including at least one of to detect a frequency level of update conflicts of the index node, to detect a frequency level of update requests for the index node, or to detect an average time to update the index node, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs;

determine whether to modify an index node update time period based on the performance level that is based on the updating of the index node;

based on a first determination to modify the index node update time period, determine whether to shorten the index node update time period;

based on a second determination to shorten the index node update time period, shorten the index node update time period; and

based on a third determination not to modify the index node update time period, re-determine the performance level that is based on the updating of the index node; and

based on a fourth determination not to shorten the index node update time period, lengthen the index node update time period based on the average time to update the index node is greater than an update time threshold.

10. The computing device of claim 9, wherein the processing circuitry is further configured to execute the operational instructions to:

determine whether to shorten the index node update time period based on a frequency level of update conflicts of the index node being less than a frequency threshold and a frequency level of update requests for the index node is less than an update threshold.

11. The computing device of claim 9, wherein:

a decode threshold number of EDSs are needed to recover the data segment;

a read threshold number of EDSs provides for reconstruction of the data segment;

a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

the set of EDSs is of pillar width and includes a pillar number of EDSs;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

12. The computing device of claim 9, wherein the computing device is located at a first premises that is remotely located from a second premises of at least one storage unit (SU) of a plurality of SUs within the DSN that distributedly store the set of EDSs.

13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

14. A method for execution by a computing device, the method comprising:

determining, via an interface of the computing device that is configured to interface and communicate with a dispersed or distributed storage network (DSN), a performance level that is based on updating of an index node that is associated with distributed storage within the DSN of a set of encoded data slices (EDSs) that is associated with a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs;

determining whether to modify an index node update time period based on the performance level that is based on the updating of the index node;

based on a first determination to modify the index node update time period, determining whether to shorten the index node update time period based on comparison of the performance level to a first threshold; and

based on a second determination to shorten the index node update time period based on a favorable comparison of the performance level to the first threshold, shortening the index node update time period.

15. The method of claim 14 further comprising:

determining the performance level that is based on the updating of the index node including at least one of to detect a frequency level of update conflicts of the index node, to detect a frequency level of update requests for the index node, or to detect an average time to update the index node.

16. The method of claim 14 further comprising:

determining whether to shorten the index node update time period based on a frequency level of update conflicts of the index node being less than a frequency threshold and a frequency level of update requests for the index node is less than an update threshold.

17. The method of claim 14 further comprising:

based on a third determination not to shorten the index node update time period, determining whether to lengthen the index node update time period based on comparison of the performance level to a second threshold; and

based on a fourth determination to lengthen the index node update time period based on a favorable comparison of the performance level to the second threshold, lengthening the index node update time period based on an average time to update the index node is greater than an update time threshold.

18. The method of claim 14 further comprising:

based on a third determination not to modify the index node update time period, re-determining the performance level that is based on the updating of the index node.

19. The method of claim 14, wherein:

a decode threshold number of EDSs are needed to recover the data segment;

a read threshold number of EDSs provides for reconstruction of the data segment;

a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN;

the set of EDSs is of pillar width and includes a pillar number of EDSs;

each of the decode threshold number, the read threshold number, and the write threshold number is less than the pillar number; and

the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.

20. The method of claim 14, wherein at least one of:

the computing device is located at a first premises that is remotely located from a second premises of at least one storage unit (SU) of a plurality of SUs within the DSN that distributedly store the set of EDSs;

or

the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).