Simultaneously servicing high latency operations in a storage system

• US 10,353,630 B1
• Filed: 09/26/2018
• Issued: 07/16/2019
• Est. Priority Date: 09/15/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Colgrove, John, Hayes, John, Hong, Bo, Wang, Feng, Miller, Ethan, Harmer, Craig
• Primary Examiner(s): Farrokh, Hashem
• Application Number:
  US16/142,690
• Time in Patent Office: 293 Days
• Field of Search: 711103
• CPC Class Codes: G06F 11/0727 in a storage system, e.g. in a DASD or network based storage system drivers for digital recording or reproducing units G06F3/06; circuits for error detection or correction within digital recording or reproducing units G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/0751 Error or fault detection not based on redundancy power supply failures G06F1/30; network fault management H04L41/06
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3485 for I/O devices
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2212/262 configured as RAID
• CPC Class Codes: G06F 2212/502 using adaptive policy
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0629 Configuration or reconfiguration of storage systems
• CPC Class Codes: G06F 3/0634 by changing the state or mode of one or more devices
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

A system and method for effectively scheduling read and write operations among a plurality of solid-state storage devices. A computer system comprises client computers and data storage arrays coupled to one another via a network. A data storage array utilizes solid-state drives and Flash memory cells for data storage. A storage controller within a data storage array comprises an I/O scheduler. The characteristics of corresponding storage devices are used to schedule I/O requests to the storage devices in order to maintain relatively consistent response times at predicted times. In order to reduce a likelihood of unscheduled behaviors of the storage devices, the storage controller is configured to schedule proactive operations on the storage devices that will reduce a number of occurrences of unscheduled behaviors.

Claims

1. A method of simultaneously servicing high latency operations in a storage system, the method comprising:

limiting, in dependence upon an amount of redundancy data in the storage system, a number of storage devices in a device group that may simultaneously service high latency operations;

detecting a condition that requires additional bandwidth for high latency operations; and

responsive to detecting the condition that requires additional bandwidth for high latency operations, increasing the number of storage devices within the device group that may simultaneously service high latency operations.

2. The method of claim 1 wherein detecting a condition that requires additional bandwidth for high latency operations further comprises detecting that one or more pending high latency operations for a given storage device have been pending for a predetermined amount of time.

3. The method of claim 1 wherein detecting a condition that requires additional bandwidth for high latency operations further comprises detecting a predetermined number of pending high latency operations which have a at least a predetermine priority.

4. The method of claim 1 wherein the high latency operations include at least one write request.

5. The method of claim 1 wherein the high latency operations include at least one cache flush.

6. The method of claim 1 wherein the high latency operations include at least one erase request.

7. The method of claim 1 further comprising:

detecting a condition that requires additional bandwidth for low latency operations; and

responsive to detecting the condition that requires additional bandwidth for low latency operations, decreasing the number of storage devices within the device group that may simultaneously service high latency operations.

8. A method of simultaneously servicing write operations in a storage system, the method comprising:

limiting, in dependence upon a Redundant Array of Independent Disks (‘

RAID’

) level utilized when storing data within the storage system, a number of storage devices that may simultaneously service write operations directed to the storage system;

detecting a condition that requires additional bandwidth for write operations directed to the storage system; and

responsive to detecting the condition that requires additional bandwidth for write operations directed to the storage system, increasing the number of storage devices that may simultaneously service the write operations directed to the storage system.

9. The method of claim 8 wherein detecting a condition that requires additional bandwidth for write operations directed to the storage system further comprises detecting that one or more pending write operations have been pending for at least a predetermined amount of time.

10. The method of claim 8 further comprising wherein detecting a condition that requires additional bandwidth for write operations directed to the storage system further comprises detecting at least a predetermined number of pending write operations which have at least a predetermined priority.

11. The method of claim 8 further comprising:

detecting a condition that requires additional bandwidth for read operations directed to the storage system; and

responsive to detecting the condition that requires additional bandwidth for read operations, decreasing the number of storage devices that may simultaneously service write operations.

12. The method of claim 11 wherein detecting a condition that requires additional bandwidth for read operations directed to the storage system further comprises detecting at least a predetermined number of pending read requests.

13. A storage system, the storage system including a computer memory and a computer processor, the computer processor including computer program instructions that, when executed by the computer processor, cause the storage system to carry out the steps of:

setting, in dependence upon an amount of redundancy data in the storage system, a number of storage devices within the storage system that may concurrently execute high latency operations;

detecting a condition that requires an increase to the number of storage devices within the storage system that may concurrently execute high latency operations; and

responsive to detecting the condition that requires an increase to the number of storage devices within the storage system that may concurrently execute high latency operations, increasing the number of storage devices within the storage system that may concurrently execute high latency operations.

14. The storage system of claim 13 wherein detecting a condition that requires an increase to the number of storage devices within the storage system that may concurrently execute high latency operations further comprises detecting that one or more pending high latency operations have been pending for a predetermined amount of time.

16. The storage system of claim 13 wherein the high latency operations include at least one write request.

17. The storage system of claim 13 wherein the high latency operations include at least one cache flush.

18. The storage system of claim 13 wherein the high latency operations include at least one erase request.

19. The storage system of claim 13 further comprising computer program instructions that, when executed by the computer processor, cause the storage system to carry out the steps of:

detecting a condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations; and

responsive to detecting the condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations, increasing the number of storage devices within the storage system that may concurrently execute low latency operations.

20. The storage system of claim 19 wherein detecting a condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations further comprises detecting at least a predetermined number of pending low latency operations.

21. The storage system of claim 20 wherein the predetermined number of pending low latency operations includes at least a predetermined number of pending read operations directed to the storage system.

15. The storage system of 35 wherein detecting a condition that requires an increase to the number of storage devices within the storage system that may concurrently execute high latency operations further comprises detecting a predetermined number of pending high latency operations which have at least a predetermined priority.

22. An apparatus, the apparatus including a computer memory and a computer processor, the computer processor including computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

setting, in dependence upon a Redundant Array of Independent Disks (‘

RAID’

) level utilized when storing data within the storage system, a number of storage devices within the storage system that may concurrently execute high latency operations;

detecting a condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations; and

responsive to detecting the condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations, decreasing the number of storage devices within the storage system that may concurrently execute high latency operations.

23. The apparatus of claim 22 wherein detecting a condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations further comprises detecting a predetermined number of pending low latency operations.

24. The apparatus of claim 22 wherein detecting a condition that requires a decrease to the number of storage devices within the storage system that may concurrently execute high latency operations further comprises detecting that one or more pending low latency operations have been pending for at least a predetermined amount of time.

25. The apparatus of claim 22 further comprising computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:

detecting a condition that requires additional bandwidth for high latency operations; and

responsive to detecting the condition that requires additional bandwidth for high latency operations, increasing the number of storage devices within the storage system that may simultaneously service high latency operations.

26. The apparatus of claim 22 wherein detecting a condition that requires additional bandwidth for high latency operations further comprises detecting that one or more pending high latency operations have been pending for a predetermined amount of time.

27. The apparatus of claim 22 wherein detecting a condition that requires additional bandwidth for high latency operations further comprises detecting a predetermined number of pending high latency operations which have at least a predetermined priority.

28. The apparatus of claim 22 wherein the high latency operations include at least one write request.

29. The apparatus of claim 22 wherein the high latency operations include at least one cache flush.

30. The apparatus of claim 22 wherein the high latency operations include at least one erase request.