Methods, systems and computer readable media for intelligent fetching of data storage device commands from submission queues

• US 9,927,983 B2
• Filed: 09/28/2015
• Issued: 03/27/2018
• Est. Priority Date: 09/28/2015
• Status: Active Grant

• Current Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Original Assignee: SanDisk Technologies LLC (Western Digital Corporation)
• Inventors: Benisty, Shay, Harari Shechter, Noga, Segev, Amir, Sharifie, Tal
• Primary Examiner(s): THAI, TUAN V
• Application Number:
  US14/868,373
• Publication Number:
  US 20170090753A1
• Time in Patent Office: 911 Days
• Field of Search: 711100, 711103, 711154
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

Methods, systems, and computer readable media for intelligent fetching of storage device commands from submission queues are disclosed. On method is implemented in a data storage device including a controller and a memory. The method includes collecting submission queue command statistics; monitoring resource state of the data storage device. The method further includes using the submission queue command statistics and the resource state to select a submission queue from which a next data storage device command should be fetched. The method further includes fetching the command from the selected submission queue. The method further includes providing the command to command processing logic.

Claims

1. A method, comprising:

collecting command statistics pertaining to storage device commands previously fetched from respective submission queues of a plurality of submission queues associated with a data storage device;

fetching storage device commands from the submission queues, wherein fetching a storage device command comprises;

selecting a submission queue from which to fetch the storage device command based on the command statistics pertaining to the storage device commands previously fetched from the respective submission queues and a state of one or more resources of the data storage device;

fetching the command from the selected submission queue; and

providing the command to command processing logic,

wherein collecting the command statistics comprises collecting information regarding whether particular submission queues historically contain a majority of random input/output (I/O) commands or a majority of sequential I/O commands.

2. The method of claim 1, wherein collecting the command statistics further comprises collecting statistics regarding a number of pending storage device commands in the respective submission queues.

3. The method of claim 1, wherein collecting the command statistics further comprises collecting an indication of a number of storage device commands previously fetched from the respective submission queues.

4. The method of claim 1, wherein collecting the command statistics further comprises collecting command type information for the storage device commands previously fetched from the respective submission queues.

5. The method of claim 1, wherein:

collecting the command statistics comprises determining whether the respective submission queues are likely to comprise one of;

read commands and write commands based on the storage device commands previously fetched from the respective submission queues, and

selecting the submission queue from which to fetch the storage device command from a submission queue determined to comprise one of;

read commands and write commands.

6. The method of claim 1, further comprising determining the state of the one or more resources of the data storage device.

7. The method of claim 6, wherein determining the state of the one or more resources of the data storage device comprises monitoring one or more of:

a read pipeline of the data storage device,

a write pipeline of the data storage device, and

a memory page being filled by the data storage device.

8. The method of claim 1, wherein selecting the submission queue further comprises:

determining one or more characteristics of storage commands likely queued within the respective submission queues based on the command statistics pertaining to the storage device commands previously fetched from the respective submission queues, and

comparing the determined characteristics to the state of the one or more resources of the data storage device.

9. The method of claim 1, wherein selecting the submission queue further comprises:

using the command statistics pertaining to the storage device commands previously fetched from the respective submission queues to identify submission queues likely to comprise one of;

read commands and write commands;

selecting a submission queue identified as being likely to comprise read commands in response to determining that the data storage device is currently unable to process write commands based on the state of the one or more resources of the data storage device; and

selecting a submission queue identified as being likely to comprise write commands in response to determining that the data storage device is currently unable to process read commands based on the state of the one or more resources of the data storage device.

10. The method of claim 1, wherein collecting the command statistics comprises continually collecting the command statistics pertaining to storage device commands being fetched from the respective submission queues.

11. The method of claim 1, further comprising:

initially operating in a static mode for selecting the submission queues from which to fetch storage device commands; and

switching to using the command statistics for selecting the submission queues from which to fetch storage device commands in response to a predetermined trigger.

12. The method of claim 1, wherein the data storage device comprises a nonvolatile memory device.

13. The method of claim 12, wherein the data storage device comprises a nonvolatile memory controller.

14. The method of claim 1, wherein the memory comprises one or more of:

two dimensional NAND flash memory, and three dimensional NAND flash memory.

15. A data storage device, comprising:

a memory;

a resource monitor for monitoring a resource state of the data storage device;

a command monitor for collecting statistics pertaining to device storage commands previously fetched from respective submission queues of a plurality of submission queues, wherein collecting the command statistics comprises collecting information regarding whether particular submission queues historically contain a majority of random input/output (I/O) commands or a majority of sequential I/O commands; and

a command fetcher for fetching storage device commands from the submission queues and providing the fetched storage device commands to command processing logic, wherein fetching a storage device command comprises;

selecting one of the plurality of submission queues from which to fetch the storage device command based on the resource state of the data storage device and the collected statistics pertaining to the device storage commands previously fetched from the respective submission queues, and

fetching the storage device command from the selected submission queue.

16. The data storage device of claim 15, wherein the command monitor is configured to collect statistics regarding one or more of a type of the storage device commands previously fetched from the respective submission queues and a size of the storage device commands previously fetched from the respective submission queues.

17. The data storage device of claim 15, wherein the command monitor is further configured to collect an indication of a number of storage device commands previously fetched from the respective submission queues.

18. The data storage device of claim 15, wherein the command monitor is configured to collect statistics regarding a ratio of read commands to write commands of the storage device commands previously fetched from the respective submission queues.

19. The data storage device of claim 15, wherein the collected statistics identify submission queues that have historically contained a majority of one of:

read commands and write commands.

20. The data storage device of claim 15, wherein the resource monitor is configured to monitor one or more of:

a read pipeline state of the data storage device, a write pipeline state of the data storage device, and a page being filled by the data storage device.

21. The data storage device of claim 15, wherein the collected statistics identify submission queues likely to contain read commands and submission queues likely to contain write commands based on the storage device commands previously fetched from the respective submission queues.

22. The data storage device of claim 21 wherein in response to the resource state of the data storage device indicating that a read pipeline of the data storage device is currently unavailable to process read commands, the command fetcher is configured to select the submission queue from the submission queues identified as being likely to contain write commands, bypassing selection from the submission queues identified as being likely to contain read commands.

23. The data storage device of claim 21, wherein in response to the resource state of the data storage device indicating that a write pipeline of the data storage device is currently unavailable to process write commands, the command fetcher is configured to select the submission queue from the submission queues identified as being likely to contain read commands, bypassing selection from the submission queues identified as being likely to contain write commands.

24. The data storage device of claim 15, wherein:

the collected statistics indicate submission queues, of the plurality of submission queues, that are likely to contain one of;

random input/output (I/O) commands and sequential I/O commands; and

the command fetcher is configured to select the submission queue from the submission queues indicated as being likely to contain random I/O commands, bypassing selection from the submission queues indicated as being likely to contain sequential I/O commands, in response to the monitored resource state indicating that the data storage device is filling a memory page with data corresponding to random I/O commands.

25. The data storage device of claim 15, wherein the command fetcher is configured to switch from a static submission queue selection mode to a dynamic selection mode based on the resource state of the data storage device and the collected statistics pertaining to the storage device commands previously fetched from the respective submission queues in response to a predetermined trigger.

26. The data storage device of claim 15, further comprising a controller, wherein the command monitor, the resource monitor, and the command fetcher are components of the controller.

27. The data storage device of claim 15, wherein the memory comprises a nonvolatile memory, and the data storage device further comprises a memory controller.

28. The data storage device of claim 27, wherein the memory controller comprises a nonvolatile memory controller.

29. The data storage device of claim 28, wherein the nonvolatile memory controller comprises a nonvolatile memory express (NVMe) controller.

30. The data storage device of claim 15, wherein the memory comprises one or more of:

two dimensional NAND flash memory and three dimensional NAND flash memory.

31. A non-transitory computer readable medium having stored thereon instructions to configure a processor of a computing device to perform operations, comprising:

collecting submission queue command statistics pertaining to commands previously fetched from respective submission queues of a plurality of submission queues, the command statistics indicating whether particular submission queues historically contain a majority of random input/output (I/O) commands or a majority of sequential I/O commands;

monitoring a resource state of a data storage device including a memory and a controller; and

fetching commands from the submission queues, wherein fetching a command comprises;

using the submission queue command statistics pertaining to the commands previously fetched from the respective submission queues and the resource state to select a submission queue from which to fetch the command,

fetching the command from the selected submission queue, and

providing the command to command processing logic.