MANAGING THE WRITE PERFORMANCE OF AN ASYMMETRIC MEMORY SYSTEM

• US 20140281133A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: Virident Systems, LLC (Western Digital Corporation)
• Inventors: Karamcheti, Vijay, Singhai, Ashish, Kumar, Ajith, Mondal, Shibabrata
• Granted Patent: US 10,073,626 B2
• Time in Patent Office: Days
• US Class Current: 711/103
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 2212/7206 Reconfiguration of flash memory system
• CPC Class Codes: G06F 2212/7208 Multiple device management, e.g. distributing data over multiple flash devices
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0655 Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0679 Non-volatile semiconductor memory device, e.g. flash memory, one time programmable memory [OTP]
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5016 the resource being the memory

Abstract

Some implementations include a method of managing a hosted non-volatile random-access memory (NVRAM) based storage subsystem that includes NVRAM devices. The method includes: receiving, at a device driver on the host computing device, write requests each requesting to write a respective unit of data to the NVRAM-based storage subsystem; categorizing the write requests into subgroups of write requests, where write requests within respective subgroups are mutually exclusive; ascertaining a load condition of each of several of the NVRAM devices of the NVRAM-based storage subsystem; identifying a target location on at least one NVRAM device to service a particular subgroup of write requests according to the ascertained load conditions of the NVRAM devices of the NVRAM-based storage subsystem; and servicing the particular subgroup of write requests by writing the corresponding units of data to the identified target location on the at least one NVRAM device of the NVRAM-based storage subsystem.

Claims

1-30. -30. (canceled)

31. A method of managing a hosted non-volatile random-access memory (NVRAM) based storage subsystem that includes NVRAM devices, the method comprising:

receiving, at a device driver on the host computing device, write requests each requesting to write a respective unit of data to the NVRAM-based storage subsystem;

categorizing the write requests into subgroups of write requests, the write requests within the subgroups being mutually exclusive; and

servicing the particular subgroup of write requests by writing data to the at least one NVRAM device of the NVRAM-based storage subsystem (a) the respective unit of data of a first write request categorized in one of the subgroup of write requests at a first target location and (b) the respective unit of data of a second write request categorized in one other of the subgroup of write requests at a second target location,

wherein the first and second target locations are located on the at least one NVRAM device of the NVRAM-based storage subsystem and are separated by a distance of memory locations within which data would otherwise be stored prior to writing the respective units of data of the first and second write requests.

32. The method of claim 31, wherein servicing the particular subgroup of write requests further comprises:

(a) projecting the corresponding unit of data of the write requests categorized within a first subgroup of write requests to have a first distribution of validity durations after being written to the at least one NVRAM of the NVRAM-based storage subsystem; and

(b) projecting the corresponding units of data of the write requests categorized within a second subgroup of write requests to have a second distribution of validity durations after being written to the at least one NVRAM of the NVRAM-based storage subsystem,

wherein the first distribution and the second distribution differ from each other statistically.

33. The method of claim 31, wherein categorizing the write requests into subgroups of write requests comprises:

(a) generating a first subgroup of write requests with a first size distribution of the respective units of data; and

(b) generating a second subgroup of write requests with a second size distribution of the respective units of data,

wherein the first size distribution and the second size distribution differ from each other statistically.

34. The method of claim 31, wherein categorizing the write requests into subgroups of write requests comprises:

(a) generating a first subgroup of write requests received at the device driver at a first arrival time distribution;

(b) generating a second subgroup of write requests received at the device driver at a second arrival time distribution,

wherein the first arrival time distribution and the second arrival time distribution differ from each other statistically.

35. The method of claim 31, wherein categorizing the write requests into subgroups of write requests comprises generating a particular subgroup of write requests that are from one group of threads associated with a particular application on a computing device hosting the NVRAM-based storage system.

36. The method of claim 31, wherein categorizing the write requests into subgroups of write requests comprise categorizing the write requests into a first subgroup of write requests and a second subgroup of write requests, and

wherein the write requests in the first subgroup request to write respective units of data that are faster changing than respective units of data of write requests in the second subgroup.

37. The method of claim 36, further comprising:

improving a write amplification overhead caused by writing the respective units of data of the write requests to the NVRAM-based storage subsystem.

38. The method of claim 37, wherein improving the write amplification overhead comprises reducing, by categorizing the received write requests, an aggregate amount of data that has been relocated, and

wherein data is relocated to release at least one location on the at least one NVRAM device in order for the NVRAM-based storage subsystem to service at least one more write request arriving after the write requests have been received.

39. The method of claim 31, further comprising:

choosing, from a pool of candidate erase blocks located on the at least one NVRAM device of the NVRAM-based storage subsystem, an active erase block that comprises more than one write page to accommodate a respective unit of data of a particular write request.

40. The method of claim 39, wherein choosing the active erase block is based on an amount of valid data on each of the candidate erase blocks.

41. The method of claim 39, wherein choosing the active erase block is based on a load distribution across the candidate erase blocks.

42. The method of claim 39, wherein choosing the active erase block is based on an expected lifetime of the respective unit of data of the particular write request.

43. The method of claim 39, further comprising tracking an erase count for each of the candidate erase blocks in the pool of candidate erase blocks, wherein the erase count indicates a number of times the erase block has undergone an erase operation.

44. The method of claim 43, further comprising:

based on the erase count of a candidate erase block indicating that the candidate erase block has underwent less than 50% of an average erase operations of the pool of candidate erase blocks, choosing the erase block to accommodate the respective write page of data.

45. The method of claim 39, further comprising tracking a utilization of a hardware resource associated with each candidate erase block, wherein choosing the active erase block causes respective utilizations of all candidate erase blocks to become less divergent from each other.

46. A system comprising a non-volatile random-access memory (NVRAM) based storage subsystem, the NVRAM-based storage subsystem comprising at least one NVRAM device in communication with a host computing device, the NVRAM-based storage subsystem is configured to perform the operations of:

receiving, at a device driver on the host computing device, write requests each requesting to write a respective unit of data to the NVRAM-based storage subsystem;

categorizing the write requests into subgroups of write requests, the write requests within the subgroups being mutually exclusive; and

servicing the particular subgroup of write requests by writing data to the at least one NVRAM device of the NVRAM-based storage subsystem (a) the respective unit of data of a first write request categorized in one of the subgroup of write requests at a first target location and (b) the respective unit of data of a second write request categorized in one other of the subgroup of write requests at a second target location,

wherein the first and second target locations are located on the at least one NVRAM device of the NVRAM-based storage subsystem and are separated by a distance of memory locations within which data would otherwise be stored prior to writing the respective units of data of the first and second write requests.

47. The system of claim 46, wherein servicing the particular subgroup of write requests further comprises:

(a) projecting the corresponding unit of data of the write requests categorized within a first subgroup of write requests to have a first distribution of validity durations after being written to the at least one NVRAM of the NVRAM-based storage subsystem; and

(b) projecting the corresponding units of data of the write requests categorized within a second subgroup of write requests to have a second distribution of validity durations after being written to the at least one NVRAM of the NVRAM-based storage subsystem,

wherein the first distribution and the second distribution differ from each other statistically.

48. The system of claim 46, wherein categorizing the write requests into subgroups of write requests comprises:

(a) generating a first subgroup of write requests with a first size distribution of the respective units of data; and

(b) generating a second subgroup of write requests with a second size distribution of the respective units of data,

wherein the first size distribution and the second size distribution differ from each other statistically.

49. The system of claim 46, wherein categorizing the write requests into subgroups of write requests comprises:

(a) generating a first subgroup of write requests received at the device driver at a first arrival time distribution;

(b) generating a second subgroup of write requests received at the device driver at a second arrival time distribution,

wherein the first arrival time distribution and the second arrival time distribution differ from each other statistically.

50. The system of claim 46, wherein categorizing the write requests into subgroups of write requests comprises generating a particular subgroup of write requests that are from one group of threads associated with a particular application on a computing device hosting the NVRAM-based storage system.

51. The system of claim 46, wherein categorizing the write requests into subgroups of write requests comprise categorizing the write requests into a first subgroup of write requests and a second subgroup of write requests, and

wherein the write requests in the first subgroup request to write respective units of data that are faster changing than respective units of data of write requests in the second subgroup.

52. The system of claim 51, wherein the operations further comprise:

improving a write amplification overhead caused by writing the respective units of data of the write requests to the NVRAM-based storage subsystem.

53. The system of claim 52, wherein improving the write amplification overhead comprises reducing, by categorizing the received write requests, an aggregate amount of data that has been relocated, and

wherein data is relocated to release at least one location on the at least one NVRAM device in order for the NVRAM-based storage subsystem to service at least one more write request arriving after the write requests have been received.

54. The system of claim 46, further comprising:

choosing, from a pool of candidate erase blocks located on the at least one NVRAM device of the NVRAM-based storage subsystem, an active erase block that comprises more than one write page to accommodate a respective unit of data of a particular write request.

55. The system of claim 54, wherein choosing the active erase block is based on an amount of valid data on each of the candidate erase blocks.

56. The system of claim 54, wherein choosing the active erase block is based on a load distribution across the candidate erase blocks.

57. The system of claim 54, wherein choosing the active erase block is based on an expected lifetime of the respective unit of data of the particular write request.

58. The system of claim 54, wherein the operations further comprise:

tracking an erase count for each of the candidate erase blocks in the pool of candidate erase blocks, wherein the erase count indicates a number of times the erase block has undergone an erase operation.

59. The system of claim 58, wherein the operations further comprise:

based on the erase count of a candidate erase block indicating that the candidate erase block has underwent less than 50% of an average erase operations of the pool of candidate erase blocks, choosing the erase block to accommodate the respective write page of data.

60. The system of claim 54, wherein the operations further comprise:

tracking a utilization of a hardware resource associated with each candidate erase block, wherein choosing the active erase block causes respective utilizations of all candidate erase blocks to become less divergent from each other.

61. A computer readable medium, comprising software instructions, which software instructions when executed causes a non-volatile random-access memory (NVRAM) based storage subsystem, which NVRAM based storage subsystem comprising at least one NVRAM device in communication with a host computing device, to perform the operations of:

receiving, at a device driver on the host computing device, write requests each requesting to write a respective unit of data to the NVRAM-based storage subsystem;

categorizing the write requests into subgroups of write requests, the write requests within the subgroups being mutually exclusive; and

servicing the particular subgroup of write requests by writing data to the at least one NVRAM device of the NVRAM-based storage subsystem (a) the respective unit of data of a first write request categorized in one of the subgroup of write requests at a first target location and (b) the respective unit of data of a second write request categorized in one other of the subgroup of write requests at a second target location,

wherein the first and second target locations are located on the at least one NVRAM device of the NVRAM-based storage subsystem and are separated by a distance of memory locations within which data would otherwise be stored prior to writing the respective units of data of the first and second write requests.