Managing I/O operations of large data objects in a cache memory device by dividing into chunks

• US 10,289,315 B2
• Filed: 03/27/2017
• Issued: 05/14/2019
• Est. Priority Date: 03/27/2017
• Status: Active Grant

• Current Assignee: Sandisk Technologies Incorporated (Western Digital Corporation)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Bhansali, Akhil, Mehta, Kirankumar, Keymolen, Bruno, Bhairy, Rajendra Kumar
• Primary Examiner(s): Schnee, Hal
• Application Number:
  US15/470,787
• Publication Number:
  US 20180275898A1
• Time in Patent Office: 778 Days
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 12/0875 with dedicated cache, e.g. instruction or stack
• CPC Class Codes: G06F 2212/1032 Reliability improvement, data loss prevention, degraded operation etc
• CPC Class Codes: G06F 2212/214 Solid state disk
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 2212/312 In storage controller
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06F 2212/455 Image or video data
• CPC Class Codes: G06F 2212/462 Track or segment
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Various aspects for managing input/output (I/O) operations for data objects (e.g., large data objects (LOBs)), in a storage system are provided. For instance, a method may include receiving an I/O request for executing an I/O operation on a data object, determining the size of the data object, and determining the data object to be a non-conforming data object relative to a storage threshold of a cache memory device. The method may further include dividing the non-conforming data object into multiple data chunks, each of the data chunks having a size that is less than the storage threshold, moving one or more data chunks stored in one or more memory spaces of the cache memory device to a long-term memory device to free up the one or more memory spaces, and storing one or more of the data chunks of the non-conforming data object in the freed-up memory space(s).

Claims

1. A non-volatile memory system, comprising:

a first non-volatile cache memory device including a storage threshold; and

a storage controller coupled to the first non-volatile cache memory device, the storage controller including one or more processing devices for executing instructions in one or more applications configured to cause the storage controller to perform operations comprising;

receiving an input/output (I/O) request for a data object;

determining the data object to be a non-conforming data object relative to the storage threshold of the first non-volatile cache memory device;

dividing the non-conforming data object into a plurality of data chunks, each of the data chunks having a size that is less than the storage threshold;

assigning a thread configured to execute on the storage controller exclusive access to the non-conforming data object, wherein the thread is configured to service I/O operations received from one or more clients for the plurality of data chunks;

periodically moving one or more data chunks to a long-term memory device at a predetermined time interval to free up one or more memory spaces of the first non-volatile cache memory device;

storing one or more data chunks of the non-conforming data object in the freed-up one or more memory spaces of the first non-volatile cache memory device; and

performing, using the thread assigned exclusive access to the non-conforming data object, I/O operations for the one or more data chunks of the non-conforming data object in the first non-volatile cache memory device.

2. The system of claim 1, wherein:

the I/O request is a write request; and

the storage controller, when performing the operations for storing the one or more data chunks of the non-conforming data object, further performs operations comprising writing each of the one or more data chunks of the non-conforming data object to the freed-up one or more memory spaces of the first non-volatile cache memory device.

3. The system of claim 1, wherein:

the I/O request is a read request; and

the storage controller, when performing the operations for storing the one or more data chunks of the non-conforming data object, further performs operations comprising reading one or more data chunks stored in the long-term memory device to the first non-volatile cache memory device.

4. The system of claim 1, wherein:

the I/O request is a read-write request; and

the storage controller, when performing the operations for storing the one or more data chunks of the non-conforming data object, further performs operations comprising;

reading one or more data chunks stored in the long-term memory device to the first non-volatile cache memory device;

modifying data in the one or more read data chunks to generate one or more modified data chunks; and

writing the one or more modified data chunks to the long-term memory device.

5. The system of claim 1, further comprising the long-term memory device coupled to the first non-volatile cache memory device, wherein:

determining the data object to be a non-conforming data object is further based on a size of the data object exceeding an available storage capacity of the first non-volatile cache memory device; and

storing the one or more data chunks of the non-conforming data object in the freed-up one or more memory spaces includes, for each data chunk of the non-conforming data object;

determining whether there is sufficient space for storing each data chunk in the first non-volatile cache memory device; and

when there is not sufficient space;

writing one or more existing data chunks that are associated with one or more data objects other than the non-conforming data object and currently stored in the first non-volatile cache memory device to the long-term memory device;

freeing up the one or more memory spaces of the one or more existing data chunks to generate the freed-up one or more memory spaces; and

writing each data chunk to the freed-up one or more memory spaces in the first non-volatile cache memory device.

6. The system of claim 1, wherein:

the first non-volatile cache memory device includes a buffer size; and

the storage threshold reflects the buffer size.

7. The system of claim 1, further comprising:

a second non-volatile cache memory device coupled to the first non-volatile cache memory device, wherein;

the storage controller further performs operations comprising replicating each data chunk stored in the first non-volatile cache memory device to the second non-volatile cache memory device.

8. The system of claim 1, wherein the long-term memory device is one of:

a portion of a local storage network; and

a portion of an enterprise storage service.

9. The system of claim 1, wherein the first non-volatile cache memory device comprises a flash memory device.

10. The system of claim 1, wherein:

the storage controller is configured to be coupled to a network that is in communication with one or more client devices; and

the storage controller is located between the one or more client devices and the first non-volatile cache memory device when coupled to the network.

11. A system, comprising:

a non-volatile cache memory device including a buffer size;

means for receiving an input/output (I/O) request for a data object;

means for determining the data object to be a non-conforming data object relative to the buffer size of the non-volatile cache memory device;

means for dividing the non-conforming data object into a plurality of data chunks, each of the data chunks having a size that is less than the buffer size;

means for assigning a thread configured to execute on the means for determining exclusive access to the non-conforming data object, wherein the thread is configured to service I/O operations received from one or more clients for the plurality of data chunks;

means for periodically moving one or more data chunks stored in one or more memory spaces of the non-volatile cache memory device to a long-term memory device at a predetermined time interval to free up one or more memory spaces of the non-volatile cache memory device;

means for storing one or more data chunks of the non-conforming data object in the freed-up one or more memory spaces of the non-volatile cache memory device; and

means for performing, using the thread assigned exclusive access to the non-conforming data object, I/O operations for the one or more data chunks of the non-conforming data object in the non-volatile cache memory device.

12. The system of claim 11, wherein:

the I/O request is a write request; and

the means for storing the one or more data chunks of the non-conforming data object further comprises means for writing each of the one or more data chunks of the non-conforming data object to the freed-up one or more memory spaces of the non-volatile cache memory device.

13. The system of claim 11, wherein:

the I/O request is a read request; and

the means for storing the one or more data chunks of the non-conforming data object further comprises means for reading one or more data chunks stored in the long-term memory device to the non-volatile cache memory device.

14. The system of claim 11, wherein:

the I/O request is a read-write request; and

the means for storing the one or more data chunks of the non-conforming data object further comprises;

means for reading one or more data chunks stored in the long-term memory device to the non-volatile cache memory device;

means for modifying data in the one or more read data chunks to generate one or more modified data chunks; and

means for writing the one or more modified data chunks to the long-term memory device.

15. The system of claim 11, further comprising the long-term memory device coupled to the non-volatile cache memory device, wherein:

the means for determining the data object to be a non-conforming data object comprises means for basing the determination on a size of the data object exceeding the buffer size; and

the means for storing the one or more data chunks of the non-conforming data object in the freed-up one or more memory spaces includes, for each data chunk of the non-conforming data object;

means for determining whether there is sufficient space for storing each data chunk in the non-volatile cache memory device; and

when there is not sufficient space;

means for writing one or more existing data chunks that are associated with one or more data objects other than the non-conforming data object and currently stored in the non-volatile cache memory device to the long-term memory device;

means for freeing up the one or more memory spaces of the one or more existing data chunks to generate the freed-up one or more memory spaces; and

means for writing each data chunk to the freed-up one or more memory spaces in the non-volatile cache memory device.

16. A method of operation in a non-volatile memory system including a set of non-volatile memory devices, the method comprising:

receiving, at a storage controller coupled to a non-volatile cache memory device including a storage threshold, an input/output (I/O) request for executing an I/O operation on a data object;

determining the data object to be a non-conforming data object relative to the storage threshold of the non-volatile cache memory device;

dividing the non-conforming data object into a plurality of data chunks, each of the data chunks having a size that is less than the storage threshold;

assigning a thread configured to execute on the storage controller exclusive access to the non-conforming data object, wherein the thread is configured to service I/O operations received from one or more clients for the plurality of data chunks;

periodically moving one or more data chunks stored in one or more memory spaces of the non-volatile cache memory device to a long-term memory device at a predetermined time interval to free up one or more memory spaces of the non-volatile cache memory device;

storing one or more data chunks of the non-conforming data object in the freed-up one or more memory spaces of the non-volatile cache memory device; and

performing, using the thread assigned exclusive access to the non-conforming data object, I/O operations for the one or more data chunks of the non-conforming data object in the non-volatile cache memory device.

17. The method of claim 16, wherein:

the I/O request is a write request; and

storing the one or more data chunks of the non-conforming data object further comprises writing each of the one or more data chunks of the non-conforming data object to the freed-up one or more memory spaces of the non-volatile cache memory device.

18. The method of claim 16, wherein:

the I/O request is a read request; and

storing the one or more data chunks of the non-conforming data object further comprises reading each of the one or more data chunks of the non-conforming data object to the freed-up one or more memory spaces of the non-volatile cache memory device.

19. The method of claim 16, wherein:

the I/O request is a read-write request; and

storing the one or more data chunks of the non-conforming data object further comprises;

reading each of the one or more data chunks of the non-conforming data object to the freed-up one or more memory spaces of the non-volatile cache memory device;

modifying data in each of the one or more read data chunks to generate one or more modified data chunks; and

writing each of the one or more modified data chunks to the long-term memory device.

20. The method of claim 16, wherein:

determining the data object to be a non-conforming data object is further based on a size of the data object exceeding an available storage capacity of the non-volatile cache memory device; and

storing the one or more data chunks of the non-conforming data object in the freed-up one or more memory spaces includes, for each data chunk of the non-conforming data object;

determining whether there is sufficient space for storing each data chunk in the non-volatile cache memory device; and

when there is not sufficient space;

writing one or more existing data chunks that are associated with one or more data objects other than the non-conforming data object and currently stored in the non-volatile cache memory device to the long-term memory device;

freeing up the one or more memory spaces of the one or more existing data chunks to generate the freed-up one or more memory spaces; and

writing each data chunk to the freed-up one or more memory spaces in the non-volatile cache memory device.