Adaptive caching replacement manager with dynamic updating granulates and partitions for shared flash-based storage system

• US 10,467,195 B2
• Filed: 01/06/2017
• Issued: 11/05/2019
• Est. Priority Date: 09/06/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yang, Zhengyu, Evans, Thomas David, Wang, Jiayin
• Primary Examiner(s): Cao, Diem K
• Application Number:
  US15/400,835
• Publication Number:
  US 20180067961A1
• Time in Patent Office: 1,033 Days
• Field of Search: None
• CPC Class Codes: G06F 12/0808 with cache invalidating means G06F12/0815 takes precedence
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0842 for multiprocessing or multitasking
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/124 being minimized, e.g. non MRU
• CPC Class Codes: G06F 12/126 with special data handling, e.g. priority of data or instructions, handling errors or pinning
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 2212/1021 Hit rate improvement
• CPC Class Codes: G06F 2212/602 Details relating to cache prefetching
• CPC Class Codes: G06F 2212/6024 History based prefetching
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0634 by changing the state or mode of one or more devices
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 9/45558 : Hypervisor-specific management and integration aspects

Abstract

A method of adjusting temporal and spatial granularities associated with operation of a virtualized file system, the method including analyzing past workloads of a plurality of virtual machines associated with the virtualized file system, and adjusting the temporal and spatial granularities to be similar to average re-access temporal and spatial distances of data sets corresponding to the past workloads.

Claims

1. A method of improving disk allocation in a virtualized file system by adjusting temporal and spatial granularities associated with operation of the virtualized file system, the method comprising:

analyzing past workloads of a plurality of virtual machines associated with the virtualized file system;

adjusting the temporal and spatial granularities to be similar to average re-access temporal and spatial distances of data sets stored in the virtualized file system and corresponding to the past workloads;

conducting load balancing, wear leveling, disk health monitoring, and disk retirement/replacement on an array of fast drives based on the past workloads of the virtual machines respectively corresponding to the array of fast drives;

separating the virtual machines into subsets based on respective wear levels of the fast drives predicted from the past workloads; and

assigning the subsets of the virtual machines across the fast drives of the array of fast drives to balance wear of the fast drives.

2. The method of claim 1, further comprising adjusting partition sizes inside a fast drive of the virtualized file system for each virtual machine disk of the virtual machines.

3. The method of claim 2, further comprising detecting IO changes of each virtual machine disk, and wherein adjusting partition sizes is based on the detected IO changes.

4. The method of claim 1, wherein adjusting the temporal and spatial granularities comprises adjusting a time interval of a workload monitor sliding window for updating content of a fast drive used as a cache of the virtualized file system.

5. The method of claim 4, wherein the analyzing past workloads of the plurality of virtual machines occurs at an end of the workload monitor sliding window.

6. The method of claim 1, wherein adjusting the temporal and spatial granularities comprises adjusting a prefetching bin size corresponding to an amount of data retrieved from a fast drive or a slow drive of the virtualized file system.

7. The method of claim 1, wherein adjusting the temporal and spatial granularities comprises adjusting a cache size or an epoch updating frequency of each of the virtual machines based on the past workloads.

8. The method of claim 1, further comprising updating a fast drive as a cache of the virtualized file system by prefetching data in a most recent epoch based on the adjusted temporal and spatial granularities.

9. A virtualized file system capable of improved disk allocation, the virtualized file system comprising:

a plurality of virtual machines;

one or more slow drives;

a fast drive as a cache for the one or more slow drives;

a memory; and

a processor coupled to the memory, the processor executing a software component that is configured to;

analyze past workloads of a plurality of virtual machines associated with the virtualized file system;

adjust temporal and spatial granularities to be similar to average re-access temporal and spatial distances of data sets stored in the virtualized file system and corresponding to the past workloads;

conduct load balancing, wear leveling, disk health monitoring, and disk retirement/replacement on an array of fast drives based on the past workloads of the virtual machines respectively corresponding to the array of fast drives;

separating the virtual machines into subsets based on respective wear levels of the fast drives predicted from the past workloads; and

assigning the subsets of the virtual machines across the fast drives of the array of fast drives to balance wear of the fast drives.

10. The virtualized file system of claim 9, wherein the software component is further configured to adjust partition sizes inside the fast drive for each virtual machine disk of the virtual machines.

11. The virtualized file system of claim 10, wherein the software component is further configured to detect IO changes of each virtual machine disk, and wherein the software component is configured to adjust the partition sizes based on detected IO changes.

12. The virtualized file system of claim 9, wherein the software component is configured to adjust the temporal and spatial granularities by adjusting a time interval of a workload monitor sliding window for updating content of a fast drive used as a cache of the virtualized file system.

13. The virtualized file system of claim 9, wherein the software component is configured to adjust the temporal and spatial granularities by adjusting a prefetching bin size corresponding to an amount of data retrieved from a fast drive or a slow drive of the virtualized file system.

14. The virtualized file system of claim 9, wherein the software component is configured to adjust the temporal and spatial granularities by adjusting a cache size or an epoch updating frequency of each of the virtual machines based on the past workloads.

15. The virtualized file system of claim 9, further comprising updating a fast drive as a cache of the virtualized file system by prefetching data in a most recent epoch based on the adjusted temporal and spatial granularities.

16. A method of improving disk allocation in a virtualized file system by adjusting temporal and spatial granularities associated with operation of the virtualized file system, the method comprising:

determining whether an end of a workload monitor sliding window corresponding to a plurality of virtual machines of the virtualized file system is reached;

updating one or more of a prefetching granularity, a cache size, and an update frequency of a content update epoch sliding window for each of the virtual machines;

determining whether an end of the content update epoch sliding window of a corresponding one of the virtual machines is reached;

updating content of the corresponding one of the virtual machines in response to a determination that the end of the content update epoch sliding window is reached;

conducting load balancing, wear leveling, disk health monitoring, and disk retirement/replacement on an array of fast drives based on past workloads of the virtual machines;

separating the virtual machines into subsets based on respective wear levels of the fast drives predicted from the past workloads; and

assigning the subsets of the virtual machines across the fast drives of the array of fast drives to balance wear of the fast drives.