Method and apparatus for using cache size estimations for guiding hot-tier insertion decisions

• US 10,698,823 B2
• Filed: 04/27/2018
• Issued: 06/30/2020
• Est. Priority Date: 04/27/2018
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Allen, Cyril Anthony, Nijhawan, Aman, Wyckoff, Peter Scott, Faith, Rickard Edward
• Primary Examiner(s): Yi, David
• Assistant Examiner(s): Fulford, Dustin B.
• Application Number:
  US15/965,721
• Publication Number:
  US 20190332536A1
• Time in Patent Office: 795 Days
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 2212/1021 Hit rate improvement
• CPC Class Codes: G06F 2212/1024 Latency reduction
• CPC Class Codes: G06F 2212/152 Virtualized environment, e.g. logically partitioned system
• CPC Class Codes: G06F 2212/608 Details relating to cache mapping
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays

Abstract

A method and apparatus for using cache size estimations for guiding hot-tier insertion decisions. The method and apparatus include an adaptive management element that determines what accesses of a resource should be logged and the parameters for logging. The determinations are used to configure an adaptive logging element to log only accesses corresponding to the selected resource(s) and to log only those accesses that match the identified parameters. The adaptive management element operates in a feedback loop: first determining what will be logged and second implementing that determination by an adaptive logging element. Upon a triggering event, the process returns to the first determination based on any then current parameters. In some embodiments, the parameters include a size estimate to achieve a given target hit rate (target hit rate size estimate) that is used in generating a score or weighting to identify the highest/best scoring/weighted disk(s) for logging.

Claims

1. A method, comprising:

receiving a respective size estimate and a respective current hit rate for each vDisk of a plurality of vDisks, the plurality of vDisks being accessible by a controller on a computing node, and the respective size estimate for each vDisk corresponding to a given hit rate change;

selectively controlling logging of access requests to individual vDisks of the plurality of vDisks based on at least the respective size estimates and the respective current hit rates for the plurality of vDisks to manage promotion of data to a hot storage tier from one or more of the plurality of vDisks, the hot storage tier being shared by the plurality of vDisks, the computing node controlling logging of access requests by;

generating a respective rank for each vDisk of the plurality of vDisks based on at least the respective size estimates and the respective current hit rates for the plurality of vDisks;

selecting a subset of the plurality of vDisks based on the respective ranks for the plurality of vDisks;

generating an entry in a log for an access request corresponding to a selected vDisk, without generating an entry in the log for a vDisk that was not selected for the subset of the plurality of vDisks;

analyzing the log to identify data to be promoted to the hot storage tier from the subset of the plurality of vDisks; and

promoting the identified data to the hot storage tier.

2. The method of claim 1, wherein the identified data is promoted from a lower storage tier to the hot storage tier, the identified data is identified for promotion to the hot storage tier based on meeting or exceeding a threshold, and the threshold comprises a minimum number of accesses in a sliding time window.

3. The method of claim 1, further comprising processing an updated respective size estimate for a second vDisk of the plurality of vDisks, selecting the second vDisk, and generating an entry in the log for an access request to the second vDisk.

4. The method of claim 1, wherein generating the respective rank for each vDisk of the plurality of vDisks is further based on cluster data corresponding to a plurality of nodes, the cluster data comprising storage pool metadata.

5. The method of claim 1, further comprising processing cluster data or node data to determine a parameter for logging, wherein only access requests that satisfy the parameter are logged in the log.

6. The method of claim 1, wherein the respective size estimates comprise an estimate of an amount of storage required to be moved to the hot storage tier to cause a given hit rate increase for each vDisk of the plurality of vDisks.

7. The method of claim 1, wherein the respective size estimates comprise values normalized using an amount of space occupied by respective vDisks of the plurality of vDisks.

8. The method of claim 1, wherein the computing node is in a cluster of computing nodes and shares the hot storage tier with other nodes in the cluster of computing nodes.

9. The method of claim 1, wherein the plurality of vDisks comprises only vDisks for which a corresponding size estimate was received.

10. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes a process comprising:

receiving a respective size estimate and a respective current hit rate for each vDisk of a plurality of vDisks, the plurality of vDisks being accessible by a controller on a computing node, and the respective size estimate for each vDisk corresponding to a given hit rate change;

selectively controlling logging of access requests to individual vDisks of the plurality of vDisks based on at least the respective size estimates and the respective current hit rates for the plurality of vDisks to manage promotion of data to a hot storage tier from one or more of the plurality of vDisks, the hot storage tier being shared by the plurality of vDisks, the computing node controlling logging of access requests by;

generating a respective rank for each vDisk of the plurality of vDisks based on at least the respective size estimates and the respective current hit rates for the plurality of vDisks;

selecting a subset of the plurality of vDisks based on the respective ranks for the plurality of vDisks;

generating an entry in a log for an access request corresponding to a selected vDisk, without generating an entry in the log for a vDisk that was not selected for the subset of the plurality of vDisks;

analyzing the log to identify data to be promoted to the hot storage tier from the subset of the plurality of vDisks; and

promoting the identified data to the hot storage tier.

11. The computer readable medium of claim 10, wherein the identified data is promoted from a lower storage tier to the hot storage tier, the identified data is identified for promotion to the hot storage tier based on meeting or exceeding a threshold, and the threshold comprises a minimum number of accesses in a sliding time window.

12. The computer readable medium of claim 10, wherein the process further comprises processing an updated respective size estimate for a second vDisk of the plurality of vDisks, selecting the second vDisk, and generating an entry in the log for an access request to the second vDisk.

13. The computer readable medium of claim 10, wherein generating the respective rank for each vDisk of the plurality of vDisks is further based on cluster data corresponding to a plurality of nodes, the cluster data comprising storage pool metadata.

14. The computer readable medium of claim 10, wherein the process further comprises processing cluster data or node data to determine a parameter for logging, wherein only access requests that satisfy the parameter are logged in the log.

15. The computer readable medium of claim 10, wherein the plurality of vDisks comprises only vDisks for which a corresponding size estimate was received.

16. The computer readable medium of claim 10, wherein the respective size estimates comprise an estimate of an amount of storage required to be moved to the hot storage tier to cause a given hit rate increase for each vDisk of the plurality of vDisks.

17. The computer readable medium of claim 10, wherein the respective size estimates comprise values normalized using an amount of space occupied by respective vDisks of the plurality of vDisks.

18. A computing system for guiding hot-tier insertion decisions, comprising:

a memory for storing data and instructions; and

a processor that executes the instructions to enable actions, including;

receiving a respective size estimate and a respective current hit rate for each vDisk of a plurality of vDisks, the plurality of vDisks being accessible by a controller on a computing node, and the respective size estimate for each vDisk corresponding to a given hit rate change;

selectively controlling logging of access requests to individual vDisks of the plurality of vDisks based on at least the respective size estimates and the respective current hit rates for the plurality of vDisks to manage promotion of data to a hot storage tier from one or more of the plurality of vDisks, the hot storage tier being shared by the plurality of vDisks, the computing node controlling logging of access requests by;

generating a respective rank for each vDisk of the plurality of vDisks based on at least the respective size estimates and the respective current hit rates for the plurality of vDisks;

selecting a subset of the plurality of vDisks based on the respective ranks for the plurality of vDisks;

generating an entry in a log for an access request corresponding to a selected vDisk, without generating an entry in the log for a vDisk that was not selected for the subset of the plurality of vDisks;

analyzing the log to identify data to be promoted to the hot storage tier from the subset of the plurality of vDisks; and

promoting the identified data to the hot storage tier.

19. The computing system of claim 18, wherein the identified data is promoted from a lower storage tier to the hot storage tier, the identified data is identified for promotion to the hot storage tier based on meeting or exceeding a threshold, and the threshold comprises a minimum number of accesses in a sliding time window.

20. The computing system of claim 18, further comprising processing an updated respective size estimate for a second vDisk of the plurality of vDisks, selecting the second vDisk, and generating an entry in the log for an access request to the second vDisk.

21. The computing system of claim 18, wherein generating the respective rank for each vDisk of the plurality of vDisks is further based on cluster data corresponding to a plurality of nodes, the cluster data comprising storage pool metadata.

22. The computing system of claim 18, further comprising processing cluster data or node data to determine a parameter for logging, wherein only access requests that satisfy the parameter are logged in the log.

23. The computing system of claim 18, wherein the respective size estimates comprise an estimate of an amount of storage required to be moved to the hot storage tier to cause a given hit rate increase for each vDisk of the plurality of vDisks.

24. The computing system of claim 18, wherein the respective size estimates comprise values normalized using an amount of space occupied by respective vDisks of the plurality of vDisks.

25. The computing system of claim 18, wherein the plurality of vDisks comprises only vDisks for which a corresponding size estimate was received.