Modified consistency hashing rings for object store controlled WAN cache infrastructure

• US 10,592,415 B2
• Filed: 12/01/2017
• Issued: 03/17/2020
• Est. Priority Date: 12/01/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Baldwin, Duane, Eda, Sasikanth, Olson, John T., Patil, Sandeep R.
• Primary Examiner(s): Dudek, Jr., Edward J
• Assistant Examiner(s): Li, Sidney
• Application Number:
  US15/829,577
• Publication Number:
  US 20190171570A1
• Time in Patent Office: 837 Days
• Field of Search: 711137
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 12/0895 of parts of caches, e.g. directory or tag array
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 2212/1021 Hit rate improvement
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/6026 Prefetching based on access pattern detection, e.g. stride based prefetch
• CPC Class Codes: H04L 12/2854 Wide area networks, e.g. public data networks
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/5681 Pre-fetching or pre-delivering data based on network characteristics

Abstract

An embodiment of the invention may include a method, computer program product and system for optimizing a wide area network caching infrastructure in a file based object storage architecture. The embodiment may include creating, by a parent partition, a heat map. The embodiment may include prioritizing prefetching by multiple dependent partitions based on the heat map. In response to prioritized prefetching by the multiple dependent partitions, the embodiment may include allocating wide area network caching threads. The embodiment may include providing, by the parent partition, objects for prefetching by the multiple dependent partitions utilizing the allocated wide area network caching threads.

Claims

1. A method for optimizing a wide area network caching infrastructure in a file based object storage architecture, the method comprising:

creating, by a parent partition, a heat map, wherein creating the heat map comprises determining a relationship between each partition of the file based object storage architecture based on a pattern of object recall per site per partition, and wherein each partition is wide area network caching enabled, and wherein the determined relationships identify the parent partition and the multiple dependent partitions;

prioritizing prefetching by multiple dependent partitions based on the heat map;

in response to prioritized prefetching by the multiple dependent partitions, allocating wide area network caching threads; and

providing, by the parent partition, objects for prefetching by the multiple dependent partitions utilizing the allocated wide area network caching threads.

2. The method of claim 1, wherein creating, by the parent partition, the heat map comprises monitoring, by the parent partition, of the multiple dependent partitions.

3. The method of claim 2, wherein monitoring, by the parent partition, of the multiple dependent partitions comprises:

logging, by the parent partition, a number of objects within each dependent partition;

logging, by the parent partition, a number of times access is requested for each object; and

preparing, by the parent partition, a list of objects in order of descending number of access requests, wherein the list of objects also specifies the dependent partition containing each object.

4. The method of claim 2, wherein a first dependent partition, containing a first object with a number of access requests greater than a second object contained in a second dependent partition, is given a priority over the second dependent partition, and wherein a number of threads working per dependent partition is dependent on the priority given to the dependent partition.

5. The method of claim 1, wherein allocating the wide area network caching threads comprises an element from the group consisting of increasing the number of threads working per partition and decreasing the number of threads working per partition.

6. The method of claim 1, further comprising:

monitoring individual partitions of the file based object storage architecture, the monitoring comprises;

enabling peer-to-peer communication;

identifying partitions that contain a maximum number of changes; and

preparing a list of partitions in order of descending number of changes.

7. A computer program product for optimizing a wide area network caching infrastructure in a file based object storage architecture, the computer program product comprising:

one or more computer-readable tangible storage devices and program instructions stored on at least one of the one or more computer-readable tangible storage devices, the program instructions being executable by a computer, the program instructions comprising;

program instructions to create, by a parent partition, a heat map, wherein program instructions to create the heat map comprise determining a relationship between each partition of the file based object storage architecture based on a pattern of object recall per site per partition, and wherein each partition is wide area network caching enabled, and wherein the determined relationships identify the parent partition and the multiple dependent partitions;

program instructions to prioritize prefetching by multiple dependent partitions based on the heat map;

in response to prioritized prefetching by the multiple dependent partitions, program instructions to allocate wide area network caching threads; and

program instructions to provide, by the parent partition, objects for prefetching by the multiple dependent partitions utilizing the allocated wide area network caching threads.

8. The computer program product of claim 7, wherein program instructions to create, by the parent partition, the heat map comprise program instructions to monitor, by the parent partition, the multiple dependent partitions.

9. The computer program product of claim 8, wherein program instructions to monitor, by the parent partition, the multiple dependent partitions comprises:

program instructions to log, by the parent partition, a number of objects within each dependent partition;

program instructions to log, by the parent partition, a number of times access is requested for each object; and

program instructions to prepare, by the parent partition, a list of objects in order of descending number of access requests, wherein the list of objects also specifies the dependent partition containing each object.

10. The computer program product of claim 8, wherein a first dependent partition, containing a first object with a number of access requests greater than a second object contained in a second dependent partition, is given a priority over the second dependent partition, and wherein a number of threads working per dependent partition is dependent on the priority given to the dependent partition.

11. The computer program product of claim 7, wherein program instructions to allocate the wide area network caching threads comprise an element from the group consisting of increasing the number of threads working per partition and decreasing the number of threads working per partition.

12. The computer program product of claim 7, further comprising:

program instructions to monitor individual partitions of the file based object storage architecture, the program instructions to monitor comprise;

program instructions to enable peer-to-peer communication;

program instructions to identify partitions that contain a maximum number of changes; and

program instructions to prepare a list of partitions in order of descending number of changes.

13. A computer system for optimizing a wide area network caching infrastructure in a file based object storage architecture, the computer system comprising:

one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more computer-readable tangible storage devices for execution by at least one of the one or more processors via at least one of the one or more computer-readable memories, the program instructions comprising;

program instructions to create, by a parent partition, a heat map, wherein program instructions to create the heat map comprise determining a relationship between each partition of the file based object storage architecture based on a pattern of object recall per site per partition, and wherein each partition is wide area network caching enabled, and wherein the determined relationships identify the parent partition and the multiple dependent partitions;

program instructions to prioritize prefetching by multiple dependent partitions based on the heat map;

in response to prioritized prefetching by the multiple dependent partitions, program instructions to allocate wide area network caching threads; and

program instructions to provide, by the parent partition, objects for prefetching by the multiple dependent partitions utilizing the allocated wide area network caching threads.

14. The computer system of claim 13, wherein program instructions to create, by the parent partition, the heat map comprise program instructions to monitor, by the parent partition, the multiple dependent partitions.

15. The computer system of claim 14, wherein program instructions to monitor, by the parent partition, the multiple dependent partitions comprises:

program instructions to log, by the parent partition, a number of objects within each dependent partition;

program instructions to log, by the parent partition, a number of times access is requested for each object; and

program instructions to prepare, by the parent partition, a list of objects in order of descending number of access requests, wherein the list of objects also specifies the dependent partition containing each object.

16. The computer system of claim 14, wherein a first dependent partition, containing a first object with a number of access requests greater than a second object contained in a second dependent partition, is given a priority over the second dependent partition, and wherein a number of threads working per dependent partition is dependent on the priority given to the dependent partition.

17. The computer system of claim 13, further comprising:

program instructions to monitor individual partitions of the file based object storage architecture, the program instructions to monitor comprise;

program instructions to enable peer-to-peer communication;

program instructions to identify partitions that contain a maximum number of changes; and

program instructions to prepare a list of partitions in order of descending number of changes.