Hierarchical pre-fetch pipelining in a hybrid memory server

• US 9,952,774 B2
• Filed: 04/27/2016
• Issued: 04/24/2018
• Est. Priority Date: 06/24/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chan, Yuk Lung, Krishnamurthy, Rajaram B., Parris, Carl Joseph
• Primary Examiner(s): BILGRAMI, ASGHAR H
• Application Number:
  US15/139,391
• Publication Number:
  US 20160239424A1
• Time in Patent Office: 727 Days
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 2212/264 Remote server
• CPC Class Codes: G06F 2212/602 Details relating to cache prefetching
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 47/24 Traffic characterised by specific attributes, e.g. priority or QoS
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A method, hybrid server system, and computer program product, prefetch data. A set of prefetch requests associated with one or more given datasets residing on the server system are received from a set of accelerator systems. A set of data is prefetched from a memory system residing at the server system for at least one prefetch request in the set of prefetch requests. The set of data satisfies the at least one prefetch request. The set of data that has been prefetched is sent to at least one accelerator system, in the set of accelerator systems, associated with the at least one prefetch request.

Claims

1. A method, with a server system in an out-of-core processing environment, for prefetching data, the method comprising:

receiving, from at least one accelerator system, a set of prefetch requests, wherein the set of prefetch requests is associated with one or more given datasets residing on the server system prior to at least one application executing on at least one user client system requiring the one or more given datasets, and wherein at least two prefetch requests in the set of prefetch requests are each associated with a different option presented to a user within the application with respect to a current state of the application;

prefetching, for at least one prefetch request in the set of prefetch requests, a first set of data and at least a second set of data from a memory system residing at the server system, wherein the first set of data and the second set of data satisfy a first and second of the least two of the set of prefetch requests, respectively; and

sending the first and second sets of data that have been prefetched to at least one accelerator system, in the set of accelerator systems, associated with the at least two of the set of prefetch requests.

2. The method of claim 1, wherein the prefetching comprises:

storing the set of prefetch requests in a prefetch request queue residing within the memory system;

assigning a score to each prefetch request in the set of prefetch requests;

determining that at least each of the two of the set of prefetch requests comprises a score above a given threshold; and

prefetching at least the first and second sets of data in response to the determining that at least each of the two of the set of prefetch requests comprises a score above a given threshold.

3. The method of claim 1, wherein the set of prefetch requests are received from a plurality of n-way prefetchers residing at the set of accelerator systems.

4. The method of claim 1, further comprising:

storing, prior to sending the prefetched first and second sets of data to the at least one accelerator system, the prefetched first and second sets of data in a temporary storage area within the memory system assigned to the at least one accelerator system, wherein each accelerator system in the set of accelerator systems is assigned a temporary storage area within the memory system.

5. The method of claim 1, further comprising:

analyzing a set of resources available at the server system for processing at least one prefetch request in the set of prefetch requests;

determining, based on the analyzing, that the server system fails to comprise sufficient resources for processing the least one prefetch request; and

ignoring, in response to the determining, the at least one prefetch request.

6. The method of claim 1, further comprising:

identifying an availability of a set of resources at the server system for processing at least one prefetch request in the set of prefetch requests; and

storing, prior to sending the set of data, a prefetched set of data associated with the least one prefetch request in a slow memory of the memory system based on the availability of the set of resources that has been identified, wherein the memory system comprises a fast memory and the slow memory.

7. The method of claim 1, further comprising:

monitoring, by a prefetcher on at least one accelerator system in the set of accelerator systems, usage of each dataset in a plurality of datasets stored within a prefetch memory of the at least one accelerator system;

updating, based on the monitoring, a counter associated with each dataset in response to the each dataset having been used;

calculating a computing complexity for each of the datasets in the plurality of datasets, wherein the computing complexity is associated with processing the dataset;

assigning a score to each of the datasets based on at least one of the counter and computing complexity associated therewith; and

removing at least one dataset in the plurality of datasets from the prefetch memory based on the score that has been assigned thereto.

8. A hybrid server system in an out-of-core processing environment, the hybrid server system comprising:

a server comprising a memory system and at least a first prefetcher communicatively coupled to the memory system; and

a set of accelerator systems communicatively coupled to the server, wherein each accelerator system in the set of accelerator systems comprises at least one processing core and at least a second prefetcher,

wherein the first prefetcher is configured to;

receive a set of prefetch requests from the second prefetcher of at least one accelerator system in the set of accelerator systems, wherein the set of prefetch requests is associated with one or more given datasets residing on the server system prior to at least one application executing on at least one user client system requiring the one or more given datasets, and wherein at least two prefetch requests in the set of prefetch requests are each associated with a different option presented to a user within the application with respect to a current state of the application;

prefetch a first set of data and at least a second set of data from the memory system residing at the server system, wherein the first set of data and the second set of data satisfy a first and second of the least two of the set of prefetch requests, respectively; and

send the first and second sets of data that have been prefetched to at least one accelerator system, in the set of accelerator systems, associated with the at least two of the set of prefetch requests.

9. The hybrid server system of claim 8, wherein the first prefetcher is configured to prefetch by:

storing the set of prefetch requests in a prefetch request queue residing within the memory system;

assigning a score to each prefetch request in the set of prefetch requests;

determining that at least each of the two of the set of prefetch requests comprises a score above a given threshold; and

prefetching at least the first and second sets of data in response to the determining that at least each of the two of the set of prefetch requests comprises a score above a given threshold.

10. The hybrid server system of claim 8, wherein the second prefetcher is an n-way prefetcher.

11. The hybrid server system of claim 8, wherein the first prefetcher is further configured to:

store, prior to sending the prefetched first and second sets of data to the at least one accelerator system, the prefetched first and second sets of data in a temporary storage area within the memory system assigned to the at least one accelerator system, wherein each accelerator system in the set of accelerator systems is assigned a temporary storage area within the memory system.

12. The hybrid server system of claim 8, wherein the first prefetcher is further configured to:

analyze a set of resources available at the server system for processing at least one prefetch request in the set of prefetch requests;

determine, based on the analyzing, that the server system fails to comprise sufficient resources for processing the least one prefetch request; and

ignore, in response to the determining, the at least one prefetch request.

13. The hybrid server system of claim 8, wherein the first prefetcher is further configured to:

identify an availability of a set of resources at the server system for processing at least one prefetch request in the set of prefetch requests; and

store, prior to sending the at least one set of data, a prefetched set of data associated with the least one prefetch request in a slow memory of the memory system based on the availability of the set of resources that has been identified, wherein the memory system comprises a fast memory and the slow memory.

14. A computer program product for managing data access in an out-of-core processing environment, the computer program product comprising:

a non-transitory storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

receiving, from at least one accelerator system, a set of prefetch requests, wherein the set of prefetch requests is associated with one or more given datasets residing on the server system prior to at least one application executing on at least one user client system requiring the one or more given datasets, and wherein at least two prefetch requests in the set of prefetch requests are each associated with a different option presented to a user within the application with respect to a current state of the application;

prefetching, for at least one prefetch request in the set of prefetch requests, a first set of data and at least a second set of data from a memory system residing at the server system, wherein the first set of data and the second set of data satisfy a first and second of the least two of the set of prefetch requests, respectively; and

sending the first and second sets of data that have been prefetched to at least one accelerator system, in the set of accelerator systems, associated with the at least two of the set of prefetch requests.

15. The computer program product of claim 14, wherein the prefetching comprises:

storing the set of prefetch requests in a prefetch request queue residing within the memory system;

assigning a score to each prefetch request in the set of prefetch requests;

determining that at least each of the two of the set of prefetch requests comprises a score above a given threshold; and

prefetching at least the first and second sets of data in response to the determining that at least each of the two of the set of prefetch requests comprises a score above a given threshold.

16. The computer program product of claim 14, wherein the set of prefetch requests are received from a plurality of n-way prefetchers residing at the set of accelerator systems.

17. The computer program product of claim 14, the method further comprising:

storing, prior to sending the prefetched first and second sets of data to the at least one accelerator system, the prefetched first and second sets of data in a temporary storage area within the memory system assigned to the at least one accelerator system, wherein each accelerator system in the set of accelerator systems is assigned a temporary storage area within the memory system.

18. The computer program product of claim 14, the method further comprising:

analyzing a set of resources available at the server system for processing at least one prefetch request in the set of prefetch requests;

determining, based on the analyzing, that the server system fails to comprise sufficient resources for processing the least one prefetch request; and

ignoring, in response to the determining, the at least one prefetch request.

19. The computer program product of claim 14, the method further comprising:

identifying an availability of a set of resources at the server system for processing at least one prefetch request in the set of prefetch requests; and

storing, prior to sending the set of data, a prefetched set of data associated with the least one prefetch request in a slow memory of the memory system based on the availability of the set of resources that has been identified, wherein the memory system comprises a fast memory and the slow memory.

20. The computer program product of claim 14, the method further comprising:

monitoring, by a prefetcher on at least one accelerator system in the set of accelerator systems, usage of each dataset in a plurality of datasets stored within a prefetch memory of the at least one accelerator system;

updating, based on the monitoring, a counter associated with each dataset in response to the each dataset having been used;

calculating a computing complexity for each of the datasets in the plurality of datasets, wherein the computing complexity is associated with processing the dataset;

assigning a score to each of the datasets based on at least one of the counter and computing complexity associated therewith; and

removing at least one dataset in the plurality of datasets from the prefetch memory based on the score that has been assigned thereto.