Data access management in a hybrid memory server

• US 8,898,324 B2
• Filed: 06/24/2010
• Issued: 11/25/2014
• Est. Priority Date: 06/24/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Chan, Yuk Lung, Krishnamurthy, Rajaram B., Parris, Carl Joseph
• Primary Examiner(s): BILGRAMI, ASGHAR H
• Application Number:
  US12/822,760
• Publication Number:
  US 20110320804A1
• Time in Patent Office: 1,615 Days
• Field of Search: 709/229, 709/203
• US Class Current: 709/229
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 21/602 Providing cryptographic facilities or services
• CPC Class Codes: G06F 2212/264 Remote server
• CPC Class Codes: G06F 2212/602 Details relating to cache prefetching
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components
• CPC Class Codes: H04L 47/24 Traffic characterised by specific attributes, e.g. priority or QoS
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A method, accelerator system, and computer program access data in an out-of-core processing environment. A data access configuration is received from a server system managing a plurality of data sets. A determination is made that data sets retrieved from the server system are to be stored locally based on the data access configuration. A request to interact with a given data set is received from a user client. At least a portion of the given data set is retrieved from the server system. The at least a portion of the given data set is stored locally a memory based on the data access configuration that has been received.

Claims

1. A method, on an accelerator processing system comprising one or more processing cores, for accessing data in an out-of-core processing environment, the accelerator processing system being separate and distinct from at least one user client system and at least one server system managing a plurality of data sets stored remotely from the accelerator processing system, the method comprising:

receiving a data access configuration from the server system managing the plurality of data sets, the data access configuration dynamically configuring how the accelerator processing system accesses and caches one or more of the plurality of data sets based on an access context associated with one or more of the plurality of data sets, the access context being based on at least one of a set of attributes associated with a user requesting access to the one or more of the plurality of datasets, a set of security attributes associated with the one or more of the plurality of datasets;

a first set of data ports being used between the server system and the accelerator system, and a second set of data ports being used between the accelerator system and the user client system;

determining, based on the data access configuration, that data sets retrieved from the server system are to be stored locally;

receiving, from the user client system, a request to interact with a given data set in the plurality of data sets;

retrieving at least a portion of the given data set from the server system; and

storing the at least a portion of the given data set locally in a memory based on the data access configuration that has been received.

2. The method of claim 1, further comprising:

receiving a request to access the given data set from the user client;

determining that the at least a portion of the given data set satisfies the request to access;

processing the at least a portion of the given data set based on the request to access; and

transmitting the at least a portion of the given data set that has been processed to the user client.

3. The method of claim 1, wherein the retrieving at least a portion of the given data set from the server system further comprises retrieving all of the given data set.

4. The method of claim 1, further comprising:

receiving a request to access the given data set from the user client;

determining that the portion of the given data set fails to fully satisfy the request to access;

retrieving, in response to the determining, at least an additional portion of the given data set from the server system;

processing the portion and the additional portion of the given data set based on the request to access; and

transmitting the portion and the additional portion of the given data set that have been processed to the user client.

5. The method of claim 1, further comprising:

initiating, in response to storing the at least a portion of the given data set locally in a memory, a security counter associated with the portion of the given data;

monitoring, with respect to the portion of the given data, a given amount of time within the memory and a number of accesses to the portion of the given data; and

incrementing the security counter based on at least one of the given amount of time within the memory and the number of accesses to the portion of the given data.

6. The method of claim 5, further comprising:

determining, in response to incrementing the security counter, that a value associated with the security counter is above a given threshold; and

removing the portion of the given data from the memory.

7. The method of claim 1, wherein the memory is a cache memory.

8. An accelerator processing system in an out-of-core processing environment comprising the accelerator processing system, at least one server system managing a plurality of data sets stored remotely from the accelerator processing system, and at least one user client system, the accelerator processing system being separate and distinct from the server system and the user client system, the accelerator system comprising:

a memory;

a processing core communicatively coupled to the memory; and

a data manager communicatively coupled to the memory and the processing core, wherein the data manager is configured to perform a method comprising;

receiving a data access configuration from the server system managing the plurality of data sets, the data access configuration dynamically configuring how the accelerator processing system accesses and caches one or more of the plurality of data sets based on an access context associated with one or more of the plurality of data sets, the access context being based on at least one of a set of attributes associated with a user requesting access to the one or more of the plurality of datasets, a set of security attributes associated with the one or more of the plurality of datasets;

a first set of data ports being used between the server system and the accelerator system, and a second set of data ports being used between the accelerator system and the user client system;

determining, based on the data access configuration, that data sets retrieved from the server system are to be stored locally;

receiving, from the user client system, a request to interact with a given data set in the plurality of data sets;

retrieving at least a portion of the given data set from the server system; and

storing the at least a portion of the given data set locally in a memory based on the data access configuration that has been received.

9. The accelerator system of claim 8, wherein the method further comprises:

receiving a request to access the given data set from the user client;

determining that the at least a portion of the given data set satisfies the request to access;

processing the at least a portion of the given data set based on the request to access; and

transmitting the at least a portion of the given data set that has been processed to the user client.

10. The accelerator system of claim 8, wherein the method further comprises:

receiving a request to access the given data set from the user client;

determining that the portion of the given data set fails to fully satisfy the request to access;

retrieving, in response to the determining, at least an additional portion of the given data set from the server system;

processing the portion and the additional portion of the given data set based on the request to access; and

transmitting the portion and the additional portion of the given data set that have been processed to the user client.

11. The accelerator system of claim 8, wherein the method further comprises:

initiating, in response to storing the at least a portion of the given data set locally in a memory, a security counter associated with the portion of the given data;

monitoring, with respect to the portion of the given data, a given amount of time within the memory and a number of accesses to the portion of the given data; and

incrementing the security counter based on at least one of the given amount of time within the memory and the number of accesses to the portion of the given data.

12. The accelerator system of claim 11, wherein the method further comprises:

determining, in response to incrementing the security counter, that a value associated with the security counter is above a given threshold; and

removing the portion of the given data from the memory.

13. A computer program product for accessing data in an out-of-core processing environment, the out-of-core processing environment comprising at least one accelerator processing system, at least one server system managing a plurality of data sets stored remotely from the at least one accelerator processing system, and at least one user client system, the at least one accelerator processing system being separate and distinct from the at least one server system, the computer program product comprising:

a non-transitory storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method, by the at least one accelerator processing system, comprising;

receiving a data access configuration from the server system managing a plurality of data sets, the data access configuration dynamically configuring how the accelerator processing system accesses and caches one or more of the plurality of data sets based on an access context associated with one or more of the plurality of data sets, the access context being based on at least one of a set of attributes associated with a user requesting access to the one or more of the plurality of datasets, a set of security attributes associated with the one or more of the plurality of datasets;

a first set of data ports being used between the server system and the accelerator system, and a second set of data ports being used between the accelerator system and the user client system;

determining, based on the data access configuration, that data sets retrieved from the server system are to be stored locally;

receiving, from the user client system, a request to interact with a given data set in the plurality of data sets;

retrieving at least a portion of the given data set from the server system; and

storing the at least a portion of the given data set locally in a memory based on the data access configuration that has been received.

14. The computer program product of claim 13, wherein the method further comprises:

receiving a request to access the given data set from the user client;

determining that the portion of the given data set fails to fully satisfy the request to access;

retrieving, in response to the determining, at least an additional portion of the given data set from the server system;

processing the portion and the additional portion of the given data set based on the request to access; and

transmitting the portion and the additional portion of the given data set that have been processed to the user client.

15. The computer program product of claim 13, wherein the method further comprises:

initiating, in response to storing the at least a portion of the given data set locally in a memory, a security counter associated with the portion of the given data;

monitoring, with respect to the portion of the given data, a given amount of time within the memory and a number of accesses to the portion of the given data;

incrementing the security counter based on at least one of the given amount of time within the memory and the number of accesses to the portion of the given data;

determining, in response to incrementing the security counter, that a value associated with the security counter is above a given threshold; and

removing the portion of the given data from the memory.