Memory server

• US 7,865,570 B2
• Filed: 08/30/2005
• Issued: 01/04/2011
• Est. Priority Date: 08/30/2005
• Status: Active Grant

• Current Assignee: Illinois Institute of Technology
• Original Assignee: Illinois Institute of Technology
• Inventors: Sun, Xian-He
• Primary Examiner(s): NGUYEN, THU HA T
• Application Number:
  US11/215,321
• Publication Number:
  US 20070067382A1
• Time in Patent Office: 1,953 Days
• Field of Search: 709/217, 709/203, 709/226, 709/231, 709/233, 711/126, 382/305
• US Class Current: 709/217
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 2212/6022 Using a prefetch buffer or dedicated prefetch cache
• CPC Class Codes: G06F 2212/6026 Prefetching based on access pattern detection, e.g. stride based prefetch

Abstract

A memory server provides data access as a service to clients and has a memory service architecture and components for removing data management burdens from the client processor and providing increased speed and utility for the client through aggressive prediction of client memory requirements and fast provision of data.

Claims

1. A system of memory service for computing, comprising:

a client device including a first processor or processing core that is capable of processing an application, and a memory system; and

the client device including a built-in memory server connected to the memory system of the client device, the memory server including;

a prefetch engine for observing a data access pattern of the application processed by the client device and predicting future data requests of the client device;

a memory management engine for fetching predicted data, and including a data propeller for transferring predicted data higher in a memory hierarchy of the client and closer to the first processor or processing core; and

a second processor or processing core for processing the prefetch engine and the memory management engine.

2. The system of memory service for computing of claim 1, wherein at least some data storage requirements of the client device are obtainable from the memory server during execution of the application.

3. The system of memory service for computing of claim 1, further comprising the prefetch engine having a predictor process for monitoring the data access patterns of the client device and predicting future data requests of the client device, wherein the prefetch engine passes the predicted data requests to the data propeller, and the data propeller module passes a predicted data to a cache or memory device of the client device.

4. The system of memory service for computing of claim 2, further comprising a search/replacement policy adopter process for dynamically adopting memory management polices for the utilization of data by memory of the client device.

19. The system of memory service for computing of claim 1 wherein the client device comprises a multi-core processor, a first core of the multi-core processor is dedicated to the processing an application, and a second core of the multi-core processor is dedicated to the memory server.

5. A memory server for improving an operation of a client memory system of a client device, comprising:

a memory server processor or processing core dedicated to the memory server;

the memory server built in with the client device and connected to a client memory system for managing data needs for the client memory system or a client application processor;

a prefetch engine for observing a data access pattern of an application processed by the client device and predicting future data requests of the client device;

a memory management engine for fetching predicted data, and including a data propeller for transferring predicted data higher in a memory hierarchy of the client and closer to the first processor or processing core; and

the memory server operably connectable to the client memory system for the client application processor or client memory system to receive data from the memory server, whereby the client application processor can be relieved of a data management burden by the memory processor to increase an application processing speed of the client application processor.

6. The memory server of claim 5 wherein the data management needs include data prediction, pre-fetch, storage, and retrieval needs.

7. The memory server of claim 5 further comprising a data access prediction algorithm.

8. The memory server of claim 5 wherein the memory server can choose from a plurality of prediction algorithms.

9. The memory server of claim 5 further comprising a means for monitoring data access patterns of the application processor.

10. The memory server of claim 5 further comprising a means for identifies data access patterns of the application processor.

11. The memory server of claim 5 wherein the memory server has a pure server model and a hybrid server model.

12. The memory server of claim 5 further comprising the memory server operably connected to a client application processor.

13. The memory server of claim 5 further comprising the memory server operably connected to a high level client memory storage location for the client application processor.

14. The memory server of claim 5 wherein the client memory system includes a prefetch cache, an L1 cache, an L2 cache or client DRAM.

15. A memory server device for installation within a client device, comprising:

a) means for communicating with the client device when installed within the client device;

b) a memory server architecture, including;

i) a memory service provider process running on the memory server and initiating data transfers to higher in a memory hierarchy of the client device;

ii) a miss handler daemon process for taking on-demand misses from the client'"'"'s local memory; and

transferring data access requests from the client to a data access pattern table;

iii) a data access pattern table for tracking and storing historical data access request patterns from the client device;

to help predict future data requests of the client device;

iv) a future access predictor daemon process for predicting the future data requests of the client based on the information from the data access pattern table and for passing a predicted reference to a data propeller module;

v) a data propeller module for handling the data transfers and pushing missed data from the miss handler daemon and predicted data from future access predictor daemon process higher in the memory hierarchy of the client device; and

c) whereby;

i) the client device requests an initial data reference;

ii) the data propeller module obtains the requested data reference and sends a plurality of lines of data from the requested address space to a client device cache;

iii) the miss handler daemon collects any missed data references from the cache and local memory and stores the missed data references access information in the data access pattern table;

the miss handler daemon alerts the data propeller module when a cache miss occurs and the data propeller sends the missed data immediately without waiting for the future access predictor daemon; and

iv) the future access predictor daemon predicts the next data reference based on previous client data accesses and the data propeller retrieves and pushes the predicted data into the cache of the client device, while tracking cache misses, and dynamically fine-tunes its predictions of future data based on an increasing amount of access pattern information for the client application.

16. The memory server device of claim 15 further comprising a search/replacement policy adopter daemon for selecting polices for the utilization by the client device of fetched data to a full extent before it is replaced, and sharing of client application data access patterns with the future access predictor daemon.

17. The memory server device of claim 15 further comprising a pure server model whereby all the data for the client application is supplied by the memory server.

18. The memory server device of claim 15 further comprising a hybrid server model whereby the client uses its memory and disk for all data access patterns with a predetermined standard of locality among its required data accesses and uses the memory server when the data locality is less than the predetermined standard of locality among required data accesses.

20. The memory server of claim 15 wherein the memory server comprises a processing core that is a dedicated processing core of a multi-core processor of a client device, and a second core of the multi-core processor is dedicated to the application processing of the client device.