Two-tier distributed memory

• US 9,632,936 B1
• Filed: 09/29/2015
• Issued: 04/25/2017
• Est. Priority Date: 12/09/2014
• Status: Active Grant

• Current Assignee: DataRobot, Inc.
• Original Assignee: Parallel Machines Ltd.
• Inventors: Zuckerman, Gal, Braverman, Avner, Khermosh, Lior
• Primary Examiner(s): Giardino, Jr., Mark
• Assistant Examiner(s): Waddy, Jr., Edward
• Application Number:
  US14/868,788
• Time in Patent Office: 574 Days
• Field of Search: 711162, 711112, 711154, 711E12001, 711E12017, 711100, 711118, 711161, 711165, 711202, 711E12083, 711170, 711206, 711221, 711E12066, 709201
• CPC Class Codes: G06F 12/0806 Multiuser, multiprocessor or multiprocessing cache systems
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0842 for multiprocessing or multitasking
• CPC Class Codes: G06F 12/0855 Overlapped cache accessing, e.g. pipeline G06F12/0846 takes precedence
• CPC Class Codes: G06F 13/1673 using buffers
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/1024 Latency reduction
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0658 Controller construction arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Described herein are systems and methods to redirect data read requests from the first tier to the second tier of a two-tier distributed memory. The first tier includes memory modules with data sets. Data interfaces associated with the first tier memory modules, receive from a second tier including compute elements and associated cache memories, requests to fetch data from the first tier. If a data set has not recently been fetched by the second tier, then the data interface will send the data set from the first tier to the cache memory associated with the requesting compute element. If a data set has recently been fetched by the second tier, the data interface will redirect the requesting compute element to fetch the data set from the cache memory in which the data set is currently located.

Claims

1. A method for redirecting data read requests, comprising:

recording, by a first data interface associated with storage of a first data set in a memory module, an identity associated with a first compute element that has requested, and has received, said first data set from the first data interface, such that an association is made between said first compute element and the first data set;

receiving, from a second compute element, in the first data interface, a current request to obtain said first data set;

detecting, by said first data interface, in conjunction with the current request, and based on the association made, that the second compute element is currently requesting the first data set that was previously requested by the first compute element; and

,

based on said detection, causing the second compute element, by the first data interface, in conjunction with the current request, to receive the first data set from the first compute element or a cache thereof instead of receiving said first data set from said first data interface or associated memory module, thereby effectively redirecting the current request.

2. The method of claim 1, wherein said causing is achieved by signaling, by the first data interface, to the second compute element, that the first data set is located with the first compute element, thereby making the second compute element fetch the first data set from the first compute element or the cache thereof.

3. The method of claim 2, wherein said signaling comprises:

sending, by the first data interface, to the second compute element, said identity associated with the first compute element, thereby enabling the second compute element to communicatively locate the first compute element in conjunction with said fetching,

in which said identity is selected from a group consisting of;

(i) an identifier or address of the first compute element, (ii) an identifier or address of the cache associated with the first compute element, (iii) an address associated with a communication interface via which the first compute element or the cache thereof is accessed, and (iv) any sort of a pointer to the first compute element or the cache thereof.

4. The method of claim 1, wherein said causing is achieved by instructing the first compute element, by the first data interface, to send the first data set from the first compute element or the cache thereof to the second compute element.

5. The method of claim 1, wherein:

the first data interface is communicatively connected with the first compute element and the second compute element via a switching network, such that;

the first compute element that has requested, and has received, the first data set from the first data interface, has done so via the switching network;

said sending of the current request by the second compute element is also done via the switching network;

but

the reception of the first data set by the second compute element from the first compute element or a cache thereof is done without involving the switching network, thereby either (i) reducing a load on the switching network, or (ii) reducing a latency associated with obtaining the first data set by the second compute element, or (iii) both reducing a load on the switching network and reducing a latency associated with obtaining the first data set by the second compute element.

6. The method of claim 5, wherein:

the first data interface and associated memory module are located in a first server;

the first compute element and the second compute element are co-located in a second server; and

the reception of the first data set by the second compute element from the first compute element or a cache thereof is done via a data bus belonging to the second server.

7. The method of claim 6, wherein the first compute element and the second compute element are co-located on a single board comprising the data bus.

8. The method of claim 6, wherein the reception of the first data set by the second compute element from the first compute element or a cache thereof is associated with a latency that is lower than 500 (five hundred) nanoseconds as facilitated by the data bus and said co-location in the second server, thereby achieving said reduction in latency associated with obtaining the first data set by the second compute element.

9. The method of claim 5, wherein the reception of the first data set by the second compute element from the first compute element or a cache thereof is done via another switching network.

10. The method of claim 5, wherein:

the first data interface is associated with a first memory module that stores the first data set, such that said receiving of the first data set from the first data interface involves the first memory module, in which said first memory module belongs to a shared memory pool;

the shared memory pool in conjunction with the first switching network and the two compute elements achieve memory disaggregation, in which said requesting and receiving of the first data set from the first data interface is done in less than 5 (five) microsecond, thereby facilitating said memory disaggregation; and

the reception of the first data set by the second compute element from the first compute element or a cache thereof is associated with a latency that is at least an order of magnitude lower than a latency associated with said requesting and receiving of the first data set from the first data interface, thereby boosting performance in conjunction with said memory disaggregation.

11. The method of claim 1, further comprising:

recording, by the first data interface, in conjunction with said identity recording, a first time stamp associated with receiving from the first compute element the request for the first data set;

calculating, by the first data interface, a time difference between current time associated with said detection and the first time stamp; and

proceeding, by the first data interface, with said causing the second compute element to receive the first data set from the first compute element or a cache thereof, but only if said time difference is below a first threshold, thereby increasing probability of receiving, by the second compute element, the first data set from the first compute element or the cache thereof.

12. A system operative to decide on redirecting data read requests in conjunction with memory disaggregation, comprising:

a plurality of compute elements;

a first data interface associated with a first memory module storing a plurality of data sets, in which the first memory module belongs to a shared memory pool operative to achieve, in conjunction with the plurality of compute elements, memory disaggregation in the system; and

a registry associated with the first data interface,

wherein;

the first data interface is configured to record, in the registry, information associated with prior distribution of any of the data sets to any of the compute elements in conjunction with operation of the memory disaggregation; and

the first data interface is further configured to;

receive from one of the compute elements a current request for one of the data sets; and

decide, based at least in part on the information recorded, whether to (i) respond to said current request by transporting, in conjunction with said memory disaggregation, the data set from the first memory module to the compute element making the current request, or (ii) respond to said current request by causing the compute element making the current request to receive the data set from one of the compute elements to which the data set was previously distributed, thereby boosting performance of the memory disaggregation.

13. The system of claim 12, wherein the information recorded comprises a list of the data sets that were previously distributed to any of the compute elements, in which the decision to receive the data set from one of the compute elements is made when the data set currently requested is in the list.

14. The system of claim 13, wherein the information recorded further comprises, per each data set in the list, at least a first identity associated with one of the compute elements to which the data set was previously distributed, in which said first identity is used to communicatively locate the compute elements to which the data set was previously distributed, thereby facilitating said causing of the compute element making the current request to receive the data set from one of the compute elements to which the data set was previously distributed.

15. The system of claim 14, further comprising:

a switching network configured to communicatively interconnect the plurality of compute elements and the first data interface,

wherein said decision is further based on an estimation of the system to whether receiving the data set from one of the compute elements via the switching network is communicatively preferable over transporting the data set via the switching network from the first memory module to the compute element making the current request.

16. The system of claim 15, wherein the switching network comprises:

a switching network communicatively interconnecting the plurality of compute elements and the first data interface; and

a communication sub-system communicatively connecting the compute element making the current request with one of the compute elements to which the data set was previously distributed,

in which the receiving of the data set from one of the compute elements is done via the communication sub-system, and is communicatively preferable as a result of a loading condition associated with the switching network.

17. The system of claim 16, wherein the communication sub-system is a data bus.

18. The system of claim 16, wherein the communication sub-system is another switching network.

19. The system of claim 13, wherein the compute element to which the data set was previously distributed is configured to:

store locally the data set in an associated cache until such time that the data set is erased from the cache; and

consequently

indicate to the first data interface that the data set in no longer stored in the cache, thereby causing the first data interface to adjust the list in the registry accordingly, such that any future request for the data set will be served from the first memory module.

20. The system of claim 12, further comprising a switching network, wherein:

said prior distribution is done via the switching network and in conjunction with the first memory module so as to result in a distribution latency of between 5 (five) microseconds and 500 (five hundred) nanoseconds, which is fast enough to facilitate the memory disaggregation in the system; and

the reception of the data set from one of the compute elements to which the data set was previously distributed is associated with a latency that is lower than 500 (five hundred) nanoseconds, thereby boosting performance in conjunction with said memory disaggregation.

21. A system operative to produce a two-tier distributed memory, comprising:

a plurality of compute elements associated respectively with a plurality of cache memories;

a plurality of data interfaces associated respectively with a plurality of memory modules storing distributively a plurality of data sets; and

a switching network configured to communicatively interconnect the plurality of compute elements and the plurality of data interfaces,

wherein;

each of the compute elements is configured to request and receive from any of the data interfaces, via the switching network, at least one of the data sets stored in the associated memory module, thereby creating, in conjunction with the plurality of cache memories, a distributed copy of at least some of the data sets; and

each of the data interfaces is configured to;

record, in conjunction with said requests for the data sets made therewith, an identity of the compute element which made the request; and

redirect any current request for one of the data sets to one of the compute elements whose identity has been recorded,

thereby producing a two-tier distributed memory, in which one tier is associated with the plurality of memory modules, and a second tier is associated with the plurality of cache memories.

22. The system of claim 21, wherein each of the compute elements currently caching a data set is configured to:

store locally the data set in the respective cache memory until such time that the data set is erased from the cache memory; and

consequently

indicate to the appropriate data interface that the data set is no longer stored in the cache memory, thereby causing the appropriate data interface to remove the identity of the compute element from the record, such that any future request for the data set will be served from the respective memory module.

23. The system of claim 21, wherein each redirected request is served from the cache memory associated with the compute element receiving the redirection.

24. The system of claim 21, wherein the two-tier distributed memory is operative to produce memory disaggregation in the system.

25. The system of claim 21, wherein said redirection is done provided that a first condition has been met, in which said first condition is selected from a group consisting of:

(i) a condition in which, according to a topology associated with the switching network, serving the current request, via the switching network, from one of the compute elements whose identity has been recorded, will result in a latency that is lower than a latency associated with serving the current request, via the switching network, from the appropriate data interface, and (ii) a condition in which the appropriate data interface is overloaded.