Memory interleave for heterogeneous computing

• US 8,095,735 B2
• Filed: 08/05/2008
• Issued: 01/10/2012
• Est. Priority Date: 08/05/2008
• Status: Active Grant

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Convey Computer
• Inventors: Brewer, Tony M., Magee, Terrell, Andrewartha, J. Michael
• Primary Examiner(s): Thai, Tuan
• Assistant Examiner(s): DINH, NGOC V
• Application Number:
  US12/186,344
• Publication Number:
  US 20100037024A1
• Time in Patent Office: 1,253 Days
• Field of Search: 711/127
• US Class Current: 711/127
• CPC Class Codes: G06F 12/0607 Interleaved addressing

Abstract

A memory interleave system for providing memory interleave for a heterogeneous computing system is provided. The memory interleave system effectively interleaves memory that is accessed by heterogeneous compute elements in different ways, such as via cache-block accesses by certain compute elements and via non-cache-block accesses by certain other compute elements. The heterogeneous computing system may comprise one or more cache-block oriented compute elements and one or more non-cache-block oriented compute elements that share access to a common main memory. The cache-block oriented compute elements access the memory via cache-block accesses (e.g., 64 bytes, per access), while the non-cache-block oriented compute elements access memory via sub-cache-block accesses (e.g., 8 bytes, per access). A memory interleave system is provided to optimize the interleaving across the system'"'"'s memory banks to minimize hot spots resulting from the cache-block oriented and non-cache-block oriented accesses of the heterogeneous computing system.

Claims

1. A system comprising:

a memory;

a first compute element that issues cache-block oriented access requests to said memory;

a second compute element that issues sub-cache-block oriented access requests to said memory; and

a memory interleave system that interleaves the cache-block oriented and sub-cache-block oriented access requests;

wherein the first compute element issues physical addresses for said cache-block oriented access requests, and wherein said second compute element issues virtual addresses for said sub-cache-block oriented access requests.

2. The system of claim 1 wherein the first compute element and second compute element comprise different instruction sets.

3. The system of claim 1 wherein the first compute element comprises a microprocessor.

4. The system of claim 3 wherein the second compute element comprises a field-programmable gate array (FPGA).

5. The system of claim 1 wherein the memory interleave system processes the issued physical addresses and virtual addresses to determine an interleaving of the cache-block oriented and sub-cache-block oriented access requests.

6. The system of claim 5 wherein the virtual addresses for the sub-cache-block oriented access requests are not first translated to physical addresses for the memory interleave system to determine said interleaving.

7. The system of claim 5 wherein the system employs page coloring to make said virtual addresses and corresponding physical addresses access a common memory location independent of whether the memory interleave system is processing an issued physical address or virtual address to determine an interleaving.

8. The system of claim 1 further comprising:

a plurality of memory controllers for the memory, wherein the memory interleave system determines, for each of the issued cache-block oriented and sub-cache-block oriented access requests, a memory controller to direct the access request to minimize hot spots in the memory.

9. A system comprising:

a memory;

a first compute element that issues physical address requests for accessing said memory;

a second compute element that issues virtual address requests for accessing said memory;

a memory interleave system that receives the physical address requests from the first compute element and the virtual address requests from the second compute element, and performs memory interleaving of the requests without requiring the received virtual address requests to be translated into a physical address; and

an operating system that employs page coloring, and wherein said page coloring is leveraged by said memory interleaving to make virtual addresses and corresponding physical addresses map to common banks of the memory.

10. The system of claim 9 wherein the first compute element issues cache-block oriented access requests to said memory;

and wherein the second compute element issues sub-cache-block oriented access requests to said memory.

11. The system of claim 10 wherein the memory interleave system processes the issued physical addresses and virtual addresses to determine an interleaving of the cache-block oriented and sub-cache-block oriented access requests.

12. The system of claim 9 wherein the first compute element comprises a microprocessor.

13. The system of claim 12 wherein the second compute element comprises a field-programmable gate array (FPGA).

14. The system of claim 9 further comprising:

a plurality of memory controllers for the memory, wherein the memory interleave system determines, for each of the issued physical address requests and virtual address requests, a memory controller to direct the request to minimize hot spots in the memory.

15. A system comprising:

a memory;

a plurality of memory controllers for said memory;

a first compute element that issues physical addresses for cache-block oriented access requests to said memory;

a second compute element that issues virtual addresses for sub-cache-block oriented access requests to said memory, wherein said first and second compute elements share a common physical and virtual address space of the memory;

a memory interleave system that receives the physical address for the cache-block oriented access requests issued by the first compute element and receives the virtual addresses for the sub-cache-block oriented access requests issued by the second compute element, and said memory interleave system determines, for each of the received cache-block oriented and sub-cache-block oriented access requests, one of the plurality of memory controllers to direct the access request for interleaving the cache-block oriented and sub-cache-block oriented access requests; and

wherein the memory interleave system employs page coloring to ensure that said received virtual addresses and corresponding physical addresses map to a common location of the memory.

16. The system of claim 15 wherein the first compute element and second compute element comprise different instruction sets.

17. The system of claim 15 wherein the first compute element comprises a microprocessor.

18. The system of claim 17 wherein the second compute element comprises a field-programmable gate array (FPGA).

19. A system comprising:

non-sequential access memory;

a cache-access path in which cache-block data is communicated between said non-sequential access memory and a cache memory; and

a direct-access path in which sub-cache-block data is communicated to/from said non-sequential access memory; and

a memory interleave system for interleaving accesses to said non-sequential access memory via the cache-access path and the direct-access path to minimize hot spots within said non-sequential access memory

wherein said memory interleave system receives a physical address for a cache-block memory access request via the cache-access path, and wherein said memory interleave system receives a virtual address for a sub-cache-block memory access request via the direct-access path;

wherein the memory interleave system determines said interleaving using the received physical address for the cache-block memory access request and the received virtual address for the sub-cache-block memory access request without requiring the virtual address to first be translated into a physical address; and

wherein the system employs page coloring to ensure that said received physical address and a corresponding virtual address map to a common location of the memory, and wherein said memory interleave system uses knowledge of the page coloring to determine said interleaving for the received physical address and the received virtual address without requiring the virtual address to first be translated into a physical address.

20. The system of claim 19 further comprising:

a processor;

a heterogeneous compute element; and

said cache memory.

21. The system of claim 20 wherein in said direct-access path said sub-cache-block data is fetched from said non-sequential access memory to said heterogeneous compute element.

22. The system of claim 20 wherein in said direct-access path said sub-cache-block data is stored to said non-sequential access memory from said heterogeneous compute element.

23. The system of claim 20 comprising:

a die on which said processor, said heterogeneous compute element, and said cache memory are included.

24. The system of claim 20 wherein said processor accesses cache-block data from said non-sequential access memory via said cache-access path;

and wherein said heterogeneous compute element is operable to access said sub-cache-block data from said non-sequential access memory via said direct-access path.

25. The system of claim 20 wherein said heterogeneous compute element comprises a dynamically-reconfigurable instruction set.

26. The system of claim 19 wherein said heterogeneous compute element comprises a field-programmable gate array (FPGA).

27. The system of claim 19 wherein said non-sequential access memory comprises:

a scatter/gather memory module.

28. The system of claim 19 further comprising:

a plurality of memory controllers for said non-sequential access memory, wherein the memory interleave system determines, for each of the accesses to said non-sequential access memory via the cache-access path and the direct-access path, one of the memory controllers to direct the access request to minimize hot spots within said non-sequential access memory.

29. The system of claim 19 wherein said memory interleave system employs a two-level hierarchical interleave scheme for said interleaving, wherein a first level interleaves across cache-block memory accesses received via said cache-access path, and a second level interleaves across sub-cache-block memory accesses received via said direct-access path.

30. A memory interleave system comprising:

an interface for receiving cache-block oriented memory access requests;

an interface for receiving sub-cache-block oriented memory access requests;

logic for processing the received cache-block oriented and sub-cache-block oriented memory access requests for interleaving the received memory access requests to minimize hot spots within a memory; and

wherein, for said cache-block oriented memory access requests, said interface receives physical memory addresses; and

wherein, for said sub-cache-block oriented memory access requests, said interface receives virtual memory addresses.

31. The memory interleave system of claim 30 wherein said logic for processing processes the received cache-block oriented and sub-cache-block oriented memory access requests according to a two-level hierarchical interleave scheme, wherein a first level interleaves across the cache-block memory access requests, and a second level interleaves across the sub-cache-block memory access requests.

32. The memory interleave system of claim 30 wherein said cache-block oriented memory access requests are issued by a first compute element of a system;

and wherein said sub-cache-block oriented memory access requests are issued by a second compute element of the system.

33. The memory interleave system of claim 32 wherein said first compute element and said second compute element comprise different instruction sets.

34. The memory interleave system of claim 30 wherein said logic for processing determines said interleaving of the cache-block and sub-cache-block oriented memory access requests without requiring the virtual addresses of the sub-cache-block oriented memory access requests to first be translated into a physical address.

35. The memory interleave system of claim 34 wherein an operating system employs page coloring to ensure that said virtual memory addresses and corresponding physical memory addresses map to a common location of the memory, and wherein said memory interleave system uses knowledge of the page coloring to determine said interleaving for the received physical address and the received virtual address without requiring the virtual address to first be translated into a physical address.

36. The memory interleave system of claim 30 wherein said logic for processing determines, for each of the received cache-block oriented and sub-cache-block oriented memory access requests, one of a plurality of memory controllers to direct the memory access request.

37. A method for performing memory interleaving, said method comprising:

receiving, by a memory interleave system, a cache-block oriented memory access request from a first compute element of a system;

receiving, by the memory interleave system, a sub-cache-block oriented memory access request from a second compute element of the system; and

performing memory interleaving, by the memory interleave system, for the received cache-block oriented and sub-cache-block oriented memory access requests, wherein the cache-block oriented memory access request is a physical address request, and wherein the sub-cache-block oriented memory access request is a virtual address request.

38. The method of claim 37 further comprising:

translating a virtual address to a physical address for said physical address request for said cache-block oriented memory access request, wherein said translating said virtual address to said physical address for said physical address request for said cache-block oriented memory access request is performed before said performing said memory interleaving.

39. The method of claim 38 wherein said translating comprises:

translating, by said first compute element, said virtual address to said physical address prior to sending said cache-block oriented memory access request to said memory interleave system.

40. The method of claim 38 further comprising:

translating said virtual address to a physical address for said sub-cache-block oriented memory access request, wherein said translating of said virtual address to said physical address for said sub-cache-block oriented memory access request is performed after said performing said memory interleaving.

41. The method of claim 40 wherein said translating of said virtual address to said physical address for said sub-cache-block oriented memory access request comprises:

translating, by one of a plurality of memory controllers in the system to which the memory interleave system sends the virtual address for the sub-cache-block oriented memory access request, said virtual address to said physical address for said sub-cache-block oriented memory access request.

42. The method of claim 37 wherein the receiving said cache-block oriented memory access request comprises:

receiving said cache-block oriented memory access request by a host interface communicatively coupled to the first compute element.

43. The method of claim 37 wherein the receiving said sub-cache-block oriented memory access request comprises:

receiving said sub-cache-block oriented memory access request by a memory interface communicatively coupled to the second compute element.

44. The method of claim 37 wherein the second compute element comprises a different instruction set than said first compute element.

45. The method of claim 37 wherein the second compute element comprises a dynamically reconfigurable instruction set.

46. The method of claim 37 wherein the received cache-block oriented request and the received sub-cache-block oriented request each request access to a common memory of the system, and wherein said first and second compute elements share a common physical and virtual address space of the common memory.

47. The method of claim 37 wherein said performing memory interleaving comprises:

determining, by the memory interleave system, which of a plurality of memory controllers in the system to direct the received requests.

48. The method of claim 47 wherein the determining is made, at least in part, to minimize hot spots within the memory.

49. The method of claim 37 wherein said performing memory interleaving comprises:

employing a two-level hierarchical interleave scheme, wherein a first level interleaves across cache-block memory accesses, and a second level interleaves across sub-cache-block memory accesses.

50. A method for performing memory interleaving comprising:

performing, in a first level of a two-level interleaving scheme, interleaving across full cache lines of a memory;

performing, in a second level of the two-level interleaving scheme, interleaving across sub-cache lines of the memory;

using 31 groups of banks for the first level of the two-level interleaving scheme; and

using 31 banks within each of said 31 groups of banks for the second level of the two-level interleaving scheme.

51. The method of claim 50 wherein said sub-cache lines comprise words within cache lines.