Multiple processor system and method including multiple memory hub modules

• US 20050050255A1
• Filed: 08/28/2003
• Published: 03/03/2005
• Est. Priority Date: 08/28/2003
• Status: Active Grant

• Current Assignee: Round Rock Research LLC
• Original Assignee: Micron Technology, Inc.
• Inventors: Jeddeloh, Joseph M.
• Granted Patent: US 7,136,958 B2
• Time in Patent Office: Days
• US Class Current: 710/317
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G11C 5/00 Details of stores covered by group G11C11/00

Abstract

A processor-based electronic system includes several memory modules arranged in first and second ranks. The memory modules in the first rank are directly accessed by any of several processors, and the memory modules in the second rank are accessed by the processors through the memory modules in the first rank. The data bandwidth between the processors and the memory modules in the second rank is varied by varying the number of memory modules in the first rank that are used to access the memory module in the second set. Each of the memory modules includes several memory devices coupled to a memory hub. The memory hub includes a memory controller coupled to each memory device, a link interface coupled to a respective processor or memory module, and a cross bar switch coupling any of the memory controllers to any of the link interfaces.

Claims

1. A memory hub for use in coupling each of a plurality of memory requestors to each of a plurality of memory devices, the memory hub, comprising:

a plurality of memory controllers, each of the memory controllers being coupled to at least one of the memory devices;

a first plurality of link interfaces; and

a cross bar switch having a first plurality of switch ports and a plurality of memory ports, each of the switch ports being coupled to a respective one of the link interfaces, and each of the memory ports being coupled to a respective one of the memory controllers, the cross bar switch being operable to selectively couple each of the link interfaces to any one of the memory controllers.

2. The memory hub of claim 1, further comprising a second plurality of link interfaces, and wherein the cross bar switch further includes a second plurality of switch ports each of which is coupled to a respective one of the link interfaces in the second plurality of link interfaces.

3. The memory hub of claim 1 wherein each of the memory controllers comprises a dynamic random access memory controller.

4. The memory hub of claim 1, further comprising a respective cache memory device associated with each of the memory controllers, each of the cache memory devices being operable to store data that is accessed through a respective one of the memory ports of the cross bar switch.

5. The memory hub of claim 1 wherein each of the link interfaces comprises a bi-directional link interface.

6. The memory hub of claim 1 wherein each of the link interfaces comprises a pair of uni-directional links one of which includes an input port and the other of which includes an output port.

7. The memory hub of claim 1, further comprising a respective write buffer associated with each of the memory controllers, each of the write buffers being operable to receive write data and associated write addresses through a respective one of the memory ports of the cross bar switch and to store the write data and addresses for subsequent coupling to a respective one of the memory controllers.

8. The memory hub of claim 7 wherein each of the write buffers accumulate write data and addresses for a plurality of write memory accesses and then sequentially couple the plurality of write data and addresses to a respective one of the memory controllers without any intervening read memory accesses.

9. The memory hub of claim 1, further comprising a direct memory access device coupled to the cross bar switch and the memory controllers, the direct memory access device being operable to cause the cross bar switch and each of the memory controllers to perform memory write and read accesses.

10. A memory module, comprising:

a substrate;

a plurality of memory devices carried by the substrate; and

a memory hub carried by the substrate, the memory hub comprising;

a plurality of memory controllers, each of the memory controllers being coupled to at least one of the memory devices;

a first plurality of link interfaces, each of the link interfaces being for coupling to each of a plurality of memory requesters; and

a cross bar switch having a first plurality of switch ports and a plurality of memory ports, each of the switch ports being coupled to a respective one of the link interfaces, and each of the memory ports being coupled to a respective one of the memory controllers, the cross bar switch being operable to selectively couple each of the link interfaces to any one of the memory controllers.

11. The memory module of claim 10 wherein the memory hub further comprising a second plurality of link interfaces, and wherein the cross bar switch further includes a second plurality of switch ports each of which is coupled to a respective one of the link interfaces in the second plurality of link interfaces.

12. The memory module of claim 10 wherein each of the memory devices comprises a dynamic random access memory device.

13. The memory module of claim 10 wherein the memory hub further comprises a respective cache memory device associated with each of the memory controllers, each of the cache memory devices being operable to store data that is accessed through a respective one of the memory ports of the cross bar switch.

14. The memory module of claim 10 wherein the substrate comprises a semiconductor substrate, and wherein the memory devices, the link interfaces, the memory controllers and the cross bar switch are fabricated as a common integrated circuit in the semiconductor substrate.

15. The memory module of claim 10 wherein each of the link interfaces in the memory hub comprises a bi-directional link interface.

16. The memory module of claim 10 wherein each of the link interfaces in the memory hub comprises a pair of uni-directional links one of which includes an input port and the other of which includes an output port.

17. The memory module of claim 10 wherein the memory hub further comprises a respective write buffer associated with each of the memory controllers, each of the write buffers being operable to receive write data and associated write addresses through a respective one of the memory ports of the cross bar switch and to store the write data and addresses for subsequent coupling to a respective one of the memory controllers.

18. The memory module of claim 17 wherein each of the write buffers accumulate write data and addresses for a plurality of write memory accesses and then sequentially couple the plurality of write data and addresses to a respective one of the memory controllers without any intervening read memory accesses.

19. The memory module of claim 10 wherein the memory hub further comprises a direct memory access device coupled to the cross bar switch and the memory controllers, the direct memory access device being operable to cause the cross bar switch and each of the memory controllers to perform memory write and read accesses.

20. A memory system, comprising:

a plurality of memory requestors;

a first rank of memory modules coupled to the memory requestors, each of the memory modules in the first rank being coupled to a plurality of the memory requestors, each of the memory modules comprising;

a plurality of memory devices; and

a memory hub comprising;

a plurality of memory controllers, each of the memory controllers being coupled to at least one of the memory devices;

a first plurality of link interfaces, each of the link interfaces being coupled to a respective one of the memory requesters; and

a cross bar switch having a first plurality of switch ports and a plurality of memory ports, each of the switch ports being coupled to a respective one of the link interfaces, and each of the memory ports being coupled to a respective one of the memory controllers, the cross bar switch being operable to selectively couple each of the link interfaces to any one of the memory controllers.

21. The memory system of claim 20 wherein the memory hub further comprising a second plurality of link interfaces, and wherein the cross bar switch further includes a second plurality of switch ports each of which is coupled to a respective one of the link interfaces in the second plurality of link interfaces.

22. The memory system of claim 20 wherein each of the memory devices comprises a dynamic random access memory device.

23. The memory system of claim 20 wherein the memory hub further comprises a respective cache memory device associated with each of the memory controllers, each of the cache memory devices being operable to store data that is accessed through a respective one of the memory ports of the cross bar switch.

24. The memory system of claim 20 wherein the substrate comprises a semiconductor substrate, and wherein the memory devices, the link interfaces, the memory controllers and the cross bar switch are fabricated as a common integrated circuit in the semiconductor substrate.

25. The memory system of claim 20 wherein each of the link interfaces in the memory hub comprises a bi-directional link interface.

26. The memory system of claim 20 wherein each of the link interfaces in the memory hub comprises a pair of uni-directional links one of which includes an input port and the other of which includes an output port.

27. The memory system of claim 20 wherein the memory hub further comprises a respective write buffer associated with each of the memory controllers, each of the write buffers being operable to receive write data and associated write addresses through a respective one of the memory ports of the cross bar switch and to store the write data and addresses for subsequent coupling to a respective one of the memory controllers.

28. The memory system of claim 27 wherein each of the write buffers accumulate write data and addresses for a plurality of write memory accesses and then sequentially couple the plurality of write data and addresses to a respective one of the memory controllers without any intervening read memory accesses.

29. The memory system of claim 20 wherein the memory hub further comprises a direct memory access device coupled to the cross bar switch and the memory controllers, the direct memory access device being operable to cause the cross bar switch and each of the memory controllers to perform memory write and read accesses.

30. The memory system of claim 20 wherein at least some of the memory requesters comprise a processor.

31. The memory system of claim 20 wherein at least some of the memory requesters comprise a direct memory access device.

32. The memory system of claim 20, further comprising a second rank of memory modules coupled to a plurality of the memory modules in the first rank, each of the memory modules in the second rank comprising:

a plurality of memory devices; and

a memory hub comprising;

a plurality of memory controllers, each of the memory controllers being coupled to at least one of the memory devices;

a first plurality of link interfaces, each of the link interfaces being coupled to a respective one of the memory modules in the first rank; and

a cross bar switch having a first plurality of switch ports and a plurality of memory ports, each of the switch ports being coupled to a respective one of the link interfaces, and each of the memory ports being coupled to a respective one of the memory controllers, the cross bar switch being operable to selectively couple each of the link interfaces to any one of the memory controllers.

33. The memory system of claim 20, further comprising a plurality of sets of memory modules in a second rank, the number of sets of memory modules in the second rank corresponding in number to the number of memory modules in the first rank, the memory modules in each set in the second rank being coupled to a respective one of the memory modules in the first rank, each of the memory modules in the second rank comprising:

a plurality of memory devices; and

a memory hub comprising;

a plurality of memory controllers, each of the memory controllers being coupled to at least one of the memory devices;

a link interface coupled to the memory module in the first rank to which the respective set of memory modules in the second rank corresponds; and

a cross bar switch having a switch port and a plurality of memory ports, the switch port being coupled to the link interface, and each of the memory ports being coupled to a respective one of the memory controllers, the cross bar switch being operable to selectively couple the link interface to any one of the memory controllers.

34. The memory system of claim 20 wherein the memory devices, the link interfaces, the memory controllers and the cross bar switch are fabricated as a common integrated circuit in a semiconductor substrate.

35. A memory system, comprising:

a plurality of memory requesters;

a first rank of memory modules coupled to the memory requesters, the memory modules in the first rank each including a first set of memory ports corresponding in number to the number of memory requestors, each of the memory ports in the first rank being coupled to a respective one of the memory requestors, the memory modules in the first rank further including a second set of memory ports, each of the memory modules in the first rank including a plurality of memory devices and a memory hub coupled to the memory devices and to the memory ports in the first and second sets;

a second rank of memory modules each including at least one memory port coupled to a memory module in the first rank through a memory port in the second set, each of the memory modules in the second rank including a plurality of memory devices and a memory hub coupled to the memory devices and to the at least one memory port.

36. The memory system of claim 35 wherein each of the memory modules in the second rank includes a plurality of memory ports corresponding in number to the number of memory modules in the first rank, the memory ports in each of the memory modules in the second rank being coupled to the memory modules in the first rank through respective memory ports in the second set.

37. The memory system of claim 35 wherein each of a plurality of sets of memory modules in the second rank are coupled to a respective one of the memory modules in the first rank, each of the memory modules in the first rank including a memory port coupled to the memory module in the first rank to which the respective set of memory modules in the second rank corresponds.

38. The memory system of claim 35 wherein the memory hub in each of the memory modules in the first rank comprise a cross bar switch operable to couple any of the memory ports in the first set to any of the memory ports in the second set and to any of the memory devices.

39. The memory system of claim 38 wherein the cross bar switch is further operable to couple any of the memory ports in the second set to any of the memory devices.

40. A method of accessing memory devices from a plurality of memory request ports, the method comprising:

arranging a first plurality of memory modules in a first rank, each of the memory modules in the first rank including a plurality of memory devices;

arranging a second plurality of memory modules in a second rank, each of the memory modules in the second rank including a plurality of memory devices;

accessing each of the memory modules in the first rank from any of the memory request ports; and

accessing each of the memory modules in the second rank from any of the memory request ports through at least one of the memory modules in the first rank.

41. The method of claim 40 wherein the act of accessing each of the memory modules in the second rank through at least one of the memory modules in the first rank comprises accessing each of the memory modules in the second through each of the memory modules in the first rank.

42. The method of claim 40 wherein the act of accessing each of the memory modules in the second rank through at least one of the memory modules in the first rank comprises accessing a set of memory modules in the second rank through a memory module in the first rank corresponding to the set of memory modules in the second rank.

43. The method of claim 40, further comprising accessing each of the memory modules in the first rank from at least some of the memory modules in the second rank.

44. The method of claim 40 wherein the act of accessing each of the memory modules in the second rank from any of the memory request ports through at least one of the memory modules in the first rank comprises varying the number of memory modules in the first rank through which a memory module in the second rank is accessed from one of the memory request ports to vary the data bandwidth between the memory request port and the memory module in the second rank that is being accessed.

45. The method of claim 40, further comprising accumulating write memory accesses in at least some of the memory modules and sequentially executing the stored write memory accesses.