System and method for arbitration of memory responses in a hub-based memory system

• US 7,412,574 B2
• Filed: 02/05/2004
• Issued: 08/12/2008
• Est. Priority Date: 02/05/2004
• Status: Active Grant

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Micron Technology, Inc.
• Inventors: Jeddeloh, Joseph M.
• Primary Examiner(s): Sough; Hyung S.
• Assistant Examiner(s): Doan; Duc T.
• Application Number:
  US10/773,520
• Publication Number:
  US 20050177677A1
• Time in Patent Office: 1,650 Days
• Field of Search: None
• US Class Current: 711/158
• CPC Class Codes: G06F 13/1642 with request queuing

Abstract

A memory hub module includes a decoder that receives memory requests determines a memory request identifier associated with each memory request. A packet memory receives memory request identifiers and stores the memory request identifiers. A packet tracker receives remote memory responses and associates each remote memory response with a memory request identifier and removes the memory request identifier from the packet memory. A multiplexor receives remote memory responses and local memory responses. The multiplexor selects an output responsive to a control signal. Arbitration control logic is coupled to the multiplexor and the packet memory and develops the control signal to select a memory response for output.

Claims

1. A memory hub, comprising:

a decoder being operable to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;

a packet memory coupled to the decoder, the packet memory being operable to receive memory request identifiers from the decoder and to store the received memory request identifiers;

a multiplexor being operable to couple either remote memory responses that are received responsive to the remote memory requests or local memory responses that are received responsive to the local memory requests to an output responsive to a control signal; and

arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal based on the determination.

2. The memory hub of claim 1 wherein the arbitration control logic generates the control signal based on an oldest memory request identifier in the packet memory.

3. The memory hub of claim 1 wherein the packet memory is a first-in, first-out (FIFO) memory.

4. The memory hub of claim 1 wherein the arbitration control logic generates the control signal such that if an oldest memory request identifier stored in the packet memory is for a local memory request, the multiplexor outputs a local memory response.

5. The memory hub of claim 1 wherein each of the local and remote memory responses comprise data and a header identifying a memory request corresponding to the memory response.

6. The memory hub of claim 1, further comprising a packet tracker coupled to the packet memory, the packet memory being operable to receive the remote memory responses and to associate each received remote memory response with a memory request identifier stored in the packet memory, the packet tracker being operable to cause the memory request identifier to be removed from the packet memory.

7. A memory hub being operable to receive local memory responses and remote memory responses, the memory hub being operable to apply an arbitration algorithm to select the order in which the local and remote memory responses are provided on an uplink output based on the ages of memory requests corresponding to the local and remote memory responses.

8. The memory hub of claim 7 wherein the memory hub further comprises a packet memory that stores memory request identifiers in an order in which the corresponding memory requests are received.

9. The memory hub of claim 8 wherein the memory hub further comprises a multiplexer coupled to the packet memory, the multiplexor providing either a local or a remote memory response on an output responsive to a control signal.

10. The memory hub of claim 9 wherein the memory hub further comprises arbitration logic coupled to the packet memory and the multiplexer, and wherein the arbitration logic applies the control signal to the multiplexer to control which memory responses are provided on the output.

11. The memory hub of claim 7 wherein each of the local and remote memory responses comprise data and a header identifying a memory request corresponding to the memory response.

12. A memory module, comprising:

a plurality of memory devices; and

a memory hub coupled to the memory devices, the memory hub comprising;

a decoder adapted to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;

a packet memory adapted to receive memory request identifiers and store the memory request identifiers;

a multiplexor adapted to receive remote memory responses that are responsive to the remote memory requests and local memory responses that are responsive to the local memory requests and being operable to select either the remote memory responses or the local memory responses in response to a control signal; and

arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal to control selection of which memory response to output based on the determination.

13. The memory module of claim 12 wherein each of the memory devices comprise an SDRAM.

14. The memory module of claim 12 wherein the arbitration control logic generates the control signal based on an oldest memory request identifier in the packet memory.

15. The memory module of claim 12 wherein the packet memory is a first-in, first-out (FIFO) memory.

16. The memory hub of claim 12 wherein the arbitration control logic generates the control signal such that if an oldest memory request identifier stored in the packet memory is for a local memory request, the multiplexor outputs a local memory response.

17. The memory hub of claim 12 wherein each of the local and remote memory responses comprise data and a header identifying a memory request corresponding to the memory response.

18. The memory module of claim 12, further comprising a packet tracker adapted to receive the remote memory responses and being operable to associate each remote memory response with a memory request identifier stored in the packet memory and remove the associated memory request identifier from the packet memory.

19. A memory system, comprising:

a memory hub controller;

a plurality of memory modules, each memory module being coupled to adjacent memory modules through respective high-speed links, at least one of the memory modules being coupled to the memory hub controller through a respective high-speed link, and each memory module comprising;

a plurality of memory devices; and

a memory hub coupled to the memory devices, the memory hub comprising,

a decoder adapted to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;

a packet memory adapted to receive memory request identifiers and store the memory request identifiers;

a multiplexor adapted to receive remote memory responses that are responsive to the remote memory requests and local memory responses that are responsive to the local memory requests and being operable to select either the remote memory responses or the local memory responses in response to a control signal; and

arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal to control selection of which memory response to output based on the determination.

20. The memory system of claim 19 wherein each of the high-speed links comprises an optical communications link.

21. The memory system of claim 19 wherein at least some of the memory devices comprise SDRAMs.

22. The memory system of claim 19 wherein the arbitration control logic generates the control signal based on the age of the memory request identifiers stored in the packet memory.

23. The memory system of claim 22 wherein the arbitration control logic generates the control signal such that if an oldest memory request identifier stored in the packet memory is for a local memory request, the multiplexor outputs a local memory response.

24. The memory system of claim 19 wherein the packet memory is a first-in, first-out (FIFO) memory.

25. The memory system of claim 19 wherein each of the local and remote memory responses comprise data and a header identifying a memory request corresponding to the memory response.

26. The memory system of claim 19, further comprising a packet tracker adapted to receive the remote memory responses and being operable to associate each remote memory response with a memory request identifier stored in the packet memory and remove the associated memory request identifier from the packet memory.

27. A computer system, comprising:

a processor;

a system controller coupled to the processor, the system controller including a memory hub controller;

an input device coupled to the processor through the system controller;

an output device coupled to the processor through the system controller;

a storage device coupled to the processor through the system controller;

a plurality of memory modules, each memory module being coupled to adjacent memory modules through respective high-speed links, at least one of the memory modules being coupled to the memory hub controller through a respective high-speed link, and each memory module comprising;

a plurality of memory devices; and

a memory hub coupled to the memory devices and coupled to the corresponding high-speed links, the memory hub including,

a decoder adapted to receive memory requests including local memory requests directed to memory devices connected directly to the memory hub and remote memory requests directed to memory devices coupled to other memory hubs, the decoder being operable to determine a memory request identifier associated with each memory request;

a packet memory adapted to receive memory request identifiers and store the memory request identifiers;

a multiplexor adapted to receive remote memory responses that are responsive to the remote memory requests and local memory responses that are responsive to the local memory requests and being operable to select either the remote memory responses or the local memory responses in response to a control signal; and

arbitration control logic coupled to the multiplexor and the packet memory and being operable to determine from the memory request identifiers stored in the packet memory the recency of the memory requests corresponding to the received remote memory responses and the local memory responses and to generate the control signal to control selection of which memory response to output based on the determination.

28. The computer system of claim 27 wherein each of the high-speed links comprises an optical communications link.

29. The computer system of claim 27 wherein at least some of the memory devices comprise SDRAMs.

30. The computer system of claim 27 wherein each of the local and downstream memory responses comprise data and a header identifying a memory request corresponding to the memory response.

31. The computer system of claim 27 wherein the processor comprises a central processing unit (CPU).

32. The computer system of claim 27, further comprising a packet tracker adapted to receive the remote memory responses and being operable to associate each remote memory response with a memory request identifier stored in the packet memory and remove the associated memory request identifier from the packet memory.

33. In a memory system including a plurality of memory modules, each memory module including a memory hub coupled to memory devices, a method of processing and forwarding memory responses in the memory hub of each memory module, comprising:

receiving memory requests, each having a memory request identifier, the memory requests including local memory requests directed to memory devices connected to the memory hub and remote memory requests directed to memory devices coupled to memory hubs in other memory modules;

storing the memory request identifiers;

storing local memory responses received from the memory devices in response to the local memory requests;

storing remote memory responses received from the other memory modules in response to the remote memory requests;

applying in at least one hub an arbitration algorithm based on the ages of the stored memory request identifiers to determine an order in which the stored local and remote memory responses are forwarded; and

forwarding the local and remote memory responses upstream according to the determined order.

34. The method of claim 33 wherein each of the local and remote memory responses comprise data and a header identifying a memory request corresponding to the memory response.

35. The method of claim 33 further comprising generating a control signal to indicate the order based on an oldest stored memory request identifier.

36. The method of claim 33, wherein the memory request identifiers are stored and accessed on a first-in, first-out (FIFO) basis.

37. The method of claim 33, further comprising generating a control signal such that if an oldest stored memory request is a local memory request, a local memory response is forwarded.