Apparatus and method for memory read-refresh, scrubbing and variable-rate refresh

• US 8,024,638 B2
• Filed: 11/10/2006
• Issued: 09/20/2011
• Est. Priority Date: 05/20/2003
• Status: Expired due to Fees

• Current Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Original Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Inventors: Resnick, David R., Snyder, Van L., Higgins, Michael F.
• Primary Examiner(s): RIZK, SAMIR WADIE
• Application Number:
  US11/558,450
• Publication Number:
  US 20070101238A1
• Time in Patent Office: 1,775 Days
• Field of Search: 714/54, 714/754, 714/758, 365/189.04, 365/222, 365/230.04
• US Class Current: 714/758
• CPC Class Codes: G06F 11/106 Correcting systematically all correctable errors, i.e. scrubbing
• CPC Class Codes: G11C 11/401 forming cells needing refreshing or charge regeneration, i.e. dynamic cells
• CPC Class Codes: G11C 2029/0401 in embedded memories
• CPC Class Codes: G11C 29/44 Indication or identification of errors, e.g. for repair
• CPC Class Codes: G11C 29/4401 for self repair
• CPC Class Codes: G11C 29/848 by adjacent switching

Abstract

A memory controller and method that provide a read-refresh (also called “distributed-refresh”) mode of operation, in which every row of memory is read within the refresh-rate requirements of the memory parts, with data from different columns within the rows being read on subsequent read-refresh cycles until all rows for each and every column address have been read, scrubbing errors if found, thus providing a scrubbing function that is integrated into the read-refresh operation, rather than being an independent operation. For scrubbing, an atomic read-correct-write operation is scheduled. A variable-priority, variable-timing refresh interval is described. An integrated card self-tester and/or card reciprocal-tester is described. A memory bit-swapping-within-address-range circuit, and a method and apparatus for bit swapping on the fly and testing are described.

Claims

1. An information-processing apparatus comprising:

a first memory controller comprising;

a memory-chip interface that outputs memory addresses in a plurality of portions, including a first-address portion that is sufficient to refresh a set of addresses for a portion of a memory and a second-address portion that specifies one or more locations within the set of addresses; and

a refresh controller, coupled to the memory-chip interface, and configured to send read-refresh requests, wherein the read-refresh requests use refresh addresses that cycle through address-bit combinations for the first-address portion and also cycle through address-bit combinations for the second-address portion, and wherein read-refresh result data is fetched to the memory-chip interface as a result of each of the read-refresh requests, wherein the refresh controller further includes a priority controller that sends a first read-refresh request at an initial priority value, and later if the first read-refresh request has not been initiated, increases the priority value.

2. The apparatus of claim 1, further comprising:

an error detector, coupled to receive read-refresh results from the memory-chip interface and configured to detect one or more bit errors in the read-refresh result data;

an atomic read-correct-write (ARCW) controller coupled to the memory-request buffer, and configured, based on detection of an error by the error detector, to control an atomic read-correct-write operation to correct the detected error in a manner that is uninterrupted by other requests that could affect correction of the erroneous data; and

a memory-request buffer configured to hold a plurality of pending memory requests, wherein the error detector detects errors based on a SECDED ECC, and wherein the ARCW controller operates to temporarily prevent subsequent conflicting requests to the memory for a period of time sufficient to allow the atomic read-correct-write operation to effectively complete.

3. The apparatus of claim 1, further comprising

an error detector, coupled to receive read-refresh results from the memory-chip interface and configured to detect one or more bit errors in the read-refresh result data from an error-affected location;

and

an atomic read-correct-write (ARCW) controller coupled to the memory-request buffer, and configured, based on detection of an error by the error detector, to control an atomic read-correct-write operation to correct the detected error in a manner that is uninterrupted by other requests to the affected location.

4. The apparatus of claim 1, further comprising

a memory-request buffer, coupled to the memory-chip interface, and configured to hold a plurality of pending memory requests that are transmitted to the memory-chip interface.

5. The apparatus of claim 1, wherein the refresh controller is further configured to send explicit-refresh requests to the memory-request buffer, wherein the explicit-refresh requests are sent from the memory-chip interface to cause memory parts to perform an internally controlled refresh function.

6. The apparatus of claim 5, wherein the refresh controller further includes a timer controller that allows a time period between explicit-refresh requests to be varied.

7. The apparatus of claim 1, wherein the refresh controller further includes a timer controller that allows timing between read-refresh requests to be varied.

8. The apparatus of claim 1, further comprising

a high-speed serial external interface connected to receive and buffer a plurality of memory-requests from a processor for sending to the memory-chip interface, and to transmit read data obtained from the memory-chip interface to the processor.

9. The apparatus of claim 1, wherein the first address portion includes address bits for a row of data in a memory chip and the second address portion includes address bits for a column of data in the memory chip.

10. The apparatus of claim 1, further comprising a dynamic memory operatively coupled to the memory-chip interface, wherein the dynamic memory includes the portion of the memory having the set of addresses.

11. An information-processing apparatus comprising:

a first memory controller comprising;

a memory-chip interface that outputs memory addresses in a plurality of portions, including a first-address portion that is sufficient to refresh a set of addresses for a portion of a memory and a second-address portion that specifies one or more locations within the set of addresses,

a refresh controller, coupled to the memory-chip interface, and configured to send read-refresh requests, wherein the read-refresh requests use refresh addresses that cycle through address-bit combinations for the first-address portion and also cycle through address-bit combinations for the second-address portion, and wherein read-refresh result data is fetched to the memory-chip interface as a result of each of the read-refresh requests, and

a memory-request buffer, coupled to the memory-chip interface, and configured to hold a plurality of pending memory requests that are transmitted to the memory-chip interface; and

an arbitration circuit having one or more inputs connected to receive output of the buffer, wherein refresh requests are presented to one or more inputs of the arbitration circuit without passing through the buffer, such that the arbitration circuit can choose such a refresh request over other requests in the buffer when the refresh request has a higher priority.

12. The apparatus of claim 11, further comprising a dynamic memory operatively coupled to the memory-chip interface, wherein the dynamic memory includes the portion of the memory having the set of addresses.

13. An information-processing apparatus comprising:

a first memory controller comprising;

a memory-chip interface that outputs memory addresses in a plurality of portions, including a first-address portion that is sufficient to refresh a set of addresses for a portion of a memory and a second-address portion that specifies one or more locations within the set of addresses, and

a refresh controller, coupled to the memory-chip interface, and configured to send read-refresh requests, wherein the read-refresh requests use refresh addresses that cycle through address-bit combinations for the first-address portion and also cycle through address-bit combinations for the second-address portion, and wherein read-refresh result data is fetched to the memory-chip interface as a result of each of the read-refresh requests;

a plurality of high-speed serial external interfaces;

a second memory controller substantially the same as the first memory controller;

a first plurality of memory chips operatively coupled to the first memory controller;

a second plurality of memory chips operatively coupled to the second memory controller; and

a crossbar switch operatively coupled to transmit and receive memory commands and data to and from the first and second memory controllers, and to and from the plurality of high-speed serial external interfaces.

14. The apparatus of claim 13, further comprising:

a second memory card substantially the same as the first memory card;

a first plurality of processors coupled to the first memory card and to the second memory card;

a third and fourth memory card each substantially the same as the first memory card;

a second plurality of processors coupled to the third memory card and to the fourth memory card;

a network operatively coupled between the first plurality of processors and the second plurality of processors; and

an input/output system operatively coupled to provide data to and from the first plurality of processors.

15. The apparatus of claim 13, further comprising:

a first plurality of processors coupled to the plurality of high-speed serial external interfaces;

a network operatively coupled to each of the first plurality of processors;

an input/output system operatively coupled to provide data to and from the first plurality of processors; and

a power supply operatively coupled to provide power to the first plurality of processors.

16. The apparatus of claim 13, further comprising a dynamic memory operatively coupled to the memory-chip interface, wherein the dynamic memory includes the portion of the memory having the set of addresses.

17. An information-processing method comprising:

sending a stream of processor memory requests to memory parts;

inserting a read-refresh request periodically into the stream of processor memory requests, wherein the periodic read-refresh requests are sent using refresh addresses that cycle through a first set of address bits sufficient for refreshing the memory parts and also cycle through a second set of address bits sufficient to read substantially all data locations in the memory parts;

fetching data as a result of each of the read-refresh requests;

sending a first read-refresh request at an initial priority value; and

later, if the first read-refresh request has not been completed, increasing the priority value.

18. The method of claim 17, further comprising:

detecting an error in the fetched read-refresh data;

preventing already-pending buffered memory requests from starting for a period of time;

preventing further memory requests from being accepted for a period of time; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched read-refresh data, to correct the detected error in a manner that is uninterrupted by other memory requests.

19. The method of claim 18, wherein the detecting of the error includes detecting a single-bit error in the fetched read-refresh data based on a SECDED ECC.

20. The method of claim 17, further comprising:

initially fetching processor data from a first location as a result of one of the pending processor memory requests;

detecting an error in the fetched processor-requested data;

preventing others of already pending buffered memory requests from starting for a period of time;

preventing further memory requests from being accepted for a period of time; and

performing an atomic read-correct-write (ARCW) operation based on detecting the error, including again fetching data from the first location, in order to correct the detected error in a manner that is uninterrupted by other memory requests.

21. The method of claim 20, further comprising correcting the detected error in the initially fetched processor data based on a SECDED ECC, and sending the corrected data to the processor that requested the data.

22. The method of claim 17, further comprising:

detecting an error in the fetched processor data; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched processor data, in order to correct the detected error in a manner that is uninterrupted by other memory requests.

23. The method of claim 17, further comprising inserting an explicit-refresh request periodically into the stream of memory requests to cause memory parts to perform an internally controlled refresh function.

24. The method of claim 23, further comprising varying a value of time between explicit-refresh requests.

25. The method of claim 17, further comprising varying a value of time between read-refresh requests.

26. The method of claim 17, further comprising:

detecting an error in the fetched read-refresh data;

preventing further memory requests from starting for a period of time;

preventing further memory requests from being accepted for a period of time; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched read-refresh data, to correct the detected error in a manner that is uninterrupted by conflicting requests while non-conflicting requests continue.

27. The method of claim 17, further comprising:

fetching processor data as a result of one of the pending processor memory requests;

detecting an error in the fetched processor data;

preventing further memory requests from starting for a period of time;

preventing further memory requests from being accepted for a period of time; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error, to correct the detected error in a manner that is uninterrupted by conflicting requests while non-conflicting requests continue.

28. The method of claim 17, further comprising receiving memory requests from a processor across a high-speed serial external interface;

sending the memory requests to memory parts; and

transmitting data obtained from the memory parts to the processor.

29. The method of claim 17, further comprising:

buffering a plurality of pending memory requests from a processor.

30. The method of claim 17, further comprising:

detecting an error in the fetched read-refresh data; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched read-refresh data, in order to correct the detected error.

31. The method of claim 30, further comprising:

detecting an error in the fetched processor data; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched processor data, in order to correct the detected error in a manner that is uninterrupted by other memory requests.

34. The method of claim 17, further comprising:

fetching processor data as a result of one of the pending processor memory requests;

detecting an error in the fetched processor data based on a SECDED ECC;

allowing already pending buffered memory requests to complete;

preventing further memory requests from being accepted for a period of time; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error, to correct the detected error in a manner that is uninterrupted by other memory requests.

32. An information-processing method comprising:

sending a stream of processor memory requests to memory parts;

inserting a read-refresh request periodically into the stream of processor memory requests, wherein the periodic read-refresh requests are sent using refresh addresses that cycle through a first set of address bits sufficient for refreshing the memory parts and also cycle through a second set of address bits sufficient to read substantially all data locations in the memory parts;

fetching data as a result of each of the read-refresh requests;

receiving memory requests from a plurality of processors across a plurality of high-speed serial external interfaces, wherein the buffering of pending memory requests is divided among a plurality of buffers including a first and a second buffer;

sending memory requests from the first buffer to a first plurality of memory chips;

sending memory requests from the second buffer to a second plurality of memory chips; and

crossbar switching to transmit and receive memory commands and data to and from the first and second buffers, and to and from the plurality of high-speed serial external interfaces.

33. The method of claim 32, wherein the method is performed on a single first memory card.

35. An information-processing system comprising:

a plurality of memory chips;

a memory-request buffer, coupled to the memory chips, and configured to hold one or more pending memory requests from a processor that are transmitted to the memory chips; and

means for read-refreshing the memory chips and fetching read-refresh data as a result, wherein the means for read-refreshing includes means for sending a first read-refresh request at an initial priority value, and later if the first read-refresh request has not been initiated, for increasing the priority value.

36. The system of claim 35, further comprising:

an error detector operatively coupled to receive read-refresh data from the memory chips; and

means for performing an atomic read-correct-write (ARCW) operation to correct a detected error, based on detecting the error in the read-refresh data.

37. The system of claim 35, further comprising:

an error detector operatively coupled to receive processor data from the memory chips; and

means for performing an atomic read-correct-write (ARCW) operation to correct a detected error, based on detecting the error in fetched processor data.

38. The system of claim 35, further comprising means for explicitly refreshing the memory chips.

39. The system of claim 35, further comprising means for explicitly refreshing the memory chips in a manner periodically alternated with the read-refreshing of the memory chips.

40. The system of claim 35, further comprising:

means for detecting an error in the fetched read-refresh data based on a SECDED ECC;

means for allowing already pending buffered memory requests to complete;

means for preventing further memory requests from being accepted for a period of time; and

means for performing an atomic read-correct-write (ARCW) operation, based on detecting the error, to correct the detected error in a manner that is uninterrupted by other memory requests.

41. An information-processing system comprising:

a dynamic memory, wherein the dynamic memory includes a first plurality of memory locations, and wherein a sufficient subset of the first plurality of memory locations must each be accessed within a refresh-period amount of time;

a memory controller, coupled to the memory, and configurable to perform a first read operation to each and every one of the first plurality of memory locations, each first read operation causing corresponding read-refresh data to be fetched in a pattern that ensures that the sufficient subset is read within the refresh-period amount of time, wherein the memory controller includes a priority controller that sends a first read-refresh request at an initial priority value, and later if the first read-refresh request has not been initiated, increases the priority value; and

an error detector operatively coupled to receive and check for errors in the corresponding read-refresh data.

42. The system of claim 41, further comprising

an atomic read-correct-write (ARCW) unit configured to correct data at a location having a detected error, based on results from the error detector'"'"'s check of the read-refresh data.

43. The system of claim 42, wherein the ARCW unit is configured to atomically perform:

a second read operation to the location having the detected error,

a correction of the error, and

a write operation to the location having the detected error.

44. The system of claim 42, wherein the ARCW unit further includes:

a hold circuit that temporarily halts from starting further ones of at least certain memory operations that otherwise would start;

a read circuit that causes a second read operation to the location having the detected error;

an SECDED ECC circuit that corrects an error in data fetched by the second read operation based on a SECDED ECC to obtain corrected data; and

a write circuit that causes a write operation with the corrected data to the location having the detected error, whereupon the hold circuit again allows the halted memory operations to start.

45. The system of claim 44, further including:

a memory request buffer configured to hold a plurality of memory operations, and coupled to the hold circuit such that the certain ones of the memory operations halted are those in the buffer; and

a circuit that prevents further memory requests from being accepted into the buffer for a period of time.

46. The system of claim 41, wherein the memory controller is also configurable to perform explicit refreshes of the memory.