APPARATUS AND METHOD FOR MEMORY ASYNCHRONOUS ATOMIC READ-CORRECT-WRITE OPERATION

• US 20070113150A1
• Filed: 11/10/2006
• Published: 05/17/2007
• Est. Priority Date: 05/20/2003
• Status: Active Grant

• Current Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Original Assignee: CRAY Incorporated (Hewlett-Packard Enterprise Company)
• Inventors: Resnick, David, Marquardt, Kelly, Grossmeier, Alan, Higgins, Michael, Schwoerer, Gerald, Snyder, Van
• Granted Patent: US 7,676,728 B2
• Time in Patent Office: Days
• US Class Current: 714/763
• CPC Class Codes: G06F 11/106 Correcting systematically all correctable errors, i.e. scrubbing
• CPC Class Codes: G11C 11/401 forming cells needing refreshing or charge regeneration, i.e. dynamic cells
• CPC Class Codes: G11C 2029/0401 in embedded memories
• CPC Class Codes: G11C 29/44 Indication or identification of errors, e.g. for repair
• CPC Class Codes: G11C 29/4401 for self repair
• CPC Class Codes: G11C 29/848 by adjacent switching

Abstract

A memory controller and method that provide a read-refresh (also called “distributed-refresh”) mode of operation, in which every row of memory is read within the refresh-rate requirements of the memory parts, with data from different columns within the rows being read on subsequent read-refresh cycles until all rows for each and every column address have been read, scrubbing errors if found, thus providing a scrubbing function that is integrated into the read-refresh operation, rather than being an independent operation. For scrubbing, an atomic read-correct-write operation is scheduled. A variable-priority, variable-timing refresh interval is described. An integrated card self-tester and/or card reciprocal-tester is described. A memory bit-swapping-within-address-range circuit, and a method and apparatus for bit swapping on the fly and testing are described.

Claims

1. An information-processing apparatus comprising:

a memory;

an error detector, coupled to receive read data from the memory and configured to detect one or more bit errors in the read data; and

an atomic read-correct-write (ARCW) controller coupled to the memory, and configured, based on detection of an error by the error detector, to control an atomic read-correct-write operation to correct the detected error at its respective address.

2. The apparatus of claim 1, further comprising

a memory-request buffer, coupled to the memory, and configured to hold one or more pending memory requests that go to the memory.

3. The apparatus of claim 2, wherein the memory-request buffer is configured to hold a plurality of pending memory requests, wherein the error detector detects errors based on a SECDED ECC, and wherein the ARCW controller operates to inhibit memory operations from the memory-request buffer, at least to a memory area to which the ARCW is directed, and prevent further memory requests from being issued for a period of time to allow the ARCW operation to effectively complete.

4. The apparatus of claim 1, further comprising

a refresh controller, coupled to the memory, and configured to send read-refresh requests the read-refresh requests using refresh addresses that cycle through address bits for a first-address portion of the refresh address needed to specify the refresh and also cycle through bits for a second-address portion of the refresh address not needed to specify the refresh, and wherein data is fetched to the memory-chip interface and checked by the error detector as a result of each one of a plurality of the read-refresh requests.

5. The apparatus of claim 4, wherein the first-address portion includes address bits for a row of data in a memory chip and the second-address portion includes address bits for a column of data in the memory chip.

6. The apparatus of claim 4, wherein the refresh controller is further configured to send explicit-refresh requests to the memory, wherein the explicit-refresh requests are sent to the memory to cause the memory to perform an internally controlled refresh function.

7. The apparatus of claim 6, wherein the refresh controller sends a plurality of explicit-refresh requests over a first period of time without intervening read-refresh requests, then a plurality of read-refresh requests over a second period of time without intervening explicit-refresh requests, then sends a plurality of explicit-refresh requests over a third period of time, and then a plurality of read-refresh requests over a fourth period of time.

8. The apparatus of claim 6, wherein the refresh controller further includes a timer controller that allows the timing between explicit-refresh requests to be varied.

9. The apparatus of claim 6, wherein the refresh controller further includes a priority controller that sends a first read-refresh request at an initial priority value, and later if the first read-refresh request has not been completed, increases the priority value.

10. The apparatus of claim 6, wherein the refresh controller further includes a priority controller that sends a first read-refresh request specifying a first address and at an initial priority value, and later if the first read-refresh request has not been completed, then sends a replacement read-refresh request specifying the first address and at a higher priority value.

11. The apparatus of claim 4, further comprising an arbitration circuit having one or more inputs connected to receive output of the buffer, wherein refresh requests are presented to one or more inputs of the arbitration circuit without passing through the buffer, such that the arbitration circuit can chose such a refresh request over other requests in the buffer when the refresh request has a higher priority.

12. The apparatus of claim 1, further comprising:

a plurality of high-speed serial external interfaces;

a second memory controller substantially the same as the first memory controller;

a first plurality of memory chips operatively coupled to the first memory controller;

a second plurality of memory chips operatively coupled to the second memory controller; and

a crossbar switch operatively coupled to transmit and receive memory commands and data to and from the first and second memory controllers, and to and from the plurality of high-speed serial external interfaces.

13. The apparatus of claim 12, further comprising:

a second memory card substantially the same as the first memory card;

a first plurality of processors coupled to the first memory card and to the second memory card;

a third and fourth memory card each substantially the same as the first memory card;

a second plurality of processors coupled to the third memory card and to the fourth memory card;

a network operatively coupled between the first plurality of processors and the second plurality of processors; and

an input/output system operatively coupled to provide data to and from the first plurality of processors.

14. The apparatus of claim 13, further comprising:

a first plurality of processors coupled to the plurality of high-speed serial external interfaces;

a network operatively coupled to each of the first plurality of processors;

an input/output system operatively coupled to provide data to and from the first plurality of processors; and

a power supply operatively coupled to provide power to the first plurality of processors.

24. The method of claim 14, further comprising:

receiving memory requests from a plurality of processors across a plurality of high-speed serial external interfaces, wherein the buffering of pending memory requests is divided among a plurality of buffers including a first and a second buffer;

sending memory requests from the first buffer to a first plurality of memory chips;

sending memory requests from the second buffer to a second plurality of memory chips; and

crossbar switching to transmit and receive memory commands and data to and from the first and second buffers, and to and from the plurality of high-speed serial external interfaces.

15. An information-processing method comprising:

buffering a plurality of pending memory requests from a processor;

sending a stream of processor memory requests from the buffered pending memory requests to a memory;

fetching data based on a first memory request;

detecting an error in the fetched data; and

performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched data, to correct the detected error.

16. The method of claim 15, further comprising:

inserting a read-refresh request periodically into the stream of processor memory requests, wherein the periodic read-refresh requests are sent using refresh addresses that cycle through address bits for a first-address portion and also cycle through bits for a second-address portion; and

fetching data as a result of each of the read-refresh requests.

17. The method of claim 16, further comprising varying a value of time between read-refresh requests.

18. The method of claim 16, further comprising varying a priority of the read-refresh requests.

19. The method of claim 16, further comprising inserting an explicit-refresh request periodically into the stream of memory requests to cause memory parts to perform an internally controlled refresh function.

20. The method of claim 19, further comprising:

inserting an explicit-refresh request periodically over a first period of time without intervening read-refresh requests, then

inserting a read-refresh request periodically over a second period of time without intervening explicit-refresh requests, then

inserting an explicit-refresh request periodically over a third period of time, and then,

inserting a read-refresh request periodically over a fourth period of time.

21. The method of claim 20, further comprising varying a value of time between explicit-refresh requests.

22. The method of claim 16, further comprising:

sending a first read-refresh request at an initial priority value, and

later, if the first read-refresh request has not been completed, increasing the priority value.

23. The method of claim 16, further comprising:

sending a first read-refresh request that specifies a first address and an initial priority value, and

later, if the first read-refresh request has not been completed, then sending a replacement read-refresh request specifying the first address and a higher priority value.

25. The method of claim 16, further comprising:

arbitrating between memory requests from one or more processors in a buffer, and refresh requests presented without passing through the buffer, such that the arbitrating chooses such a refresh request over other requests in the buffer when the refresh request has a higher priority.

26. The method of claim 15, wherein the performing of the ARCW operation further includes:

inhibiting execution of already-pending buffered memory requests; and

preventing further memory requests from being issued for a period of time.

27. The method of claim 16, wherein the first-address portion includes address bits for a row of data in a memory chip and the second-address portion includes address bits for a column of data in the memory chip.

29. The system of claim 27, further comprising means for read-refreshing the memory and fetching read-refresh data as a result.

30. The system of claim 29, further comprising means for varying timing between read-refresh requests.

31. The system of claim 29, wherein read-refresh requests have a priority, and further comprising means for varying the priority of the read-refresh requests.

32. The system of claim 29, further comprising means for explicitly refreshing the memory instead of read-refreshing the memory.

33. The system of claim 29, further comprising means for explicitly refreshing the memory in a manner periodically alternated with the read-refreshing of the memory.

34. The system of claim 27, further comprising means for explicitly refreshing the memory instead of read-refreshing the memory.

35. The system of claim 34, further comprising means for varying timing between explicit-refresh requests.

36. The system of claim 34, wherein explicit-refresh requests have a priority, and wherein the system further comprises means for varying the priority of the explicit requests.

37. The system of claim 27, further comprising:

means for inhibiting already pending buffered memory requests from executing; and

means for preventing further memory requests from being issued for a period of time.

28. An information-processing system comprising:

a memory;

a memory-request buffer, coupled to the memory, and configured to hold one or more pending memory requests from a processor that are transmitted to the memory while another memory request is executing in the memory; and

means for detecting an error in the fetched data and for performing an atomic read-correct-write (ARCW) operation, based on detecting the error in the fetched data, to correct the detected error.

38. An information-processing system comprising:

a memory;

a memory controller configured to send a first read operation to the memory and to receive read data from a first location specified by the first read operation;

an error detector, configured to check the received read data and to detect one or more bit errors in the read data; and

an atomic read-correct-write (ARCW) controller coupled to the memory controller, and configured, based on detection of an error by the error detector, to control an atomic read-correct-write operation that includes, atomically, a second read operation to the first location, a correction operation, and a write operation to the first location.

39. The system of claim 38, further comprising a memory-request buffer, coupled to the memory, and configured to hold one or more pending memory requests that go to the memory.

40. The system of claim 39, wherein the memory-request buffer is configured to hold a plurality of pending memory requests, and wherein the ARCW controller operates to inhibit memory operations from the memory-request buffer, at least to a memory area to which the ARCW is directed, and prevent further memory requests from being issued for a period of time to allow the ARCW operation to effectively complete.

41. The system of claim 40, wherein the error detector detects errors based on a SECDED ECC.

42. The system of claim 38, further comprising a refresh controller, coupled to the memory, and configured to send read-refresh requests the read-refresh requests using refresh addresses that cycle through address bits for a first-address portion of the refresh address and also cycle through bits for a second-address portion of the refresh address, and wherein data is fetched to the memory-chip interface and checked by the error detector as a result of each one of a plurality of the read-refresh requests.