Enhanced protection for memory modification tracking with redundant dirty indicators

• US 6,950,907 B2
• Filed: 08/24/2001
• Issued: 09/27/2005
• Est. Priority Date: 11/29/2000
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Harris, Jeremy Graham, Garnett, Paul Jeffrey
• Primary Examiner(s): KIM, HONG CHONG
• Application Number:
  US09/939,077
• Publication Number:
  US 20020065986A1
• Time in Patent Office: 1,495 Days
• Field of Search: 711/120, 711/144, 711/145, 711/133, 711/158, 711/159, 711/162, 711/117, 711/172, 714/6, 714/13, 714/43, 710/22
• US Class Current: 711/144
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
• CPC Class Codes: G06F 11/1641 where the comparison is not performed by the redundant processing components
• CPC Class Codes: G06F 11/165 with continued operation after detection of the error
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/167 Error detection by comparing the memory output
• CPC Class Codes: G06F 11/1679 at clock signal level
• CPC Class Codes: G06F 13/4027 using bus bridges G06F13/4022 takes precedence

Abstract

A dirty memory subsystem includes storage operable to store redundant copies of dirty indicators. Each dirty indicator is associated with a respective block of main memory and is settable to a predetermined state to indicate that the block of main memory associated therewith has been dirtied. By providing redundant storage for the dirty indicators, any difference between the stored copies of the dirty indicators can be considered as indicative of memory corruption, for example as a result of a cosmic ray impact. As the different copies can be stored in different locations, it is unlikely that a cosmic ray impact would affect all copies equally. If a difference between the stored copies is detected, then the dirty indicator can be take as being unreliable and remedial action can be taken. For example, it can be assumed that a block of main memory has been dirtied if any of the copies of the dirty indicator has the predetermined state. Such a memory finds application in a method of managing reinstatement of an equivalent memory state in the main memory of a plurality of processing sets of a fault tolerant computer following a lock step error.

Claims

1. A dirty memory subsystem for a computer system, the dirty memory subsystem comprising:

storage operable to store dirty indicators and corresponding redundant copies of the dirty indicators, wherein each dirty indicator and the corresponding redundant dirty indicator being associated with a respective block of main memory and being settable to a predetermined state to indicate that the block of main memory associated therewith has been dirtied; and

a control logic operable to read the dirty indicator and the corresponding redundant dirty indicator from storage and to treat the block of memory associated therewith as dirtied if all or any one of the dirty indicator and the corresponding redundant dirty indicator has the predetermined state.

2. The dirty memory subsystem of claim 1, wherein the dirty indicators being stored in groups with each group having associated therewith a parity indicator computed from the dirty indicator values of the group, wherein the control logic being operable on reading the dirty indicators of a group and a corresponding parity indicator to calculate a parity indicator value based on the dirty indicator values read for the group to determine an integrity of the group, wherein the control logic is configured to identify all dirty indicators of the group as representing a dirtied state where the calculated parity indicator value is different from the parity indicator value read for that group.

3. The dirty memory subsystem of claim 1, wherein in response to all or any of the dirty indicator and the corresponding redundant dirty indicator having the predetermined state, the control logic is operable to cause the block of memory associated with the dirty indicator and the corresponding redundant dirty indicator to be copied from the main memory to another memory.

4. The dirty memory subsystem of claim 3, wherein the other memory is another main memory.

5. The dirty memory subsystem of claim 1, operable to reset each of the dirty indicator and the corresponding redundant dirty indicator to a state other than the predetermined state after reading the dirty indicator and the corresponding redundant dirty indicator.

6. The dirty memory subsystem of claim 1, wherein the storage comprises at least two memory units, one memory unit for storing the dirty indicators and another memory unit for storing the corresponding redundant dirty indicators.

7. The dirty memory subsystem of claim 6, wherein the control logic includes comparison logic for each of the memory units for determining whether all or any of the dirty indicator and the corresponding redundant dirty indicator is set to the predetermined state.

8. The dirty memory subsystem of claim 7, wherein the control logic includes output logic responsive to the output of each comparison logic for determining whether a block of memory associated with a dirty indicator and the corresponding redundant dirty indicator is to be treated as dirtied.

9. The dirty memory subsystem of claim 1, wherein each dirty indicator comprises a single bit.

10. The dirty memory subsystem of claim 1, wherein a block of main memory is a page of main memory.

11. The dirty memory subsystem of claim 1, wherein the storage comprises a lower level memory that includes groups of the dirty indicators and groups of the corresponding redundant copies of the dirty indicators, and at least one higher level memory that includes groups of dirty group indicators and groups of corresponding redundant copies of the dirty group indicators, wherein each dirty group indicator and a corresponding redundant dirty group indicator being settable to a given state indicative that a group of dirty indicators of the lower level memory has at least one dirty indicator in the predetermined state indicative that a block of memory associated therewith has been dirtied.

12. The dirty memory subsystem of claim 11, wherein the control logic is operable to read a dirty group indicator and a corresponding redundant dirty group indicator from the higher level memory and to treat the group of dirty indicators associated therewith as dirtied in response to all or any one of the dirty group indicator and the corresponding redundant dirty group indicator having the given state.

13. The dirty memory subsystem of claim 12, wherein, in response to all or any one of the dirty group indicator and the corresponding redundant dirty group indicator having the given state, for each dirty indicator in the group of dirty indicators and for each corresponding redundant dirty indicator of the group of corresponding redundant dirty indicators, the control logic is operable to read the dirty indicator and the corresponding redundant dirty indicator from the lower level memory and to treat the block of memory associated therewith as dirtied in response to all or any one of the dirty indicator and the corresponding redundant dirty indicator having the predetermined state.

14. The dirty memory subsystem of claim 1, wherein, after both the dirty indicator and the corresponding redundant dirty indicator are read from storage, any difference between a state of the stored dirty indicator and corresponding dirty indicator is considered as indicative of memory corruption.

15. A computer system comprising a dirty memory subsystem and at least one processing set that includes main memory, the dirty memory subsystem comprising:

storage operable to store dirty indicators and corresponding redundant copies of the dirty indicators, wherein each dirty indicator and the corresponding redundant dirty indicator being associated with a respective block of main memory and being settable to a predetermined state to indicate that the block of main memory associated therewith has been dirtied; and

a control logic operable to read the dirty indicator and the corresponding redundant dirty indicator from storage and to treat the block of memory associated therewith as dirtied if all or any one of the dirty indicator and the corresponding redundant dirty indicator has the predetermined state.

16. The computer system of claim 15, comprising a plurality of processing sets that each include main memory.

17. The computer system of claim 16, wherein the processing sets are operable in lockstep, the computer system comprising logic operable to attempt to reinstate an equivalent memory state in the main memory of each of the processor following a lockstep error.

18. A method of managing reinstatement of an equivalent memory state in the main memory of a plurality of processing sets of a fault tolerant computer following a lock step error, wherein a dirty memory subsystem stores dirty indicators and corresponding redundant copies of the dirty indicators that are settable to a predetermined state indicative that a block of main memory associated therewith has been dirtied, the method including the performance of at least one cycle of copying any page of main memory that has been dirtied from a first processing set to each other processing set, each cycle including reading a dirty indicator and a corresponding redundant indicator from storage and treating a block of memory associated with the dirty indicator and the corresponding redundant dirty indicator as dirtied if all or any one of the dirty indicator and the corresponding redundant dirty indicator has the predetermined state.

19. The method of claim 18, further comprising in response to all or any of the dirty indicator and the corresponding redundant dirty indicator having the predetermined state, copying a block of memory associated with the dirty indicator and the corresponding redundant dirty indicator from the main memory to another memory.

20. The method of claim 19, wherein the other memory is another main memory.

21. The method of claim 18, further comprising resetting each of the dirty indicator and the corresponding redundant dirty indicator to a state other than the predetermined state after reading the dirty indicator and the corresponding redundant dirty indicator.

22. The method of claim 18, comprising maintaining the dirty indicators in one memory unit and the corresponding redundant dirty indicators in another memory unit.

23. The method of claim 22, comprising separately assessing the state of a dirty indicator and a corresponding redundant dirty indicator for each of the memory units for determining whether all or any one of the dirty indicator and the corresponding redundant dirty indicator is set to the predetermined state.

24. The method of claim 18, wherein the storage comprises a lower level memory that includes groups of the dirty indicators and groups of the corresponding redundant copies of the dirty indicators, and at least one higher level memory that includes groups of dirty group indicators and groups of corresponding redundant copies of the dirty group indicators, wherein each dirty group indicator and a corresponding redundant dirty group indicator being settable to a given state indicative that a group of dirty indicators of the lower level memory has at least one dirty indicator in the predetermined state indicative that a block of memory associated therewith has been dirtied.

25. The method of claim 24, further comprising reading a dirty group indicator and a corresponding redundant dirty group indicator from the higher level memory and treating the group of dirty indicators associated therewith as dirtied in response to all or any one of the dirty group indicator and the corresponding redundant dirty group indicator having the given state.

26. The method of claim 18, wherein each dirty indicator comprises a single bit.

27. The method of claim 18, wherein a block of main memory is a page of main memory.

28. A dirty memory subsystem for a computer system, the dirty memory subsystem comprising:

a lower level memory operable to store groups of dirty indicators and groups of corresponding redundant copies of the dirty indicators, wherein each dirty indicator and the corresponding redundant dirty indicator being associated with a respective block of main memory and being settable to a predetermined state to indicate that the block of main memory associated therewith has been dirtied;

at least one higher level memory operable to store groups of dirty group indicators and groups of corresponding redundant copies of the dirty group indicators, wherein each dirty group indicator and a corresponding redundant dirty group indicator being settable to a given state indicative that a group of dirty indicators of the lower level memory has at least one dirty indicator in the predetermined state indicative that a block of memory associated therewith has been dirtied;

a control logic operable to read a dirty group indicator and a corresponding redundant dirty group indicator from the higher level memory and to treat the group of dirty indicators associated therewith as dirtied in response to all or any one of the dirty group indicator and the corresponding redundant dirty group indicator having the given state; and

wherein, in response to all or any one of the dirty group indicator and the corresponding redundant dirty group indicator having the given state, for each dirty indicator in the group of dirty indicators and for each corresponding redundant dirty indicator of the group of corresponding redundant dirty indicators, the control logic is operable to read the dirty indicator and the corresponding redundant dirty indicator from the lower level memory and to treat the block of memory associated therewith as dirtied in response to all or any one of the dirty indicator and the corresponding redundant dirty indicator having the predetermined state.