Hybrid NUMA COMA caching system and methods for selecting between the caching modes

• US 5,893,144 A
• Filed: 12/22/1995
• Issued: 04/06/1999
• Est. Priority Date: 12/22/1995
• Status: Expired due to Term

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Wood, David, Hagersten, Erik
• Primary Examiner(s): Swann, Tod R.
• Assistant Examiner(s): Tran, Denise
• Application Number:
  US08/575,787
• Time in Patent Office: 1,201 Days
• Field of Search: 395/471, 395/449, 395/470, 395/445, 395/446, 395/447, 395/448, 395/468, 395/472, 364/243.41, 364/243.44, 364/243.45, 711/118-122, 711/141, 711/143-145, 711/130, 711/147-148
• US Class Current: 711/122
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0813 with a network or matrix configuration
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 2212/2542 Non-uniform memory access [NUMA] architecture
• CPC Class Codes: G06F 2212/272 Cache only memory architecture [COMA]

Abstract

The present invention provides a hybrid Non-Uniform Memory Architecture (NUMA) and Cache-Only Memory Architecture (COMA) caching architecture together with a cache-coherent protocol for a computer system having a plurality of sub-systems coupled to each other via a system interconnect. In one implementation, each sub-system includes at least one processor, a page-oriented COMA cache and a line-oriented hybrid NUMA/COMA cache. Such a hybrid system provides flexibility and efficiency in caching both large and small, and/or sparse and packed data structures. Each sub-system is able to independently store data in COMA mode or in NUMA mode. When caching in COMA mode, a sub-system allocates a page of memory space and then stores the data within the allocated page in its COMA cache. Depending on the implementation, while caching in COMA mode, the sub-system may also store the same data in its hybrid cache for faster access. Conversely, when caching in NUMA mode, the sub-system stores the data, typically a line of data, in its hybrid cache.

Claims

1. A method of storing data in a computer system having a plurality of sub-systems coupled to each other by a system interconnect, each said sub-system including a processor, a hybrid non-uniform-memory-architecture/cache-only-memory-architecture(NUMA/COMA) cache, a COMA cache and a directory, said method comprising the steps of:

a designated processor of one said subsystem initiating a data transaction associated with a data line;

determining whether a caching mode for said data line is a COMA mode or a NUMA mode;

storing data associated with said data line in said COMA cache of said one sub-system in response to determining that said caching mode for said data line is said COMA mode; and

storing said data in the hybrid NUMA/COMA cache of said one sub-system.

2. The method of claim 1 wherein if said determining step determines that the data should also be stored in said COMA cache in the COMA mode, then the method further comprises the steps of:

allocating a page of memory space in said COMA cache of said one sub-system; and

storing said data within said allocated page of said COMA cache.

3. The method of claim 1 further comprising determining that said data stored in said hybrid NUMA/COMA cache of said one sub-system was previously discarded by said one sub-system a predetermined number of times due to a capacity miss or a conflict miss.

4. The method of claim 3 further comprising the steps of:

writing-back said data to a home sub-system;

marking said one sub-system as a last owner of said data; and

discarding said data stored in said hybrid NUMA/COMA cache of said one sub-system.

5. The method of claim 4 wherein upon a subsequent read-to-own (RTO) or read-to-share (RTS) request for said data by said one sub-system, the method further comprising the steps of:

detecting that said one sub-system was the last owner of said data;

retrieving said data from said home sub-system; and

incrementing a NUMA_-- miss counter.

6. The method of claim 5 wherein said NUMA_-- miss counter is associated with a page of memory which includes said data line.

7. The method of claim 5 wherein if said NUMA_-- miss counter exceeds a predetermined threshold, then the method further includes the steps of:

assigning said caching mode for said data line to said COMA mode;

allocating a page of memory space in said COMA cache of said one sub-system; and

storing said data line within said allocated page of said COMA cache upon a subsequent access to said data line.

8. The method of claim 3 further comprising the steps of:

marking said one sub-system as a sharer of said data; and

discarding said data stored in said hybrid NUMA/COMA cache of said one sub-system without re-marking said one sub-system.

9. The method of claim 8 wherein upon a subsequent read-to-share (RTS) request for said data by said one sub-system, the method further comprising the steps of:

detecting that said one sub-system was the sharer of said data;

retrieving said data from said home sub-system; and

incrementing a NUMA_-- miss counter.

10. The method of claim 9 wherein said NUMA_-- miss counter is associated with a page of memory which includes said data line.

11. The method of claim 9 wherein said determining whether a caching mode for said data line is a COMA mode or a NUMA mode further comprises checking an address associated with said data transaction to determine whether said caching mode is said COMA mode or said NUMA mode.

12. The method of claim 11 wherein a default initial caching mode for said data line is said NUMA mode.

13. The method of claim 9 wherein if said NUMA_-- miss counter exceeds a predetermined threshold, then the method further includes the steps of:

assigning said caching mode for said data line to said COMA mode;

allocating a page of memory space in said COMA cache of said one sub-system; and

storing said data line within said allocated page of said COMA cache upon a subsequent access to said data line.

14. The method of claim 1 wherein said determining whether a caching mode for said data line is a COMA mode or a NUMA mode further comprises checking an address associated with said data transaction to determine whether said caching mode is said COMA mode or said NUMA mode.

15. The method of claim 1 wherein a default initial caching mode for said data line is said NUMA mode.

16. A computer program product including a computer-usable medium having computer-readable code embodied therein configured to cause a computer system to select between a non-uniform-memory-architecture (NUMA) mode and a cache-only-memory-architecture (COMA) mode, the computer system having a plurality of sub-systems coupled to each other by a system interconnect, each said sub-system including a processor, a hybrid NUMA/COMA cache, a COMA cache and a directory, the computer-readable code comprising:

a routine configured to cause one said sub-system to store data associated with a data line in the hybrid NUMA/COMA cache of said one sub-system; and

a routine configured to cause said one sub-system to determine whether a caching mode for the data is a COMA mode or a NUMA mode to thereby cause the data to be stored in the COMA cache of said one sub-system if said caching mode for the data is said COMA mode.

17. The computer program product of claim 16 wherein the computer-readable code further comprises:

a routine configured to cause said one sub-system to determine if said data stored in said hybrid NUMA/COMA cache of said one sub-system was previously discarded by said one sub-system a predetermined number of times due to a capacity miss or a conflict miss.