Shared memory multiprocessing system employing mixed broadcast snooping and directory based coherency protocols

• US 20030018739A1
• Filed: 05/01/2002
• Published: 01/23/2003
• Est. Priority Date: 05/01/2001
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Singhal, Ashok, Cypher, Robert
• Granted Patent: US 7,032,078 B2
• Time in Patent Office: Days
• US Class Current: 709/213
• CPC Class Codes: G06F 12/0817 using directory methods
• CPC Class Codes: G06F 12/0831 using a bus scheme, e.g. with bus monitoring or watching means

Abstract

A multiprocessor computer system is configured to selectively transmit address transactions using either a broadcast mode or a point-to-point mode. Depending on the mode of transmission selected, either a directory-based coherency protocol or a broadcast snooping coherency protocol is implemented to maintain coherency within the system. A computing node is formed by a group of clients which share a common address and data network. The address network is configured to determine whether a particular transaction is to be conveyed in broadcast mode or point-to-point mode. In one embodiment, the address network includes a mode table with entries which are configurable to indicate transmission modes corresponding to different regions of the address space within the node. Upon receiving a coherence request transaction, the address network may then access the table in order to determine the transmission mode, broadcast or point-to-point, which corresponds to the received transaction. In a further embodiment, network congestion may be monitored and transmission modes adjusted accordingly. For example, when network utilization is high, the number of transactions which are broadcast may be reduced. Alternatively, when network utilization is low, the number of broadcasts may be increased to take advantage of available bandwidth.

Claims

1. A multiprocessing system comprising:

a plurality of processing subsystems, each including a cache memory;

a memory subsystem including a directory;

a network interconnecting said plurality of processing subsystems and said memory subsystem;

a coherency mode storage unit configured to store an indication to control whether a given coherency request is transmitted through said network according to a directory protocol or a broadcast protocol.

2. The multiprocessing system as recited in claim 1 wherein said directory includes a plurality of entries corresponding to different memory locations mapped to said memory subsystem, wherein each entry contains information indicative of whether a cached copy of a corresponding block has been created in one or more of said plurality of processing subsystems.

3. The multiprocessing system as recited in claim 2 wherein when said given coherency request is transmitted through said network according to said broadcast protocol, said coherency request is broadcasted to said memory subsystem and to each of said plurality of processing subsystems regardless of information contained within said directory.

4. The multiprocessing system as recited in claim 3 wherein said given coherency request is initiated by a requesting processing subsystem, and wherein said given coherency request is transmitted to said network through a point-to-point link.

5. The multiprocessing system as recited in claim 4 wherein said given coherency request transmitted from said requesting processing subsystem through said point-to-point link does not specify whether said given coherency request is transmitted through said network according to said directory protocol or said broadcast protocol.

6. The multiprocessing system as recited in claim 3 wherein when said coherency request is transmitted through said network according to said directory protocol, said given coherency request is transmitted through said network to said memory subsystem.

7. The multiprocessing system as recited in claim 6 wherein when said given coherency request is transmitted through said network according to said directory protocol, said directory is accessed by said memory subsystem and responsive coherency commands are provided to one or more of said plurality of processing subsystems dependent upon information contained within said directory.

8. The multiprocessing system as recited in claim 1 wherein said coherency mode storage unit is configured to store a plurality of additional indications to control whether other coherency requests are transmitted through said network according to said directory protocol or said broadcast protocol.

9. The multiprocessing system as recited in claim 8 wherein selected coherency requests are transmitted through said network according to said directory protocol while other coherency requests are transmitted through said network according to said broadcast protocol.

10. The multiprocessing system as recited in claim 9 wherein an address of each coherency request is used to dictate whether said each request is transmitted through said network according to said directory protocol or said broadcast protocol.

11. A multiprocessing system comprising:

a plurality of processing subsystems, each including a cache memory;

a memory subsystem including a directory;

a network interconnecting said plurality of processing subsystems and said memory subsystem;

wherein ownership transitions for a first set of cache lines are governed according to a sequence of actions within said network;

and wherein ownership transitions for a second set of cache lines are governed according to a sequence of actions within said directory.

12. The multiprocessing system as recited in claim 11 wherein said directory includes a plurality of entries corresponding to different memory locations mapped to said memory subsystem, wherein each entry contains information indicative of whether a cached copy of a corresponding block has been created in one or more of said plurality of processing subsystems.

13. The multiprocessing system as recited in claim 12 further comprising a coherency mode storage unit configured to store an indication to control whether a given coherency request is transmitted through said network according to a directory protocol or a broadcast protocol, wherein when said given coherency request is transmitted through said network according to said broadcast protocol, said coherency request is broadcasted to said memory subsystem and to each of said plurality of processing subsystems regardless of information contained within said directory.

14. The multiprocessing system as recited in claim 13 wherein said given coherency request is initiated by a requesting processing subsystem, and wherein said given coherency request is transmitted to said network through a point-to-point link.

15. The multiprocessing system as recited in claim 14 wherein said given coherency request transmitted from said requesting processing subsystem through said point-to-point link does not specify whether said given coherency request is transmitted through said network according to said directory protocol or said broadcast protocol.

16. The multiprocessing system as recited in claim 13 wherein when said coherency request is transmitted through said network according to said directory protocol, said given coherency request is transmitted through said network to said memory subsystem.

17. The multiprocessing system as recited in claim 16 wherein when said given coherency request is transmitted through said network according to said directory protocol, said directory is accessed by said memory subsystem and responsive coherency commands are provided to one or more of said plurality of processing subsystems dependent upon information contained within said directory.

18. The multiprocessing system as recited in claim 13 wherein said coherency mode storage unit is configured to store a plurality of additional indications to control whether other coherency requests are transmitted through said network according to said directory protocol or said broadcast protocol.

19. The multiprocessing system as recited in claim 18 wherein selected coherency requests are transmitted through said network according to said directory protocol while other coherency requests are transmitted through said network according to said broadcast protocol.

20. The multiprocessing system as recited in claim 19 wherein an address of each coherency request is used to dictate whether said each request is transmitted through said network according to said directory protocol or said broadcast protocol.

21. A multiprocessing system comprising:

a memory subsystem including a directory, the directory including a plurality of directory entries corresponding to memory locations mapped to said memory device;

a plurality of processing subsystems each including a cache for storing blocks corresponding to selected ones of said memory locations;

a network interconnecting said plurality of processing subsystems and said memory subsystem;

a coherency mode storage unit configured to store an indication of whether a given block is a point-to-point mode block or a broadcast mode block;

wherein the network is configured to route a coherency request for said given block initiated by one of said processing subsystems to said directory in response to said indication indicating said given block is a point-to-point mode block, and wherein said network is configured to broadcast said coherency request to said memory subsystem and to each of said plurality of processing subsystems in response to said indication indicating said given block is a broadcast mode block.

22. A multiprocessing system comprising:

a plurality of processing subsystems, each including a cache memory;

a memory subsystem including a directory;

a network interconnecting said plurality of processing subsystems and said memory subsystem;

wherein transitions of access rights for a first set of cache lines are governed according to a sequence of actions within said network;

and wherein transitions of access rights for a second set of cache lines are governed according to a sequence of actions within said directory.

23. A multiprocessing system comprising:

a plurality of processing subsystems, each including a cache memory;

a memory subsystem;

a network interconnecting said plurality of processing subsystems and said memory subsystem;

a control unit configured to monitor bandwidth utilization of said network and to control whether a given coherency request is transmitted through said network according to a directory protocol or a broadcast protocol depending upon the bandwidth utilization.

24. A method of operating a multiprocessing system including a plurality of processing subsystems and a memory subsystem interconnected through a network, the method comprising:

storing an indication to control whether a given coherency request is transmitted through said network according to a directory protocol or a broadcast protocol;

a first processing subsystem initiating a coherency request; and

said network conveying said coherency request according to said indication.