Method, system, and apparatus for dynamic reconfiguration of resources

• US 9,223,738 B2
• Filed: 11/02/2012
• Issued: 12/29/2015
• Est. Priority Date: 12/13/2004
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Ayyar, Mani, Delano, Eric Richard, Kumar, Akhilesh, Jayasimha, Doddaballapur, Vargas, Jose A., Schoinas, Ioannis Y.
• Primary Examiner(s): Osman, Ramy M
• Application Number:
  US13/668,053
• Publication Number:
  US 20130304957A1
• Time in Patent Office: 1,152 Days
• Field of Search: 709/201, 709/221, 709/223, 709/226, 709/215, 712/15, 712/29, 710/105, 710/250, 710/312
• US Class Current: 1/1
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: G06F 13/24 using interrupt G06F13/32 takes precedence
• CPC Class Codes: G06F 13/4081 Live connection to bus, e.g. hot-plugging current or voltage limitation during live insertion H02H9/004
• CPC Class Codes: G06F 13/409 Mechanical coupling back panels H05K7/1438
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5011 the resources being hardware resources other than CPUs, Servers and Terminals

Abstract

A dynamic reconfiguration to include on-line addition, deletion, and replacement of individual modules of to support dynamic partitioning of a system, interconnect (link) reconfiguration, memory RAS to allow migration and mirroring without OS intervention, dynamic memory reinterleaving, CPU and socket migration, and support for global shared memory across partitions is described. To facilitate the on-line addition or deletion, the firmware is able to quiesce and de-quiesce the domain of interest so that many system resources, such as routing tables and address decoders, can be updated in what essentially appears to be an atomic operation to the software layer above the firmware.

Claims

1. An apparatus comprising:

input/output (“

I/O”

) logic, at a computing system, coupled to a point-to-point interconnect to couple caching agents and home agents, the I/O logic including a physical layer to receive an operation based on an interrupt generated in response to an insertion or removal of a resource, wherein, in response to the physical layer receiving the operation, the I/O logic to enter a state where no protocol transactions are generated, and wherein during state routing, routing tables associated with the resource are updated without rebooting an operating system (“

OS”

) of the computing system by firmware performing a quiesce operation of the caching agents and the home agents on one or more partitions; and

dynamic configuration logic to facilitate at least one of on-line addition and on-line deletion of the resource relating to at least one of the one or more partitions without requiring the rebooting of the OS, wherein the on-line addition of an individual resource comprises

performing physical and logical set up for the individual resource,

defining common system interface (CSI) links between a running system (RS) and the individual resource,

setting a path from the individual resource to a firmware,

running self test and initialization on the individual resource,

indicating to firmware execution on the RS that the individual resource is ready to join at least one partition, and

indicating to the OS on the RS that the individual resource is ready to join at least one partition.

2. The apparatus of claim 1, wherein if multiple resources are added to multiple OS partitions, multiple interrupts are sent to the multiple OS partitions.

3. The apparatus of claim 1, wherein the caching agents and the home agents are coupled to a network fabric adhered to layered protocol scheme and used for transporting messages from one protocol to another protocol, wherein the network fabric includes one or more of a link layer, a physical layer, a protocol layer, a routing layer, and a transport layer.

4. The apparatus of claim 1, wherein the resource comprises a processor node, a memory-only node, or an I/O hub node.

5. The apparatus of claim 1, wherein the firmware comprises an inband firmware, and wherein the on-line addition is performed at least in part on the inband firmware being able to quiesce or de-quiesce a domain of interest to update one or more resources in an atomic operation to a software layer about the inband firmware.

6. The apparatus of claim 1, wherein performing the quiesce operation is further ensure that one or more in-transit snoop operations are completed, wherein a quiesced state includes a state where there are no protocol transactions in at least one of a full system and a subset thereof except those generated by at least one of the firmware and one or more service system processors, wherein, in response to the quiesce operation, a signal is generated indicating that the resource is ready for insertion or removal.

7. A method comprising

generating, at a computing system, an interrupt in response to an insertion or removal of a resource;

receiving, at a physical layer of an input/output logic, an operation based on the interrupt, wherein the I/O logic is coupled to a point-to-point interconnect to couple caching agents and home agents;

in response to the physical layer receiving the operation, entering the I/O logic into a state where no protocol transactions are generated;

propagating the interrupt to the operating system (“

OS”

) level through a dynamic update without having to restart the OS, wherein propagating includes during state routing, updating routing tables associated with the resource without rebooting the operating system by firmware performing a quiesce operation of the caching agents and the home agents on one or more partitions; and

facilitating at least one of on-line addition and on-line deletion of the resource relating to at least one of the one or more partitions without requiring the rebooting of the OS, wherein the on-line addition of an individual resource comprises

performing physical and logical set up for the individual resource,

defining common system interface (CSI) links between a running system (RS) and the individual resource,

setting a path from the individual resource to a firmware,

running self test and initialization on the individual resource,

indicating to firmware execution on the RS that the individual resource is ready to join at least one partition, and

indicating to the OS on the RS that the individual resource is ready to join at least one partition.

8. The method of claim 7, wherein if multiple resources are added to multiple OS partitions, multiple interrupts are sent to the multiple OS partitions.

9. The method of claim 7, wherein the caching agents and the home agents are coupled to a network fabric adhered to layered protocol scheme and used for transporting messages from one protocol to another protocol, wherein the network fabric includes one or more of a link layer, a physical layer, a protocol layer, a routing layer, and a transport layer, wherein the resource comprises a processor node, a memory-only node, or an I/O hub node.

10. The method of claim 7, wherein the firmware comprises an inband firmware, and wherein the on-line addition is performed at least in part on the inband firmware being able to quiesce or de-quiesce a domain of interest to update one or more resources in an atomic operation to a software layer about the inband firmware.

11. The method of claim 7, wherein performing the quiesce operation is further ensure that one or more in-transit snoop operations are completed, wherein a quiesced state includes a state where there are no protocol transactions in at least one of a full system and a subset thereof except those generated by at least one of the firmware and one or more service system processors, wherein, in response to the quiesce operation, a signal is generated indicating that the resource is ready for insertion or removal.

12. A system comprising:

a computing system having memory and a processing device coupled to the memory, the computing system further having

input/output (“

I/O”

) logic coupled to a point-to-point interconnect to couple caching agents and home agents, the I/O logic including a physical layer to receive an operation based on an interrupt generated in response to an insertion or removal of a resource, wherein, in response to the physical layer receiving the operation, the I/O logic to enter a state where no protocol transactions are generated, and wherein during state routing, tables associated with the resource are updated without rebooting an operating system (“

OS”

) of the computing system by firmware performing a quiesce operation of the caching agents and the home agents on one or more partitions; and

dynamic configuration logic to facilitate at least one of on-line addition and on-line deletion of the resource relating to at least one of the one or more partitions without requiring the rebooting of the OS, wherein the on-line addition of an individual resource comprises

performing physical and logical set up for the individual resource,

defining common system interface (CSI) links between a running system (RS) and the individual resource,

setting a path from the individual resource to a firmware,

running self test and initialization on the individual resource,

indicating to firmware execution on the RS that the individual resource is ready to join at least one partition, and

indicating to the OS on the RS that the individual resource is ready to join at least one partition.

13. The system of claim 12, wherein if multiple resources are added to multiple OS partitions, multiple interrupts are sent to the multiple OS partitions.

14. The system of claim 12, wherein the caching agents and the home agents are coupled to a network fabric adhered to layered protocol scheme and used for transporting messages from one protocol to another protocol, wherein the network fabric includes one or more of a link layer, a physical layer, a protocol layer, a routing layer, and a transport layer.

15. The system of claim 12, wherein the resource comprises a processor node, a memory-only node, or an I/O hub node.

16. The system of claim 12, wherein the firmware comprises an inband firmware, and wherein the on-line addition is performed at least in part on the inband firmware being able to quiesce or de-quiesce a domain of interest to update one or more resources in an atomic operation to a software layer about the inband firmware.

17. The system of claim 12, wherein performing the quiesce operation is further ensure that one or more in-transit snoop operations are completed, wherein a quiesced state includes a state where there are no protocol transactions in at least one of a full system and a subset thereof except those generated by at least one of the firmware and one or more service system processors, wherein, in response to the quiesce operation, a signal is generated indicating that the resource is ready for insertion or removal.