System and method of initializing the fabric of a distributed multi-processor computing system

• US 20020103995A1
• Filed: 01/31/2001
• Published: 08/01/2002
• Est. Priority Date: 01/31/2001
• Status: Active Grant

• Current Assignee: Advanced Micro Devices, Inc., Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Owen, Jonathan M., Hummel, Mark D., Meyer, Derrick R.
• Granted Patent: US 6,760,838 B2
• Time in Patent Office: Days
• US Class Current: 713/1
• CPC Class Codes: G06F 15/177 Initialisation or configuration control processor initialisation G06F9/4405

Abstract

A method for initializing a computing system comprising a plurality of devices which communicate on a communication link comprising a plurality of independent point-to-point links is provided, each of the point-to-point links interconnecting a respective pair of the plurality of devices. The method includes a link initialization procedure comprising initially configuring each respective pair of devices to communicate on the respective interconnecting link using common communication parameters, including a common frequency and a common link width. The link initialization procedure also may include an optimization procedure for determining maximum communication parameters for each interconnected pair of devices. If the maximum compatible parameters differ from the common parameters for any pair of devices, then the pair of devices may be reconfigured to communicate on the interconnecting link using the maximum compatible parameters. Once a communication capability has been established, the establishment of one or more communication fabrics for the computer system may be performed. This scheme includes designating a bootstrap processor, locating the boot ROM, establishing the manner in which the devices are interconnected, and defining routing directions for routing communications among the various devices in the computing system.

Claims

1. A method of initializing a computing system comprising a plurality of devices and a plurality of point-to-point links, each of the point-to-point links interconnecting a respective pair of devices of the plurality of devices, at least a portion of the plurality of devices comprising a respective processor, the method comprising the acts of:

designating a first processor of the plurality of respective processors as a bootstrap processor (BSP);

configuring each of the plurality of devices to accept a request directed to a default identifier; and

executing, by the BSP, initialization code stored in a first device of the plurality of devices to uniquely identify each of the plurality of devices and to establish routing directions, such that a communication directed to a uniquely identified device is routed to the uniquely identified device.

2. The method as recited in claim 1, wherein the act of configuring each of the plurality of devices to accept a request from the BSP comprises the act of assigning a default identifier to each of the plurality of devices.

3. The method as recited in claim 1, wherein the act of executing the initialization code comprises the acts of:

transmitting a plurality of BSP requests, each of the BSP requests being directed to the default identifier;

assigning a unique identifier to each of the plurality of devices which accept one of the plurality of BSP requests; and

establishing routing directions for the uniquely identified devices, such that a request directed to a uniquely identified device is routed to the uniquely identified device, and a response from a uniquely identified device directed to the BSP is routed to the BSP.

4. The method as recited in claim 1, wherein the act of designating a first processor as a BSP comprises the acts of:

transmitting an initial request from each of the plurality of processors to locate the initialization code;

configuring the first device;

to accept at least a first initial request of the plurality of initial requests, the first initial request being received prior to the others of the plurality of initial requests, the first initial request being received on a first point-to-point link of the plurality of point-to-point links, and

to transmit a response to the first initial request on the first point-to-point link.

5. The method as recited in claim 4, comprising the act of configuring the first device to respond to any requests received subsequent to the first initial request only if received on the first point-to-point link,

6. The method as recited in claim 4, comprising the act of configuring the first device to stall any requests received subsequent to the first initial request if the subsequently received requests are not received on the first point-to-point link.

7. The method as recited in claim 4, wherein each of the plurality of initial requests comprises an unassigned address, and the first device is configured to accept any request having an unassigned address.

8. The method as recited in claim 1, wherein the plurality of devices comprises a plurality of input/output (I/O) devices and a bridge, wherein the bridge connects at least the first processor of the plurality of processors to the plurality of I/O devices, and wherein the plurality of I/O devices is interconnected in a daisy-chain structure comprising a first end connected to the bridge, and the act of establishing routing directions comprises the act of:

establishing, for each of the plurality of I/O devices, a routing direction directed toward the bridge.

9. The method as recited in claim 8, wherein a first I/O device of the plurality of I/O devices comprises the first device.

10. The method as recited in claim 1, wherein the act of establishing routing directions comprises the act of:

defining a response routing direction for each uniquely identified device, the response routing direction corresponding to a direction from which a request was mostly recently received by the uniquely identified device.

11. The method as recited in claim 1, comprising the act of configuring each respective pair of devices to communicate on a respective interconnecting point-to-point link using common communication parameters.

12. The method as recited in claim 11, wherein the common communication parameters for a first pair of devices differ from the common communication parameters for a second pair of devices.

13. The method as recited in claim 11, wherein the common communication parameters for each respective pair of devices are the same.

14. The method as recited in claim 11, wherein the common communication parameters comprise a link width.

15. The method as recited in claim 11, wherein the common communication parameters comprise a link frequency.

16. The method as recited in claim 11, wherein the act of executing the initialization code comprises the acts of:

directing a request to each of the plurality of devices to determine maximum compatible communication parameters for each respective pair of devices; and

reconfiguring each respective pair of devices to communicate on the respective interconnecting point-to-point link using the maximum compatible communication parameters, if the maximum compatible communication parameters differ from the common communication parameters.

18. The method as recited in claim 17, wherein the plurality of devices comprise a bridge device and a plurality of I/O devices, the plurality of I/O devices connected in a daisy-chain structure comprising a first end and a second end, the bridge device being connected between the first end and the BSP, and wherein act (e) comprises establishing, for each of the plurality of I/O devices, a routing direction directed toward the bridge device.

19. The method as recited in claim 18, wherein act (g) comprises the act of:

establishing the second end of the daisy-chain structure.

20. The method as recited in claim 17, wherein the default identifier is a maximum identifier that corresponds to a maximum number of the plurality of devices, and where in act (g) comprises the acts of:

incrementing the value of the next available unique identifier until the value is equal to the maximum identifier;

assigning the maximum identifier to a new device; and

transmitting a request from the BSP directed to the default identifier on each point-to-point link connected to the new device.

17. A method of establishing a communication fabric for a computing system comprising a plurality of devices and a plurality of point-to-point links, each of the plurality of devices connected to at least one of the plurality of point-to-point links, the method comprising the acts of:

(a) configuring each device of the plurality of devices to accept a request directed to a default identifier;

(b) assigning a first unique identifier to a first device of the plurality of devices, the first device comprising a bootstrap processor (BSP);

(c) transmitting a request from the BSP, the request directed to the default identifier;

(d) assigning a next available unique identifier to a new device of the plurality of devices, if the new device accepts the request directed to the default identifier;

(e) establishing routing directions such that a request directed to the new device is received and accepted by the new device and a response directed to the BSP from the new device is received by the BSP;

(f) incrementing the value of the next available unique identifier; and

(g) repeating acts (c), (d), (e), and (f) until each of the plurality of devices has been assigned a unique identifier and a request directed to the default identifier has been transmitted on each of the plurality of point-to-point links.

21. A computing system, comprising:

a first communication medium comprising a first plurality of point-to-point links;

an input/output (I/O) subsystem comprising a plurality of (I/O) devices, wherein each respective pair of the plurality of I/O devices is interconnected by one of the first plurality of point-to-point links, and wherein a first I/O device of the plurality of I/O devices comprises a boot read only memory (ROM), the boot ROM storing initialization code;

a bootstrap processor to execute the initialization code; and

a bridge connecting the bootstrap processor to the I/O subsystem,

wherein the bootstrap processor is configured to transmit a request to the bridge to locate the boot ROM, the request comprising an unassigned address,

wherein the bridge is configured to forward any request comprising an unassigned address to the plurality of I/O devices, and

wherein the first I/O device comprising the boot ROM is configured to accept any request comprising an unassigned address.

22. The computing system as recited in claim 21, wherein the first I/O device is configured to transmit a response to any request comprising an unassigned address, thereby acknowledging location of the boot ROM.

23. The computing system as recited in claim 22, wherein, upon receipt of the response acknowledging the location of the boot ROM, the bootstrap processor is configured to execute the initialization code to assign an identifier to each of the plurality of I/O devices and to define a routing direction for each of the plurality of I/O devices to route communications to the bridge device.

24. The computing system as recited in claim 23, wherein the plurality of I/O devices are interconnected in a daisy-chain structure, the daisy-chain structure comprising a first end and a second end, the first end being connected to the bridge, and wherein the bootstrap processor is configured to establish the second end of the daisy-chain structure.

25. The computing system as recited in claim 21, comprising:

a second communication medium comprising a second plurality of point-to-point links; and

a plurality of processing devices, wherein each of the second plurality of point-to-point links interconnects a respective pair of the plurality of processing devices,

wherein a first processing device of the processing devices comprises the bootstrap processor, and

wherein the bootstrap processor is configured to execute the initialization code to assign an identifier to each of the plurality of processing devices, and to define routing directions to route communications directed to each of the plurality of processing devices.

26. The computing system as recited in claim 21, wherein each of the respective pairs of I/O devices is configured to communicate on a respective interconnecting point-to-point link of the first plurality of point-to-point links using a common link frequency and a common link width.

27. The computing system as recited in claim 26, wherein the common link frequency and the common link width for communications on the point-to-point link interconnecting a first pair of I/O devices differs from the common link frequency and the common link width for communications on the point-to-point link interconnecting a second pair of I/O devices.

28. The computing system as recited in claim 26, wherein the bootstrap processor is configured to transmit a communication to each of the plurality of I/O devices to determine a respective maximum compatible link frequency and a respective maximum compatible link width for each respective pair of I/O devices, and, wherein, if either the respective maximum compatible link frequency or the respective maximum compatible link width for a respective pair of I/O devices differs from the common link frequency and the common link width, the bootstrap processor is configured to reconfigure the respective pair of I/O devices to communicate at the respective maximum compatible link frequency and the respective maximum compatible link width.

29. The computing system as recited in claim 25, wherein each of the respective pairs of processing devices is configured to communicate on a respective interconnecting point-to-point link of the second plurality of point-to-point links using a common link frequency and a common link width.

30. The computing system as recited in claim 29, wherein the bootstrap processor is configured to transmit a communication to each of the plurality of processing devices to determine a respective maximum compatible link frequency and a respective maximum compatible link width for each respective pair of processing devices, and, wherein, if either the respective maximum compatible link frequency or the respective maximum compatible link width for a respective pair of processing devices differs from the common link frequency and the common link width, the bootstrap processor is configured to reconfigure the respective pair of processing devices to communicate on the interconnecting point-to-point link at the respective maximum compatible link frequency and the respective maximum compatible link width.