Clock distribution in a distributed system with multiple clock domains over a switched fabric

• US 8,583,957 B2
• Filed: 07/27/2010
• Issued: 11/12/2013
• Est. Priority Date: 07/27/2010
• Status: Active Grant

• Current Assignee: National Instruments Corporation (Emerson Electric Company)
• Original Assignee: National Instruments Corporation (Emerson Electric Company)
• Inventors: Chandhoke, Sundeep, Mohrmann, Lee E., Ullrich, Adam C., Greenstreet, Rodney D.
• Primary Examiner(s): Dang, Khanh
• Application Number:
  US12/844,248
• Publication Number:
  US 20120030495A1
• Time in Patent Office: 1,204 Days
• Field of Search: 713/375, 713/400, 713/502, 713/503, 713/600
• US Class Current: 713/401
• CPC Class Codes: G06F 1/10 Distribution of clock signals , e.g. skew
• CPC Class Codes: G06F 1/12 Synchronisation of different clock signals provided by a plurality of clock generators
• CPC Class Codes: G06F 1/14 Time supervision arrangements, e.g. real time clock
• CPC Class Codes: G06F 2213/0038 System on Chip

Abstract

System and method for synchronizing devices. A device reads a first counter coupled to and associated with a master clock and a second counter coupled to and associated with the device, where the device is one of one or more devices coupled to the master clock and each other via a switched fabric, where each device includes a respective clock, and is coupled to and associated with a respective second counter. Each of the first counter and the second counters is accessible by each of the one or more devices. The device determines a difference between the device'"'"'s associated second counter and the first counter, and determines and stores a time reference for the device relative to the master clock based on the determined difference, where the time reference is useable to timestamp events or synchronize future events.

Claims

1. A system, comprising:

a master clock;

a first counter, coupled to and associated with the master clock;

one or more devices coupled to the master clock and each other via a switched fabric, wherein each of the one or more devices includes a respective clock;

one or more second counters, wherein each second counter is coupled to and associated with a respective one of the one or more devices, and wherein each of the first counter and second counters is accessible by each of the one or more devices;

wherein each of the one or more devices is configured to;

read the first counter and the device'"'"'s associated second counter;

determine a difference between the first counter and the device'"'"'s associated second counter, wherein the difference indicates a timing difference between the device and the master clock;

determine and store a time reference for the device relative to the master clock based on the determined difference, wherein the time reference is useable to timestamp events or synchronize future events;

wherein each of the one or more devices has an associated propagation delay corresponding to a signal path between the device'"'"'s associated clock and the master clock, wherein each of the one or more devices is further configured to determine its associated propagation delay;

wherein to determine and store a time reference for the device, each of the one or more devices is further configured to determine and store the time reference for the device further based on the determined associated propagation delay;

wherein the switched fabric comprises a switch, wherein the switch comprises the one or more second counters, wherein each of the one or more devices further comprises a respective third counter which is accessible by each of the other one or more devices, and wherein to determine the associated propagation delay, each of the one or more devices is configured to;

determine a difference between the device'"'"'s associated second counter and the device'"'"'s third counter; and

determine the propagation delay based on the determined difference between the device'"'"'s associated second counter and the device'"'"'s third counter;

wherein to determine and store a time reference for the device, each of the one or more devices is configured to;

determine a difference between the first counter and a propagation delay of the master clock, thereby generating a first delay value;

determine a difference between the device'"'"'s second counter and the device'"'"'s propagation delay, thereby generating a second delay value; and

determine a difference between the first delay value and the second delay value, thereby determining the time reference for the device.

2. The system of claim 1, wherein the master clock is comprised in a host device, wherein the host device comprises a functional unit.

3. The system of claim 2,

wherein the host device is configured to:

send a multicast reset signal over the switched fabric to synchronize the first and second counters on startup; and

wherein each of the one or more devices is configured to;

synchronize the device'"'"'s associated second counter with the first counter based on the reset signal on startup.

4. The system of claim 1, wherein to determine its associated propagation delay, each of the one or more devices is configured to:

determine the associated propagation delay via a heuristic.

5. The system of claim 4, wherein to determine its associated propagation delay via a heuristic, each of the one or more devices is configured to:

determine the associated propagation delay based on estimates of signal path length and signal propagation velocity.

6. The system of claim 1, wherein each of the one or more devices further comprises a time control loop and a logical clock driven by the device'"'"'s clock, and wherein each of the one or more devices is further configured to:

determine a drift adjustment value for the device based on the determined difference; and

adjust the device'"'"'s logical clock via the device'"'"'s time control loop to synchronize the device to the master clock based on the drift adjustment value.

7. The system of claim 1, wherein each of the one or more second counters is comprised in its respective associated device.

8. The system of claim 1, wherein the switched fabric comprises a switch, wherein the master clock is an external clock that is independent from the devices, and wherein the first counter is comprised in the switch.

9. A method for synchronizing devices, the method comprising:

a device performing;

reading a first counter coupled to and associated with a master clock and a second counter coupled to and associated with the device, wherein the device is one of one or more devices coupled to the master clock and each other via a switched fabric, wherein each device includes a respective clock, and is coupled to and associated with a respective second counter, wherein each of the first counter and the second counters is accessible by each of the one or more devices;

determining a difference between the device'"'"'s associated second counter and the first counter;

determining and storing a time reference for the device relative to the master clock based on the determined difference, wherein the time reference is useable to timestamp events or synchronize future events;

wherein each of the one or more devices has an associated propagation delay corresponding to a signal path between the device'"'"'s clock and the master clock, the method further comprising;

the device performing;

determining the device'"'"'s associated propagation delay;

wherein said determining and storing a time reference for the device relative to the master clock based on the determined difference comprises;

determining and storing a time reference for the device further based on the determined associated propagation delay;

wherein the switched fabric comprises a switch, wherein the switch comprises the one or more second counters, wherein each of the one or more devices further comprises a respective third counter which is accessible by each of the other one or more devices, and wherein said determining the associated propagation delay comprises;

determining a difference between the device'"'"'s associated second counter and the device'"'"'s third counter; and

determining the propagation delay based on the determined difference between the device'"'"'s associated second counter and the device'"'"'s third counter;

wherein said determining and storing a time reference for the device relative to the master clock based on the determined difference comprises;

determining a difference between the first counter and a propagation delay of the master clock, thereby generating a first delay value;

determining a difference between the device'"'"'s second counter and the device'"'"'s propagation delay, thereby generating a second delay value; and

determining a difference between the first delay value and the second delay value, thereby determining the time reference for the device.

10. The method of claim 9, wherein the master clock is comprised in a host device, wherein the host device comprises a functional unit.

11. The method of claim 10, further comprising:

the device performing;

receiving a multicast reset signal from the host device over the switched fabric to synchronize the first and second counters on startup; and

synchronizing the device'"'"'s associated second counter with the first counter based on the reset signal on startup.

12. The method of claim 9, wherein said determining the device'"'"'s associated propagation delay comprises:

determining the associated propagation delay via a heuristic.

13. The method of claim 12, wherein said determining the device'"'"'s associated propagation delay via a heuristic comprises:

determining the associated propagation delay based on estimates of signal path length and signal propagation speed.

14. The method of claim 9, wherein each of the one or more devices further comprises a time control loop and a logical clock driven by the device'"'"'s clock, the method further comprising:

the device performing;

determining a drift adjustment value for the device based on the determined difference; and

adjusting the device'"'"'s logical clock via the device'"'"'s time control loop to synchronize the device to the master clock based on the drift adjustment value.

15. The method of claim 9, wherein each of the one or more second counters is comprised in its respective associated device.

16. The method of claim 9, wherein the switched fabric comprises a switch, wherein the master clock is an external clock that is independent from the devices, and wherein the first counter is comprised in the switch.