Method and system for synchronizing communications links in a hub-based memory system

• US 20050213611A1
• Filed: 03/29/2004
• Published: 09/29/2005
• Est. Priority Date: 03/29/2004
• Status: Active Grant

• Current Assignee: Micron Technology, Inc.
• Original Assignee: Micron Technology, Inc.
• Inventors: James, Ralph
• Granted Patent: US 7,447,240 B2
• Time in Patent Office: Days
• US Class Current: 370/503
• CPC Class Codes: G06F 13/1684 using multiple buses
• CPC Class Codes: G06F 13/1689 Synchronisation and timing concerns synchronisation on a memory bus G06F13/4234

Abstract

A method is disclosed for synchronizing communications links in a memory hub system. The system includes a system controller and a plurality of memory hubs coupled in series, with pairs of downstream and upstream links being coupled between adjacent modules and the controller. The method includes synchronizing each upstream and downstream link. In a clockwise order starting with the downstream link coupled between the controller and the first memory module, the next adjacent clockwise link is signaled that the prior clockwise link has been synchronized. The method detects through the upstream link coupled between the controller and the first memory module when all links have been synchronized. In a clockwise order starting with the downstream link coupled between the controller and the first memory module, each link is enabled. The method detects through the upstream link coupled between the controller and first memory module when all links have been enabled.

Claims

1. A method of synchronizing communications links in a memory system including a system controller and a plurality of memory hubs coupled in series, with pairs of downstream and upstream links being coupled between adjacent modules and the controller, and the method comprising:

synchronizing an upstream and downstream link coupled to the controller;

sequentially synchronizing downstream links starting with the downstream link coupled between the controller and the first hub;

sequentially synchronizing upstream links starting with the upstream link coupled between the last memory hub and the next upstream hub;

providing an indication to the controller when the upstream link between the first and second hubs has been synchronized;

sequentially enabling downstream links to process functional commands;

sequentially enabling upstream links to process functional commands, and providing an indication to the controller that all links have been enabled.

2. The method of claim 1 wherein synchronizing each of the links comprises applying test data signals over the links, and adjusting a phase shift of a generated receive clock signal relative to the data signals.

3. The method of claim 2 wherein providing an indication to the controller when the upstream link between the first and second hubs has been synchronized comprises inverting the test data signals and providing the inverted test data signals over the upstream link coupled to the controller.

4. The method of claim 1 wherein providing an indication to the controller that all links have been enabled comprises providing an enablement command over the upstream link coupled to the controller.

5. The method of claim 4 wherein the enablement command comprises a NOP command.

6. A method of synchronizing communications links in a memory hub system including a system controller and a plurality of memory hubs coupled in series, with pairs of downstream and upstream links being coupled between adjacent modules and the controller, and the method comprising:

synchronizing each upstream and downstream link;

in a clockwise order starting with the downstream link coupled between the controller and the first memory module, signaling to the next adjacent clockwise link that the prior clockwise link has been synchronized;

detecting through the upstream link coupled between the controller and the first memory module when all links have been synchronized;

in a clockwise order starting with the downstream link coupled between the controller and the first memory module, enabling each link; and

detecting through the upstream link coupled between the controller and the first memory module when all links have been enabled.

7. The method of claim 6 wherein synchronizing each of the links comprises applying test data signals and adjusting a phase shift of a generated receive clock signal relative to the data signals.

8. The method of claim 6 wherein signaling to the next adjacent clockwise link that the prior clockwise link has been synchronized comprises providing an inversion signal to next adjacent clockwise link.

9. The method of claim 8 wherein in response to the inversion signal inverted test data signals are applied over the next adjacent link.

10. The method of claim 8 wherein each link includes a transmission port and a reception port, and wherein enabling each link comprises first enabling the transmission port and thereafter enabling the reception port.

11. The method of claim 10 wherein the reception port in each link is enabled by the transmission port applying an enable command to the reception port.

12. The method of claim 11 wherein the enable command comprises a NOP command.

13. The method of claim 10 wherein once the reception port of a given link has been enabled, an enable signal is supplied to the transmission port of the next adjacent clockwise port to thereby enable the transmission port of the next adjacent clockwise port.

14. A memory hub, comprising:

a downstream reception interface operable in an initialization mode to adjust a phase of a generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface in the normal mode of operation;

a downstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the downstream reception interface, and operable in the enablement mode responsive to the enablement signal from the downstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation;

an upstream reception interface operable in the initialization mode to adjust a phase of the generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface into the normal mode of operation; and

an upstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the upstream reception interface, and operable in the enablement mode responsive to the enablement signal from the upstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation.

15. The memory hub of claim 14 wherein the enablement command comprises a NOP command.

16. The memory hub of claim 14 wherein the downstream and upstream transmission interfaces adjust the value of the corresponding test data signals responsive to the inversion signal by inverting the test data signals.

17. The memory hub of claim 14 further comprising local hub circuitry coupled to the interfaces, the local control circuitry operable process memory requests during the normal mode of operation and to develop corresponding memory signals on a memory bus output.

18. The memory hub of claim 17 wherein the memory signals comprise address, data, and control signals.

19. The memory hub of claim 14 wherein the reception interfaces include optical interfaces adapted to receive data words from an optical communications link.

20. The memory hub of claim 19 wherein the downstream reception interface and the upstream transmission interface are adapted to be coupled to the same optical communications link, and wherein the downstream transmission interface and the upstream reception interface are adapted to be coupled to the same optical communications link.

21. The memory hub of claim 20 wherein the optical communications link comprises an optical fiber.

22. A memory module, comprising:

a plurality of memory devices; and

a memory hub, comprising;

a downstream reception interface operable in an initialization mode to adjust a phase of a generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface in the normal mode of operation;

a downstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the downstream reception interface, and operable in the enablement mode responsive to the enablement signal from the downstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation;

an upstream reception interface operable in the initialization mode to adjust a phase of the generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface into the normal mode of operation;

an upstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the upstream reception interface, and operable in the enablement mode responsive to the enablement signal from the upstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation; and

local hub circuitry coupled to the interfaces and to the memory devices.

23. The memory module of claim 22 wherein the enablement command comprises a NOP command.

24. The memory module of claim 22 wherein the downstream and upstream transmission interfaces adjust the value of the corresponding test data signals responsive to the inversion signal by inverting the test data signals.

25. The memory module of claim 22 wherein the memory signals comprise address, data, and control signals.

26. The memory module of claim 22 wherein the memory devices comprise SDRAMs.

27. The memory module of claim 22 wherein the reception interfaces include optical interfaces adapted to receive data words from an optical communications link.

28. The memory module of claim 27 wherein the downstream reception interface and the upstream transmission interface are adapted to be coupled to the same optical communications link, and wherein the downstream transmission interface and the upstream reception interface are adapted to be coupled to the same optical communications link.

29. The memory module of claim 28 wherein the optical communications link comprises an optical fiber.

30. A memory system, comprising:

a plurality of memory modules coupled in series, each module being coupled to adjacent modules through respective downstream and upstream high-speed communications links, each memory module comprising;

a plurality of memory devices; and

a memory hub, comprising;

a downstream reception interface operable in an initialization mode to adjust a phase of a generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface in the normal mode of operation;

a downstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the downstream reception interface, and operable in the enablement mode responsive to the enablement signal from the downstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation;

an upstream reception interface operable in the initialization mode to adjust a phase of the generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface into the normal mode of operation;

an upstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the upstream reception interface, and operable in the enablement mode responsive to the enablement signal from the upstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation; and

local hub circuitry coupled to the interfaces and to the memory devices; and

a system controller coupled to a first one of the memory modules through respective downstream and upstream high-speed communications links.

31. The memory system of claim 30 wherein the system controller further comprises:

a downstream transmission interface coupled to the downstream high-speed communications link, the interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to an inversion signal, and operable in the enablement mode responsive to an enablement signal to provide an enablement command on the output and to place the interface into the normal mode of operation; and

an upstream reception interface coupled to the upstream high-speed communications link, the interface operable in the initialization mode to adjust a phase of a generated receive clock signal relative to applied test data signals and to generate the inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface into the normal mode of operation, and further operable responsive to receiving the enablement command to generate a ready signal indicating all the memory modules have been synchronized.

32. The memory system of claim 30 wherein the enablement command comprises a NOP command.

33. The memory system of claim 30 wherein the downstream and upstream transmission interfaces adjust the value of the corresponding test data signals responsive to the inversion signal by inverting the test data signals.

34. The memory system of claim 30 wherein the memory signals comprise address, data, and control signals.

35. The memory system of claim 30 wherein the memory devices comprise SDRAMs.

36. The memory system of claim 30 wherein the reception interfaces include optical interfaces adapted to receive data words from an optical communications link.

37. The memory system of claim 36 wherein in each memory module the downstream reception interface and the upstream transmission interface are coupled to the same optical communications link, and wherein the downstream transmission interface and the upstream reception interface are coupled to the same optical communications link.

38. The memory system of claim 37 wherein the optical communications link comprises an optical fiber.

39. A computer system, comprising:

a processor;

a system controller coupled to the processor through respective downstream and upstream high-speed communications links;

a memory system, comprising;

a plurality of memory modules coupled in series, each module being coupled to adjacent modules through respective downstream and upstream high-speed communications links, and a first one of the modules being coupled to the processor through respective downstream and upstream high-speed communications links, each memory module comprising;

a plurality of memory devices; and

a memory hub, comprising;

a downstream reception interface operable in an initialization mode to adjust a phase of a generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface in the normal mode of operation;

a downstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the downstream reception interface, and operable in the enablement mode responsive to the enablement signal from the downstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation;

an upstream reception interface operable in the initialization mode to adjust a phase of the generated receive clock signal relative to applied test data signals and to generate an inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface into the normal mode of operation;

an upstream transmission interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to the inversion signal from the upstream reception interface, and operable in the enablement mode responsive to the enablement signal from the upstream reception interface to provide the enablement command on the output and to place the interface into the normal mode of operation; and

local hub circuitry coupled to the interfaces and to the memory devices.

40. The computer system of claim 39 wherein the system controller further comprises:

a downstream transmission interface coupled to the downstream high-speed communications link, the interface operable in the initialization mode to apply test data signals on an output and operable to adjust the value of the test data signals responsive to an inversion signal, and operable in the enablement mode responsive to an enablement signal to provide an enablement command on the output and to place the interface into the normal mode of operation; and

an upstream reception interface coupled to the upstream high-speed communications link, the interface operable in the initialization mode to adjust a phase of a generated receive clock signal relative to applied test data signals and to generate the inversion signal once a final phase of the generated receive clock signal is determined, and operable in an enablement mode responsive to receiving an enablement command to generate an enablement signal and to place the interface into the normal mode of operation, and further operable responsive to receiving the enablement command to generate a ready signal indicating all the memory modules have been synchronized.

41. The memory system of claim 39 wherein the enablement command comprises a NOP command.

42. The computer system of claim 39 wherein the memory devices comprise SDRAMs.

43. The computer system of claim 39 wherein each of the high-speed links comprises an optical communications link.

44. The computer system of claim 39 wherein the processor comprises a central processing unit (CPU).