Separate memory controllers to access data in memory

• US 10,691,344 B2
• Filed: 05/30/2013
• Issued: 06/23/2020
• Est. Priority Date: 05/30/2013
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Yoon, Doe Hyun, Li, Sheng, Chang, Jichuan, Chen, Ke, Ranganathan, Parthasarathy, Jouppi, Norman Paul
• Primary Examiner(s): Bertram, Ryan
• Assistant Examiner(s): Ta, Trang K
• Application Number:
  US14/785,120
• Publication Number:
  US 20160070483A1
• Time in Patent Office: 2,581 Days
• CPC Class Codes: G06F 13/1636 using refresh
• CPC Class Codes: G06F 13/1668 Details of memory controller
• CPC Class Codes: G06F 13/1689 Synchronisation and timing concerns synchronisation on a memory bus G06F13/4234
• CPC Class Codes: G06F 13/1694 Configuration of memory controller to different memory types
• CPC Class Codes: G06F 13/4243 with synchronous protocol
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0646 Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A first memory controller receives an access command from a second memory controller, where the access command is timing non-deterministic with respect to a timing specification of a memory. The first memory controller sends at least one access command signal corresponding to the access command to the memory, wherein the at least one access command signal complies with the timing specification. The first memory controller determines a latency of access of the memory. The first memory controller sends feedback information relating to the latency to the second memory controller.

Claims

1. A system comprising:

a memory;

a first memory controller coupled to the memory;

a second memory controller separate from the first memory controller, the second memory controller to send at least one access command to the first memory controller, the at least one access command to read or write data of the memory and being timing non-deterministic with respect to a timing specification of the memory,

wherein the first memory controller is responsive to the at least one access command to issue at least one command signal to the memory, the at least one command signal satisfying the timing specification of the memory,

wherein the first memory controller is to notify the second memory controller regarding a latency associated with accessing the memory,

wherein the second memory controller is to issue a refresh command to the first memory controller in response to the second memory controller detecting an idle period of the second memory controller,

wherein the memory comprises one or more memory types and the first memory controller comprises one or more separate memory controllers, each memory of the one or more memory types corresponds to each separate memory controller of the first memory controller, respectively; and

wherein the timing specification of the memory relates to a timing parameter associated with a power constraint and a timing parameter associated with a thermal constraint of each memory of the one or more memory types.

2. The system of claim 1, wherein the second memory controller is to detect the idle period of the second memory controller in response to detecting that a memory request queue of the second memory controller is empty, and the second memory controller is to issue the refresh command to the first memory controller in response to the second memory controller detecting that the memory request queue is empty.

3. The system of claim 1, wherein the second memory controller is to issue a precharge command to the first memory controller to precharge a portion of the memory.

4. The system of claim 1, wherein the first memory controller is to send information regarding the latency in a data packet responsive to a dummy access command from the second memory controller.

5. The system of claim 1, wherein the first memory controller is to send information regarding the latency over a handshaking signal channel between the first and second memory controllers.

6. The system of claim 1, wherein the second memory controller is to adjust scheduling of access commands for accessing data of the memory, the adjusting being responsive to the latency.

7. The system of claim 1, wherein the first memory controller is to manage ordering of responses to access commands from the second memory controller, the ordering being according to an order of the access commands received from the second memory controller.

8. The system of claim 1, wherein the first memory controller is to monitor an access history of the memory, and to skip portions of the memory to refresh based on the monitored access history.

9. The system of claim 1, wherein the second memory controller is to issue a refresh termination command to the first memory controller in response to the second memory controller receiving a new memory request.

10. The system of claim 9, wherein the first memory controller is responsive to the refresh command to schedule a refresh operation of a portion of the memory.

11. The system of claim 10, wherein the first memory controller is to stop the refresh operation of the memory in response to receiving the refresh termination command.

12. The system of claim 1, further comprising a processor, wherein the second memory controller is part of the processor.

13. The system of claim 12, further comprising a memory module including the first memory controller, the memory, and a memory interconnect that connects the first memory controller and the memory, wherein the second memory controller is outside of the memory module.

14. The system of claim 1, further comprising the second memory controller to send the at least one access command to a separate memory controller of the first memory controller coupled to a non-volatile memory type and the second memory controller to send the at least one access command to a separate memory controller of the first memory controller coupled to a volatile memory type.

15. A method comprising:

receiving, by a device-side memory controller, at least one access command from a host-side memory controller, wherein the at least one access command is timing non-deterministic with respect to a timing specification of a memory;

sending, by the device-side memory controller, at least one access command signal corresponding to the at least one access command to the memory, wherein the at least one access command signal complies with the timing specification;

determining, by the device-side memory controller, a latency of access of the memory;

sending, by the device-side memory controller, feedback information relating to the latency to the host-side memory controller;

receiving, by the device-side memory controller, a refresh command issued by the host-side memory controller responsive to the host-side memory controller detecting an idle period of the host-side memory controller; and

in response to the refresh command, performing, by the device-side memory controller, a refresh operation of the memory;

wherein the memory comprises one or more memory types and the first memory controller comprises one or more separate memory controllers, each memory of the one or more memory types corresponding to each separate memory controller of the first memory controller, respectively; and

wherein the timing specification relates to a timing parameter associated with a power constraint and a timing parameter associated with a thermal constraint of each memory of the one or more memory types.

16. The method of claim 15, wherein the host-side memory controller is part of a processor.

17. The method of claim 15, further comprising:

receiving, by the host-side memory controller, a memory request from a data requester; and

in response to the memory request, sending, by the host-side memory controller, the access command to the device-side memory controller.

18. The method of claim 17, further comprising:

adjusting, by the host-side memory controller, scheduling of access commands based on the latency indicated by the feedback information, the adjusting of the scheduling of the access commands based on the latency comprising delaying a transmission of an access command from the host-side memory controller to the device-side memory controller in response to the latency indicated by the feedback information.

19. The method of claim 15, further comprising:

receiving, by the device-side memory controller, a refresh termination command from the host-side memory controller; and

stopping, by the device-side memory controller, the refresh operation in response to the refresh termination command.

20. An article comprising at least one non-transitory machine-readable storage medium storing instructions that upon execution cause a first memory controller to:

receive a memory request from a data requester;

send at least one access command to a second memory controller, the at least one access command being timing non-deterministic with respect to a timing specification of a memory, and wherein the at least one access command is to cause the second memory controller to issue at least one access command signal to the memory that satisfies a timing constraint of the timing specification, the timing constraint governing a timing of the access command signal on a memory bus between the second memory controller and the memory;

receive, from the second memory controller, information relating to a latency of accessing the memory; and

issue a refresh command to the second memory controller in response to detecting an idle period of the first memory controller, the refresh command to cause the second memory controller to schedule refresh operations of portions of the memory;

wherein the memory comprises one or more memory types and the first memory controller comprises one or more separate memory controllers, each memory of the one or more memory types corresponds to each separate memory controller of the first memory controller, respectively; and

wherein the timing specification relates to a timing parameter associated with a power constraint and a timing parameter associated with a thermal constraint of each memory of the one or more memory types.