Memory module having volatile and non-volatile memory subsystems and method of operation

• US 10,198,350 B2
• Filed: 05/07/2015
• Issued: 02/05/2019
• Est. Priority Date: 07/28/2011
• Status: Active Grant

• Current Assignee: Netlist, Inc.
• Original Assignee: Netlist, Inc.
• Inventors: Lee, Hyun
• Primary Examiner(s): Lo, Ann J
• Assistant Examiner(s): Taeuber, Michelle L
• Application Number:
  US14/706,873
• Publication Number:
  US 20160019138A1
• Time in Patent Office: 1,370 Days
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/0638 Combination of memories, e.g. ROM and RAM such as to permit replacement or supplementing of words in one module by words in another module address formation of the next microinstruction G06F9/26; masking faults in memories by using spares or by reconfiguring G11C29/70
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 12/0897 with two or more cache hierarchy levels with multilevel cache hierarchies G06F12/0811
• CPC Class Codes: G06F 13/10 Program control for peripheral devices G06F13/14 - G06F13/42 take precedence
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 2206/1014 One time programmable [OTP] memory, e.g. PROM, WORM
• CPC Class Codes: G06F 2212/205 Hybrid memory, e.g. using both volatile and non-volatile memory
• CPC Class Codes: G06F 2212/214 Solid state disk
• CPC Class Codes: G06F 2212/313 In storage device
• CPC Class Codes: G06F 2212/7201 Logical to physical mapping or translation of blocks or pages
• CPC Class Codes: G06F 2212/7208 Multiple device management, e.g. distributing data over multiple flash devices
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/068 Hybrid storage device
• CPC Class Codes: G06F 3/0685 Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: G11C 7/1072 for memories with random access ports synchronised on clock signal pulse trains, e.g. synchronous memories, self timed memories

Abstract

A memory module comprises a volatile memory subsystem configured to coupled to a memory channel in computer system and capable of serving as main memory for the computer system, a non-volatile memory subsystem providing storage for the computer system, and a module controller coupled to the volatile memory subsystem, the non-volatile memory subsystem, and the C/A bus. The module controller reads first data from the non-volatile memory subsystem in response to a Flash access request received via the memory channel, and causes at least a portion of the first data to be written into the volatile memory subsystem in response to a dummy write memory command received via the C/A bus. The module control device includes status registers accessible by the computer system via the memory bus.

Claims

1. A memory module for use in a computer system, the computer system including a memory controller coupled to the memory module via a memory channel, the memory channel including a data bus and a control/address (C/A) bus, comprising:

a volatile memory subsystem coupled to the memory channel, the volatile memory subsystem including a registered control device (RCD) configured to receive input C/A signals from the C/A bus and memory devices configured to perform memory operations in response to output C/A signals from the RCD that are derived from the input C/A signals;

a non-volatile memory subsystem; and

a module control device coupled to the volatile memory subsystem, the non-volatile memory subsystem, and the memory channel, wherein the module control device is configured to perform a page-fault operation in response to a set of signals received from the memory channel indicating a non-volatile memory access request to transfer first data from the non-volatile memory subsystem to the volatile memory subsystem, the page fault operation including reading the first data from the non-volatile memory subsystem in response to the set of signals, and providing at least a portion of the first data to the volatile memory subsystem in response to a dummy write memory command received via the C/A bus, the dummy write memory command being received by both the module control device and the RCD and including a memory address related to the page-fault operation.

2. The memory module of claim 1, further comprising buffer memory, wherein the module control device is further configured to, as part of the page fault operation, store the first data into the buffer memory after reading the first data from the non-volatile memory subsystem and send a read command to the buffer memory to cause the buffer memory to output another portion of the first data after receiving the dummy write memory command;

wherein the volatile memory subsystem is configured to receive the at least a portion of the first data in response to the dummy write memory command, and wherein the volatile memory subsystem, the non-volatile memory subsystem, the module control device and the buffer memory are mounted on a same printed circuit board.

3. The memory module of claim 2, wherein the module control device is configured to read second data from the volatile memory subsystem in response to the set of signals indicating the non-volatile memory access request, wherein the buffer memory includes a first memory and a second memory, wherein the module control device is further configured to write the first data into the first memory concurrently with writing the second data into the second memory.

4. The memory module of claim 3, wherein the module control device is configured to perform error correction on the first data stored in the first memory by reading data from the first memory, performing error detection and correction on data read from the first memory, and storing corrected data back into the first memory, and wherein the module control device is further configured to transfer the second data from the second memory to the non-volatile memory subsystems concurrently with performing error correction on the first data stored in the first memory.

5. The memory module of claim 1, wherein the set of signals indicating the non-volatile memory access request include a set of C/A signals received via the C/A bus and a set of data signals received via the data bus, the set of data signals including information about one or more locations in the non-volatile memory subsystem.

6. The memory module of claim 2, wherein the module control device includes status registers accessible by the computer system via the memory channel, the status registers for storing therein status bits for the non-volatile memory access request, and wherein the module control device is further configured to, as part of the page fault operation, provide the status bits to the memory controller via the memory channel in response to a read command and a read address received from the memory controller, the read address being in an address space assigned to the status registers.

7. The memory module of claim 6, wherein the status registers is further configured to store an ECC code associated with the status bits, and wherein the module control device provides the ECC code along with the status bits.

8. The memory module of claim 6, wherein the status bits include information regarding an estimated wait time for the first data to be stored into the buffer memory.

9. The memory module of claim 1, wherein the at least a portion of the first data includes a first portion of the first data, wherein the memory devices in the volatile memory subsystem include double data rate dynamic random access memory devices, wherein the memory module is configured to receive a normal write memory command via the C/A bus after receiving the dummy write memory command, and to receive write data via the data bus in response to the normal write memory command, and wherein the module control device is further configured to cause a second portion of the first data to be written into the volatile memory subsystem in response to another dummy write memory command received after the normal write memory command.

10. The memory module of claim 9, further comprising a serial presence detect device coded with a memory density bigger than a memory density of the volatile memory subsystem thereby allowing for a non-volatile memory access address space that is outside a memory address space for the volatile memory subsystem, wherein the non-volatile memory access request includes an address in the non-volatile memory access address space.

21. The memory module of claim 1, wherein the volatile memory subsystem is configured to receive the at least a portion of the first data in response to the dummy write memory command.

22. The memory module of claim 21, further comprising a printed circuit board, wherein the volatile memory subsystem, the non-volatile memory subsystem, and the module control device are mounted on the printed circuit board.

23. The memory module of claim 22, further comprising buffer memory configured to buffer the first data, wherein the module control device is further configured to, before receiving the dummy write memory command, send a first read command to the buffer memory to output the at least a portion of the first data.

24. The memory module of claim 23, wherein the module control device is further configured to send a second read command to the buffer memory to cause the buffer memory to output another portion of the first data after receiving the dummy write memory command, and wherein the module control device is further configured to monitor memory commands on the C/A bus while the first data is being transferred to the volatile memory subsystem, and to send a pause command to the buffer memory in response to receiving a normal memory command for system access to the volatile memory subsystem via the C/A bus.

25. The memory module of claim 23, wherein the module control device is configured to perform error correction on the first data after storing the first data into the buffer memory and before sending the read command to the buffer memory to output the at least a portion of the first data in response to the dummy write memory command, and wherein the module control device is configured to perform the error correction on the first data stored in the buffer memory by reading data from the buffer memory, performing error detection and correction on data read from the buffer memory, and storing corrected data back into the buffer memory.

11. A method of operating a memory module in a computer system, the computer system including a memory controller coupled to the memory module via a memory channel, the memory channel including a data bus and a control/address (C/A) bus, the method comprising:

receiving, by a module control device on the memory module, a set of signals from the memory controller via the memory channel; and

performing a page fault operation in response to the set of signals indicating a non-volatile memory access request to transfer first data from a non-volatile memory subsystem on the memory module to a volatile memory subsystem on the memory module, the page fault operation including;

reading first data from the non-volatile memory subsystem in response to receiving the set of signals;

receiving, by the module control device, a dummy write memory command via the C/A bus, the dummy write memory command including an address related to the page-fault operation; and

providing at least a portion of the first data to the volatile memory subsystem in response to the dummy write memory command, wherein the volatile memory subsystem includes a registered control device (RCD) configured to receive the dummy write memory command via the C/A bus, and wherein the volatile memory subsystem is configured to receive the at least a portion of the first data in response to the dummy write memory command.

12. The method of claim 11, wherein the memory module further includes buffer memory, wherein the page-fault operation further includes:

storing the first data into the buffer memory after reading the first data from the non-volatile memory subsystem, and causing the buffer memory to output another portion of the first data after receiving the dummy write memory command.

13. The method of claim 12, wherein the buffer memory includes first memory and a second memory, the method further comprising:

reading second data from the volatile memory subsystem in response to the set of signals indicating the non-volatile memory access request; and

writing the first data into the first memory concurrently with writing the second data into the second memory.

14. The method of claim 13, further comprising:

transferring the second data from the second memory to the non-volatile memory subsystems; and

performing error correction on the first data stored in the first memory concurrently with transferring the second data from the second memory to the non-volatile memory subsystems;

wherein performing error correction comprises reading data from the first memory, performing error detection and correction on data read from the first memory, and storing corrected data back into the first memory.

15. The method of claim 11, wherein receiving the set of signals indicating the non-volatile memory access request includes receiving a set of C/A signals via the C/A bus and receiving a set of data signals via the data bus, the set of data signals including information about one or more locations in the non-volatile memory subsystem.

16. The method of claim 12, further comprising, as part of the page-fault operation, storing status information for the non-volatile memory access request, and providing the status information to the memory controller via the memory channel in response to a read command for reading the status information.

17. The method of claim 16, further comprising storing an ECC code associated with the status information, and providing the ECC code along with the status information.

18. The method of claim 16, wherein the status information includes information regarding an estimated wait time for the first data to be stored into a buffer memory before being written into the volatile memory subsystem.

19. The method of claim 11, wherein the at least a portion of the first data includes a first portion of the first data, and wherein the volatile memory subsystem includes double data rate dynamic random access memory, the method further comprising:

receiving a normal write memory command via the C/A bus after receiving the dummy write memory command;

receiving at the volatile memory subsystem write data via the data bus in response to the normal write memory command;

receiving an additional dummy write memory command via the C/A bus after receiving the normal write memory command; and

causing a second portion of the first data to be written into the volatile memory subsystem in response to the additional dummy write memory command.

20. The method of claim 19, further comprising coding a serial presence detect device with a memory density bigger than a memory density of the volatile memory subsystem thereby allowing for a non-volatile memory access address space that is outside a memory address space for the volatile memory subsystem, wherein the non-volatile memory access request includes an address in the non-volatile memory access address space.