Apparatus and method for implementing a multi-level memory hierarchy

• US 9,600,416 B2
• Filed: 09/30/2011
• Issued: 03/21/2017
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Ramanujan, Raj K., Agarwal, Rajat, Cheng, Kai, Polepeddi, Taarinya, Raad, Camille C., Zimmerman, David J., Swaminathan, Muthukumar P., Ziakas, Dimitrios, Kumar, Mohan J., Coury, Bassam N., Hinton, Glenn J.
• Primary Examiner(s): Mamo, Elias
• Application Number:
  US13/997,189
• Publication Number:
  US 20140129767A1
• Time in Patent Office: 1,999 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0895 of parts of caches, e.g. directory or tag array
• CPC Class Codes: G06F 12/0897 with two or more cache hierarchy levels with multilevel cache hierarchies G06F12/0811
• CPC Class Codes: G06F 2212/2024 Rewritable memory not requiring erasing, e.g. resistive or ferroelectric RAM
• CPC Class Codes: G06F 2212/205 Hybrid memory, e.g. using both volatile and non-volatile memory
• CPC Class Codes: G11C 11/40615 Internal triggering or timing of refresh, e.g. hidden refresh, self refresh, pseudo-SRAMs
• CPC Class Codes: G11C 14/009 and the nonvolatile element is a resistive RAM element, i.e. programmable resistors, e.g. formed of phase change or chalcogenide material
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A system and method are described for integrating a memory and storage hierarchy including a non-volatile memory tier within a computer system. In one embodiment, PCMS memory devices are used as one tier in the hierarchy, sometimes referred to as “far memory.” Higher performance memory devices such as DRAM placed in front of the far memory and are used to mask some of the performance limitations of the far memory. These higher performance memory devices are referred to as “near memory.”

Claims

1. A computer system comprising:

a processor having a plurality of cores to execute instructions and process data and one or more processor caches to cache instructions and data;

a first architectural level of a main memory (“

first level memory”

) having a first set of characteristics associated therewith, the first set of characteristics including a first read access speed and a first write access speed; and

a second architectural level of the main memory (“

second level memory”

) having a second set of characteristics associated therewith, the second set of characteristics including second read and write access speeds at least one of which is relatively slower than either the first read access speed or first write access speed, respectively, non-volatility such that the second level memory maintains its content when power is removed, random access and memory subsystem addressability such that instructions or data stored therein may be randomly accessed at a granularity equivalent to that used by the main memory of the computer system;

wherein at least a portion of the first level memory is configured as a memory side cache for instructions and data stored in the second level memory according to a selected one of multiple cache management mode options.

2. The system as in claim 1 wherein one of the first set of characteristics comprises a first power consumption level and the second set of characteristics comprises a second power consumption level which is relatively lower than the first power consumption level.

3. The system as in claim 1 wherein one of the first set of characteristics comprises a first density and the second set of characteristics comprises a second density which is relatively higher than the first density.

4. The system as in claim 1 wherein one of the second set of characteristics comprises the second level memory being directly writable so as to not require erasing of existing data prior to writing.

5. The system as in claim 1 wherein the first level memory comprises a dynamic random access memory (DRAM) and wherein the one or more processor caches comprise static random access memories (SRAMs).

6. The system as in claim 5 wherein the second level memory comprises chalcogenide.

7. The system as in claim 1 wherein the second level memory comprises vertically stacked storage cells.

8. The system as in claim 1 further comprising:

a mass storage device for persistently storing instructions and data, the mass storage device communicatively coupled to the first level memory and the second level memory through an interface.

9. The system as in claim 1 wherein the second level memory is logically subdivided into a first portion and a second portion, the first portion being allocated as part of the main memory and the second portion being allocated to persistent mass storage, wherein the first level memory is configured as a cache for instructions and data stored only in the first portion of the second level memory.

10. The system as in claim 1 wherein the first level memory is logically subdivided into a first portion and a second portion, the first portion being allocated as part of the main memory and the second portion being allocated as a cache for instructions and data stored in the second level memory according to the selected cache management mode.

11. The system as in claim 1 wherein the first write access speed is relatively faster than the second write access speed but the first read access speed approximates the second read access speed.

12. The system as in claim 11 wherein the first write access speed is at least an order of magnitude higher than the second write access speed.

13. The system as in claim 1 wherein the first set of characteristics includes a first read access latency and a first write access latency and the second set of characteristics includes a second read access latency and a second write access latency at least one of which is relatively higher than either the first read access latency or second write access latency, respectively.

14. The system as in claim 1 wherein the second level memory is cheaper to manufacture per unit size than the first level memory.

15. The system as in claim 1 wherein the one or more processor caches operate independently from the selected cache management mode.

16. The system as in claim 1 wherein memory subsystem addressability comprises addressability at the granularity of a cache line.

17. An apparatus, comprising:

host side main memory control logic circuitry to interface to a multi-level main memory comprising;

a first architectural level of memory (“

first level memory”

) having a first set of characteristics associated therewith, the first set of characteristics including a first read access speed and a first write access speed; and

a second architectural level of memory (“

second level memory”

) having a second set of characteristics associated therewith, the second set of characteristics including second read and write access speeds at least one of which is relatively lower than either the first read access speed or first write access speed, respectively, non-volatility such that the second level memory maintains its content if power is removed, random access and memory subsystem addressability such that instructions or data stored therein may be randomly accessed at a granularity equivalent to that used by a memory subsystem of the computer system, wherein at least a portion of the first level memory is to behave as a memory side cache for the second level memory;

wherein the host side memory control logic circuitry is configurable to select any of a plurality of caching modes for the first level memory.

18. The apparatus as in claim 17 wherein one of the first set of characteristics comprises a first power consumption level and the second set of characteristics comprises a second power consumption level which is relatively lower than the first power consumption level.

19. The apparatus as in claim 17 wherein one of the first set of characteristics comprises a first density and the second set of characteristics comprises a second density which is relatively higher than the first density.

20. The apparatus as in claim 17 wherein one of the second set of characteristics comprises the second level memory being directly writable so as to not require erasing of existing data prior to writing.

21. The apparatus as in claim 17 wherein the first level memory comprises a dynamic random access memory (DRAM) and wherein the one or more processor caches comprise static random access memories (SRAMs).

22. The apparatus as in claim 21 wherein the second level memory comprises chalcogenide.

23. The apparatus as in claim 21 wherein the second level memory comprises a phase change memory (PCM).

24. The apparatus as in claim 17 further comprising:

a mass storage device for persistently storing instructions and data, the mass storage device communicatively coupled to the first level memory and the second level memory through an interface.

25. The apparatus as in claim 17 wherein the first level memory is logically subdivided into a first portion and a second portion, the first portion being allocated as part of the multi-level main memory and the second portion being allocated as a cache for instructions and data stored in the second level memory according to the selected cache mode.

26. The apparatus as in claim 17 wherein the first write access speed is relatively higher than the second write access speed but the first read access speed approximates the second read access speed.

27. The apparatus as in claim 26 wherein the first write access speed is at least an order of magnitude higher than the second write access speed.

28. The apparatus as in claim 17 wherein the first set of characteristics includes a first read access latency and a first write access latency and the second set of characteristics includes a second read access latency and a second write access latency at least one of which is relatively higher than either the first read access latency or second write access latency, respectively.

29. The apparatus as in claim 17 wherein the second level memory is cheaper to manufacture per unit size than the first level memory.

30. The apparatus as in claim 17 wherein the plurality of caching modes include at least two of:

write-back caching mode;

near memory bypass mode;

near memory read-cache write bypass mode;

near memory read-cache write-through mode.

31. The apparatus of claim 17 wherein the second level memory comprises any of the following:

a phase change memory;

byte addressable persistent memory;

storage class memory;

universal memory;

Ge2Sb2Te5;

programmable metallization cell;

resistive memory;

Ovshinsky memory;

ferroelectric memory;

ferromagnetic memory;

spin-transfer torque random access memory;

spin tunneling random access memory;

magnetoresistive memory;

magnetic random access memory;

dielectric memory.