Intelligent high bandwidth memory appliance

• US 10,545,860 B2
• Filed: 10/27/2017
• Issued: 01/28/2020
• Est. Priority Date: 08/10/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Malladi, Krishna T., Zheng, Hongzhong, Brennan, Robert, Kim, Hyungseuk, Kim, Jinhyun
• Primary Examiner(s): Bertram, Ryan
• Application Number:
  US15/796,743
• Publication Number:
  US 20190050325A1
• Time in Patent Office: 823 Days
• CPC Class Codes: G06F 12/023 Free address space management
• CPC Class Codes: G06F 12/0862 with prefetch
• CPC Class Codes: G06F 2212/1008 Correctness of operation, e.g. memory ordering
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/1056 Simplification
• CPC Class Codes: G06F 2212/602 Details relating to cache prefetching
• CPC Class Codes: G06F 9/3001 Arithmetic instructions
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G11C 5/025 Geometric lay-out considerations of storage- and peripheral-blocks in a semiconductor storage device geometrical lay-out of the components in integrated circuits, H01L27/0207
• CPC Class Codes: G11C 5/04 Supports for storage elements , e.g. memory modules; Mounting or fixing of storage elements on such supports
• CPC Class Codes: H01L 25/0652 the devices being arranged next and on each other, i.e. mixed assemblies
• CPC Class Codes: H01L 25/0655 the devices being arranged next to each other
• CPC Class Codes: H01L 25/18 the devices being of types provided for in two or more different subgroups of the same main group of groups H01L27/00 - H01L33/00, or in a single subclass of H10K, H10N
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Inventive aspects include An HBM+ system, comprising a host including at least one of a CPU, a GPU, an ASIC, or an FPGA; and an HBM+ stack including a plurality of HBM modules arranged one atop another, and a logic die disposed beneath the plurality of HBM modules. The logic die is configured to offload processing operations from the host. A system architecture is disclosed that provides specific compute capabilities in the logic die of high bandwidth memory along with the supporting hardware and software architectures, logic die microarchitecture, and memory interface signaling options. Various new methods are provided for using in-memory processing abilities of the logic die beneath an HBM memory stack. In addition, various new signaling protocols are disclosed to use an HBM interface. The logic die microarchitecture and supporting system framework are also described.

Claims

1. An HBM+ system, comprising:

a host including at least one of a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gate array (FPGA); and

an HBM+ stack including a plurality of high bandwidth memory (HBM) modules arranged one atop another, and a logic die disposed beneath the plurality of HBM modules;

wherein the logic die comprises;

an HBM controller including a memory controller configured to interface with the plurality of HBM modules; and

an offload processing logic section configured to offload processing operations from the host, and

wherein;

the offload processing logic section is configured to receive a first flag set by the host to indicate that the offload processing operations should begin;

the offload processing logic section is configured to perform the offload processing operations using the HBM controller responsive to the first flag; and

the HBM controller is configured to set a second flag to indicate that the offload processing operations are complete.

2. The HBM+ system of claim 1, wherein the logic die is configured to offload processing operations from the host.

3. The HBM+ system of claim 1, further comprising an interposer coupled to the host and to the logic die.

4. The HBM+ system of claim 3, further comprising a substrate coupled to the interposer.

5. The HBM+ system of claim 4, wherein the plurality of HBM modules are communicatively coupled to the logic die, and the logic die is communicatively coupled to the host.

6. The HBM+ system of claim 1, wherein:

the HBM+ stack is referred to as a first HBM+ stack;

the plurality of HBM modules is referred to as a first plurality of HBM modules;

the logic die is referred to as a first logic die; and

the system further includes a second HBM+ stack including a second plurality of HBM modules arranged one atop another, and a second logic die disposed beneath the second plurality of HBM modules.

7. The HBM+ system of claim 6, wherein:

the first and second logic dies are each configured to offload processing operations from the host;

the first plurality of HBM modules are communicatively coupled to the first logic die, and the first logic die is communicatively coupled to the host;

the second plurality of HBM modules are communicatively coupled to the second logic die, and the second logic die is communicatively coupled to the host; and

the system further comprises an interposer coupled to the host and to the first and second logic dies, and a substrate coupled to the interposer.

8. The HBM+ system of claim 1, further comprising:

a memory;

wherein the logic die comprises;

a host manager including an interface PHY and a host queue manager, wherein the host manager is configured to interface with the host via the interface PHY, and to queue communications received from the host; and

a memory controller including a prefetch engine and a cache controller, wherein the memory controller is configured to interface with the memory via the prefetch engine and the cache controller.

9. The HBM+ system of claim 8, wherein:

the offload processing logic section is configured to receive information about the offload processing operations from the host via the host manager; and

the offload processing logic section is configured to perform the offload processing operations dependent on the received information about the offload processing operations.

10. The HBM+ system of claim 9, wherein the received information includes a command.

11. The HBM+ system of claim 10, further comprising:

a command decode logic section configured to decode the command received from the host.

12. The HBM+ system of claim 11, wherein the offload processing logic section is configured to perform the offload processing operations responsive to the issued command.

13. The HBM+ system of claim 8, wherein the offload processing logic section includes at least one of an arithmetic logic unit (ALU), a floating-point unit (FPU), fixed logic, or reconfigurable logic.

14. The HBM+ system of claim 8, wherein the offload processing logic section is configured to perform the offload processing operations dependent on data stored in the plurality of HBM modules.

15. The HBM+ system of claim 8, wherein:

the memory controller is an SRAM memory controller; and

the memory is an SRAM memory.

16. A logic die, comprising:

a host manager including an interface PHY and a host queue manager, wherein the host manager is configured to interface with a host via the interface PHY, and to queue communications received from the host;

a memory controller including a prefetch engine and a cache controller, wherein the memory controller is configured to interface with a memory via the prefetch engine and the cache controller;

a High Bandwidth Memory (HBM) controller including a memory controller configured to interface with a stack of HBM modules; and

an offload processing logic section configured to offload processing operations from the host,

wherein;

the received information includes a first flag set by the host to indicate processing should begin;

the offload processing logic section is configured to perform the processing operations using the HBM controller responsive to the first flag; and

the HBM controller is configured to set a second flag to indicate that the processing operations are complete.

17. The logic die of claim 16, wherein:

the offload processing logic section is configured to receive information about the offload processing operations from the host via the host manager; and

the offload processing logic section is configured to perform the offload processing operations dependent on the received information about the offload processing operations.

18. The logic die of claim 16, wherein the received information includes a command.

19. The logic die of claim 18, further comprising:

a command decode logic section configured to decode the command received from the host.

20. The logic die of claim 19, wherein the offload processing logic section is configured to perform the offload processing operations responsive to the issued command.

21. The logic die of claim 19, wherein the offload processing logic section includes at least one of an arithmetic logic unit (ALU), a floating-point unit (FPU), fixed logic, or reconfigurable logic.

22. The logic die of claim 19, wherein the offload processing logic section is configured to perform the offload processing operations dependent on data stored in the stack of HBM modules.

23. The logic die of claim 16, wherein:

the memory controller is an SRAM memory controller; and

the memory is an SRAM memory.