Systems and methods for dynamically modifying coherence domains

• US 10,540,286 B2
• Filed: 04/30/2018
• Issued: 01/21/2020
• Est. Priority Date: 04/30/2018
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Inventors: Milojicic, Dejan S, Packard, Keith, Woodacre, Michael S., Wheeler, Andrew R
• Primary Examiner(s): Nguyen, Than
• Application Number:
  US15/967,596
• Publication Number:
  US 20190332538A1
• Time in Patent Office: 631 Days
• Field of Search: 711141
• CPC Class Codes: G06F 12/0815 Cache consistency protocols
• CPC Class Codes: G06F 12/0837 with software control, e.g. non-cacheable data
• CPC Class Codes: G06F 2212/1032 Reliability improvement, data loss prevention, degraded operation etc
• CPC Class Codes: G06F 2212/1048 Scalability
• CPC Class Codes: G06F 2212/601 Reconfiguration of cache memory
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements

Abstract

Systems and methods for dynamically modifying coherence domains are discussed herein. In various embodiments, a hardware controller may be provided that is configured to automatically recognize application behavior and dynamically reconfigure coherence domains in hardware and software to tradeoff performance for reliability and scalability. Modifying the coherence domains may comprise repartitioning the system based on cache coherence independently of one or more software layers of the system. Memory-driven algorithms may be invoked to determine one or more dynamic coherence domain operations to implement. In some embodiments, declarative policy statements may be received from a user via one or more interfaces associated with the controller. The controller may be configured to dynamically adjust cache coherence policy based on the declarative policy statements received from the user.

Claims

1. A system for dynamically modifying coherence domains, the system comprising:

a plurality of partitions, wherein each of one or more of the plurality of partitions is executing at least one application and is associated with one of a set of cache coherence domains; and

a controller configured to;

reconfigure the set of cache coherence domains while one or more of the plurality of partitions is executing at least one application;

obtain a cache coherence policy for one or more of the plurality of partitions;

receive user input indicating one or more modifications to the cache coherence policy; and

responsive to the user input, implement the modifications to the cache coherence policy based on the user input while one or more of the plurality of partitions is executing at least one application.

2. The system of claim 1, wherein the controller is further configured to:

monitor cache traffic involving one or more of the set of cache coherence domains; and

automatically modify the set of cache coherence domains based on the cache traffic.

3. The system of claim 1, wherein to reconfigure the set of cache coherence domains, the controller is further configured to:

grow or shrink individual cache coherence domains of the set of cache coherence domains.

4. The system of claim 3, wherein to grow or shrink the individual cache coherence domains, the controller is further configured to:

assign available memory to at least one of the individual cache coherence domains.

5. The system of claim 3, wherein to grow or shrink the individual cache coherence domains, the controller is further configured to:

reassign at least one memory or computing resource from a first domain of the individual cache coherence domain to a second domain of the individual cache coherence domains.

6. The system of claim 5, wherein the controller is further configured to:

flush a cache associated with the at least one memory or computing resource prior to reassigning the at least one memory or computing resource to the second domain.

7. The system of claim 1, wherein to reconfigure the set of cache coherence domains, the controller is further configured to:

determine one or more operations related to the set of cache coherence domains based on one or more memory-driven algorithms and the cache traffic, wherein to reconfigure the set of cache coherence domains, the controller is configured to execute the one or more operations.

8. The system of claim 7, wherein the controller is further configured to:

transmit an upcall to synchronize the reconfigured set of cache coherence domains across multiple layers of the system.

9. The system of claim 1, wherein the controller is further configured to:

receive instructions from one or more applications executed on the plurality of partitions, wherein the one or more applications are configured to monitor behavior metrics for the one or more applications and determine one or more operations to implement based on the behavior metrics, wherein the controller is configured to reconfigure the set of cache coherence domains based on the one or more operations determined by the one or more applications.

10. The system of claim 9, wherein the one or more application behavior metrics comprise one or more of cache metrics, local and remote latency histograms, number of outstanding requests, and fabric backpressure.

11. A method for dynamically modifying coherence domains in a system comprising a plurality of partitions, wherein each of one or more of the plurality of partitions is executing at least one application and is associated with one of a set of cache coherence domains, the method comprising:

monitoring cache traffic involving one or more of the set of cache coherence domains;

determining one or more operations to reconfigure the set of cache coherence domains based on one or more memory-driven algorithms and the cache traffic;

executing the one or more operations while one or more of the plurality of partitions is executing at least one application;

obtaining a cache coherence policy for one or more of the plurality of partitions;

receiving user input indicating one or more modifications to the cache coherence policy; and

responsive to the user input, implementing the modifications to the cache coherence policy based on the user input while one or more of the plurality of partitions is executing at least one application.

12. The method of claim 11, wherein the one or more operations are executed by a controller.

13. The method of claim 11, the method further comprising:

automatically executing the one or more operations to reconfigure the set of cache coherence domains based on the cache traffic.

14. The method of claim 11, wherein executing the one or more operations comprises:

growing or shrinking individual cache coherence domains of the set of cache coherence domains.

15. The method of claim 14, wherein growing or shrinking the individual cache coherence domains comprises:

assigning available memory to at least one of the individual cache coherence domains.

16. The method of claim 14, wherein growing or shrinking the individual cache coherence domains comprises:

reassigning at least one memory or computing resource from a first domain of the individual cache coherence domain to a second domain of the individual cache coherence domains.

17. The method of claim 16, the method further comprising:

flushing a cache associated with the at least one memory or computing resource prior to reassigning the at least one memory or computing resource to the second domain.

18. The method of claim 11, the method further comprising:

transmitting an upcall to synchronize the reconfigured set of cache coherence domains across multiple layers of the system.

19. A hardware controller configured to dynamically modify coherence domains in a system comprising a plurality of partitions, wherein each of one or more of the plurality of partitions is executing at least one application and is associated with one of a set of cache coherence domains, the controller comprising:

one or more physical processors programmed by computer program instructions that, when executed by the one or more physical processors, configure the controller to;

monitor cache traffic involving one or more of the set of cache coherence domains; and

reconfigure the set of cache coherence domains while one or more of the plurality of partitions is executing at least one application;

obtain a cache coherence policy for one or more of the plurality of partitions;

receive user input indicating one or more modifications to the cache coherence policy; and

responsive to the user input, implement the modifications to the cache coherence policy based on the user input while one or more of the plurality of partitions is executing at least one application.

20. The controller of claim 19, wherein to reconfigure the set of cache coherence domains, the controller is further configured to:

grow or shrink individual cache coherence domains of the set of cache coherence domains.

21. The controller of claim 19, wherein the controller is further configured to:

determine one or more operations related to the set of cache coherence domains based on one or more memory-driven algorithms and the cache traffic, wherein to modify the set of cache coherence domains, the controller is configured to execute the one or more operations.

22. The controller of claim 19, wherein the controller is further configured to:

receive instructions from one or more applications executed on the plurality of partitions, wherein the one or more applications are configured to monitor behavior metrics for the one or more applications and determine one or more operations to implement based on the behavior metrics, wherein the controller is configured to reconfigure the set of cache coherence domains based on the one or more operations determined by the one or more applications.

23. The controller of claim 19, wherein the controller is further configured to:

transmit an upcall to synchronize the reconfigured set of cache coherence domains across multiple layers of the system.