CACHE MANAGEMENT IN A STREAM COMPUTING ENVIRONMENT THAT USES A SET OF MANY-CORE HARDWARE PROCESSORS

• US 20190087340A1
• Filed: 11/05/2018
• Published: 03/21/2019
• Est. Priority Date: 03/04/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Cook, Alexander, Kleinheksel, Cory J., Koster, David M., Nikolai, Jason A.
• Granted Patent: US 10,452,550 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0842 for multiprocessing or multitasking
• CPC Class Codes: G06F 13/14 Handling requests for interconnection or transfer
• CPC Class Codes: G06F 13/161 with latency improvement
• CPC Class Codes: G06F 13/1652 in a multiprocessor architecture interprocessor communication using common memory G06F15/167
• CPC Class Codes: G06F 13/1657 Access to multiple memories
• CPC Class Codes: G06F 13/1663 Access to shared memory
• CPC Class Codes: G06F 13/18 based on priority control G06F13/1605 takes precedence
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/502 using adaptive policy
• CPC Class Codes: G06F 2212/62 Details of cache specific to multiprocessor cache arrangements
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

Disclosed aspects relate to cache management in a stream computing environment that uses a set of many-core hardware processors to process a stream of tuples by a plurality of processing elements which operate on the set of many-core hardware processors. The stream of tuples to be processed by the plurality of processing elements which operate on the set of many-core hardware processors may be received. A tuple-processing hardware-route on the set of many-core hardware processors may be determined based on a cache factor associated with the set of many-core hardware processors. The stream of tuples may be routed based on the tuple-processing hardware-route on the set of many-core hardware processors. The stream of tuples may be processed by the plurality of processing elements which operate on the set of many-core hardware processors.

Claims

1. A computer-implemented method for cache management in a stream computing environment that uses a set of many-core hardware processors to process a stream of tuples by a plurality of processing elements which operate on the set of many-core hardware processors, the method comprising:

receiving the stream of tuples to be processed by the plurality of processing elements which operate on the set of many-core hardware processors;

determining, based on a cache factor associated with the set of many-core hardware processors, a tuple-processing hardware-route on the set of many-core hardware processors;

routing, based on the tuple-processing hardware-route on the set of many-core hardware processors, the stream of tuples; and

processing, utilizing the set of many-core hardware processors, the stream of tuples by the plurality of processing elements which operate on the set of many-core hardware processors.

2. The method of claim 1, further comprising:

computing, for cache management in the stream computing environment, a first cache utilization factor for a first cache of a first core of the set of many-core hardware processors;

computing, for cache management in the stream computing environment, a second cache utilization factor for a second cache of the first core of the set of many-core hardware processors;

resolving, by comparing the first cache utilization factor for the first cache of the first core of the set of many-core hardware processors with the second cache utilization factor for the second cache of the first core of the set of many-core hardware processors, the tuple-processing hardware-route to prioritize utilization of the first cache of the first core of the set of many-core hardware processors with respect to the second cache of the first core of the set of many-core hardware processors; and

prioritizing, pertaining to processing the stream of tuples utilizing the set of many-core hardware processors, the first cache of the first core of the set of many-core hardware processors with respect to the second cache of the first core of the set of many-core hardware processors.

3. The method of claim 2, further comprising:

detecting, for cache management in the stream computing environment, that the first and second caches of the first core of the set of many-core hardware processors are local caches to the first core of the set of many-core hardware processors.

4. The method of claim 3, further comprising:

detecting, for cache management in the stream computing environment, that the first and second caches of the first core of the set of many-core hardware processors are local caches only to the first core of the set of many-core hardware processors.

5. The method of claim 2, further comprising:

ascertaining, for cache management in the stream computing environment, that a second cache size of the second cache of the first core of the set of many-core hardware processors exceeds a first cache size of the first cache of the first core of the set of many-core hardware processors; and

ascertaining, for cache management in the stream computing environment, that a second cache access burden of the second cache of the first core of the set of many-core hardware processors exceeds a first cache access burden of the first cache of the first core of the set of many-core hardware processors.

6. The method of claim 1, further comprising:

computing, for cache management in the stream computing environment, a first cache utilization factor for a first cache of a first core of the set of many-core hardware processors;

computing, for cache management in the stream computing environment, a second cache utilization factor for a second cache of a second core of the set of many-core hardware processors;

resolving, by comparing the first cache utilization factor for the first cache of the first core of the set of many-core hardware processors with the second cache utilization factor for the second cache of the second core of the set of many-core hardware processors, the tuple-processing hardware-route to prioritize utilization of the second cache of the second core of the set of many-core hardware processors with respect to the first cache of the first core of the set of many-core hardware processors; and

prioritizing, pertaining to processing the stream of tuples utilizing the set of many-core hardware processors, the second cache of the second core of the set of many-core hardware processors with respect to the first cache of the first core of the set of many-core hardware processors.

7. The method of claim 6, further comprising:

detecting, for cache management in the stream computing environment, that the first cache utilization factor for the first cache of the first core of the set of many-core hardware processors includes a data transfer factor which indicates a burden of data movement between caches of the first and second cores.

8. The method of claim 6, further comprising:

ascertaining, for cache management in the stream computing environment, that a second cache size of the second cache of the second core of the set of many-core hardware processors exceeds a first cache size of the first cache of the first core of the set of many-core hardware processors; and

ascertaining, for cache management in the stream computing environment, that a second cache access burden of the second cache of the second core of the set of many-core hardware processors exceeds a first cache access burden of the first cache of the first core of the set of many-core hardware processors.

9. The method of claim 1, further comprising:

computing, for cache management in the stream computing environment, a third cache utilization factor for a third cache of a first core of the set of many-core hardware processors;

computing, for cache management in the stream computing environment, a third cache utilization factor for a third cache of a second core of the set of many-core hardware processors, wherein the third cache of the first core is a different physical cache from the third cache of the second core;

resolving, by comparing the third cache utilization factor for the third cache of the first core of the set of many-core hardware processors with the third cache utilization factor for the third cache of the second core of the set of many-core hardware processors, the tuple-processing hardware-route to prioritize utilization of the third cache of the first core of the set of many-core hardware processors with respect to the third cache of the second core of the set of many-core hardware processors; and

prioritizing, pertaining to processing the stream of tuples utilizing the set of many-core hardware processors, the third cache of the first core of the set of many-core hardware processors with respect to the third cache of the second core of the set of many-core hardware processors.

10. The method of claim 9, further comprising:

detecting, for cache management in the stream computing environment, that the third cache of the first core of the set of many-core hardware processors is shared with a third core of the set of many-core hardware processors; and

detecting, for cache management in the stream computing environment, that the third cache of the second core of the set of many-core hardware processors is neither shared with the first core of the set of many-core hardware processors nor the third core of the set of many-core hardware processors.

11. The method of claim 1, further comprising:

configuring, for cache management in the stream computing environment, the cache factor associated with the set of many-core hardware processors to include a set of physical distances between a plurality of cores of the set of many-core hardware processors; and

determining, using the set of physical distances, the tuple-processing hardware-route on the set of many-core hardware processors to route the stream of tuples a shorter distance between cores.

12. The method of claim 11, further comprising:

routing, for cache management in the stream computing environment, the stream of tuples based on a max-flow min-cut technique.

13. The method of claim 1, further comprising:

managing, for cache management in the stream computing environment, a set of cache hits by;

prioritizing, based on the cache factor associated with the set of many-core hardware processors, a single job tenancy characteristic; and

deterring, based on the cache factor associated with the set of many-core hardware processors, a multiple job tenancy characteristic.

14. The method of claim 1, further comprising:

detecting that the set of many-core hardware processors is selected from the group consisting of;

one or more ring architectures;

one or more two-dimensional mesh architectures;

one or more three-dimensional mesh architectures; and

one or more torus architectures.

15. The method of claim 1, further comprising:

determining, based on the tuple-processing hardware-route on the set of many-core hardware processors, a placement arrangement for a set of stream operators on the set of many-core hardware processors;

compiling, for cache management in the stream computing environment, a stream computing application which uses the placement arrangement for the set of stream operators on the set of many-core hardware processors; and

carrying-out the stream computing application utilizing the set of many-core hardware processors to process the stream of tuples by the plurality of processing elements which operate on the set of many-core hardware processors.

16. The method of claim 15, further comprising:

calculating, for cache management in the stream computing environment, an expected data rate for operation of a plurality of sequential stream operators;

identifying, for cache management in the stream computing environment, that a threshold data rate for a single core of the set of many-core hardware processors exceeds the expected data rate for operation of the plurality of sequential stream operators; and

structuring, for cache management in the stream computing environment, the placement arrangement to deploy the plurality of sequential stream operators on the single core of the set of many-core hardware processors.

17. The method of claim 1, further comprising:

executing, in a dynamic fashion to streamline cache management in the stream computing environment that uses the set of many-core hardware processors to process the stream of tuples by the plurality of processing elements which operate on the set of many-core hardware processors, each of;

the receiving, the determining, the routing, and the processing.

18. The method of claim 1, further comprising:

executing, in an automated fashion without user intervention, each of;

the receiving, the determining, the routing, and the processing.

19. A system for cache management in a stream computing environment that uses a set of many-core hardware processors to process a stream of tuples by a plurality of processing elements which operate on the set of many-core hardware processors, the system comprising:

a memory having a set of computer readable computer instructions, and

a processor for executing the set of computer readable instructions, the set of computer readable instructions including;

receiving the stream of tuples to be processed by the plurality of processing elements which operate on the set of many-core hardware processors;

determining, based on a cache factor associated with the set of many-core hardware processors, a tuple-processing hardware-route on the set of many-core hardware processors;

routing, based on the tuple-processing hardware-route on the set of many-core hardware processors, the stream of tuples; and

processing, utilizing the set of many-core hardware processors, the stream of tuples by the plurality of processing elements which operate on the set of many-core hardware processors.

20. A computer program product for cache management in a stream computing environment that uses a set of many-core hardware processors to process a stream of tuples by a plurality of processing elements which operate on the set of many-core hardware processors, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

receiving the stream of tuples to be processed by the plurality of processing elements which operate on the set of many-core hardware processors;

determining, based on a cache factor associated with the set of many-core hardware processors, a tuple-processing hardware-route on the set of many-core hardware processors;

routing, based on the tuple-processing hardware-route on the set of many-core hardware processors, the stream of tuples; and

processing, utilizing the set of many-core hardware processors, the stream of tuples by the plurality of processing elements which operate on the set of many-core hardware processors.