MULTI-GRANULAR STREAM PROCESSING

• US 20110145429A1
• Filed: 12/15/2009
• Published: 06/16/2011
• Est. Priority Date: 12/15/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Krishnamurthy, Rajaram B., Parris, Carl J., Boice, Charles, Liu, Lurng-Kuo
• Granted Patent: US 8,892,762 B2
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: H04L 65/765 intermediate

Abstract

Stream processing is facilitated by distributing responsibility for processing the stream to multiple components of a computing environment. A programmable unit receives one or more streams and determines the operations to be performed for the one or more streams and which components of the computing environment are to perform those operations. It forwards data relating to the one or more streams to one or more components of the computing environment for processing and/or information purposes.

Claims

1. A computer program product for facilitating stream processing in a computing environment, said computer program product comprising:

a storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

obtaining, by a programmable unit, a description of one or more streams to be received by the programmable unit, the programmable unit programmed, at least in part, by a first component of the computing environment;

receiving, by the programmable unit, the one or more streams; and

processing by the programmable unit the one or more streams based on a type of the description obtained, wherein the processing includes at least one of;

providing data relating to the one or more streams to the first component;

performing one set of one or more operations for the one or more streams at the programmable unit; and

forwarding information relating to the one or more streams to a second component of the computing environment to perform another set of one or more operations for the one or more streams.

2. The computer program product of claim 1, wherein the obtaining comprises receiving a summary record from at least one source of the one or more streams, the summary record including one or more characteristics of the one or more streams.

3. The computer program product of claim 2, wherein the first component comprises a server coupled to the programmable unit, and wherein the providing data comprises providing the summary record to the server.

4. The computer program product of claim 3, wherein the method further comprises updating by the server one or more stream data structures, in response to receiving the summary record.

5. The computer program product of claim 1, wherein the obtaining comprises receiving a workload directive from the first component, the workload directive describing one or more characteristics relating to the one or more streams.

6. The computer program product of claim 1, wherein the information comprises a result of performing an operation on the one or more streams.

7. The computer program product of claim 1, wherein the processing further comprises determining by the programmable unit the one set of operations to be performed by the programmable unit and the another set of operations to be performed by the second component.

8. The computer program product of claim 7, wherein the determining is based at least in part on programming of the programmable unit by the first component.

9. The computer program product of claim 7, wherein the second component comprises one or more computing units of a stream cluster coupled to the programmable unit.

10. The computer program product of claim 9, wherein the one or more computing units comprise one or more accelerators.

11. The computer program product of claim 1, wherein the method further comprises providing output resulting from processing the stream to the first component, wherein the providing output is performed by at least one of the programmable unit and the second component.

12. A computer system for facilitating stream processing in a computing environment, the computer system comprising:

a memory; and

a processor in communications with the memory, wherein the computer system is capable of performing a method, said method comprising;

obtaining, by a programmable unit, a description of one or more streams to be received by the programmable unit, the programmable unit programmed, at least in part, by a first component of the computing environment;

receiving, by the programmable unit, the one or more streams; and

processing by the programmable unit the one or more streams based on a type of the description obtained, wherein the processing includes at least one of;

providing data relating to the one or more streams to the first component;

performing one set of one or more operations for the one or more streams at the programmable unit; and

forwarding information relating to the one or more streams to a second component of the computing environment to perform another set of one or more operations for the one or more streams.

13. The computer system of claim 12, wherein the obtaining comprises receiving a summary record from at least one source of the one or more streams, the summary record including one or more characteristics of the one or more streams.

14. The computer system of claim 12, wherein the obtaining comprises receiving a workload directive from the first component, the workload directive describing one or more characteristics relating to the one or more streams.

15. The computer system of claim 12, wherein the processing further comprises determining by the programmable unit the one set of operations to be performed by the programmable unit and the another set of operations to be performed by the second component.

16. The computer system of claim 15, wherein the determining is based at least in part on programming of the programmable unit by the first component.

17. The computer system of claim 12, wherein the method further comprises providing output resulting from processing the stream to the first component, wherein the providing output is performed by at least one of the programmable unit and the second component.

18. A method of facilitating stream processing in a computing environment, said method comprising:

obtaining, by a programmable unit, a description of one or more streams to be received by the programmable unit, the programmable unit programmed, at least in part, by a first component of the computing environment;

receiving, by the programmable unit, the one or more streams; and

processing by the programmable unit the one or more streams based on a type of the description obtained, wherein the processing includes at least one of;

providing data relating to the one or more streams to the first component;

performing one set of one or more operations for the one or more streams at the programmable unit; and

forwarding information relating to the one or more streams to a second component of the computing environment to perform another set of one or more operations for the one or more streams.

19. The method of claim 18, wherein the obtaining comprises receiving a summary record from at least one source of the one or more streams, the summary record including one or more characteristics of the one or more streams.

20. The method of claim 18, wherein the obtaining comprises receiving a workload directive from the first component, the workload directive describing one or more characteristics relating to the one or more streams.