Stream processor with compiled programs

• US 20080134158A1
• Filed: 12/22/2006
• Published: 06/05/2008
• Est. Priority Date: 12/04/2006
• Status: Active Grant

• Current Assignee: Autoliv AB (Autoliv Incorporated), Cloud Software Group
• Original Assignee: StreamBase Systems, Inc. (Cloud Software Group)
• Inventors: Tibbetts, Richard S., Salz, Jonathan
• Granted Patent: US 9,038,041 B2
• Time in Patent Office: Days
• US Class Current: 717/148
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06F 8/41 Compilation

Abstract

A stream processing platform that provides fast execution of stream processing applications within a safe runtime environment. The platform includes a stream compiler that converts a representation of a stream processing application into executable program modules for a safe environment. The platform allows users to specify aspects of the program that contribute to generation of modules that execute as intended. A user may specify aspects to control a type of implementation for loops, order of execution for parallel paths, whether multiple instances of an operation can be performed in parallel or whether certain operations should be executed in separate threads. In addition, the stream compiler may generate executable modules in a way that cause a safe runtime environment to allocate memory or otherwise operate efficiently.

Claims

1. A method of generating machine-executable instructions for processing a stream of values, the method comprising:

receiving a representation of processing to be performed on at least one input stream to produce at least one output stream, the representation including a plurality of operators and data flow between the operators; and

compiling the representation into machine-executable instructions that, when executed, perform operations on a value of an input data stream of the at least one input stream, the operations being defined by the plurality of operators being executed to completion in accordance with the data flow.

2. The method of claim 1, wherein receiving a representation comprises receiving the representation through a graphical user interface.

3. The method of claim 1, wherein compiling the representation comprises:

generating a set of ordered statements in a source code for a computer programming language;

generating an executable representation of the ordered statements.

4. The method of claim 3, wherein:

generating a set of ordered statements comprises generating a JAVA program; and

generating an executable representation comprises compiling the set of ordered statements in a JAVA compiler.

5. The method of claim 1, wherein:

the method additionally comprises performing at least one intermediate representation of the representation; and

compiling comprises compiling an intermediate representation of the at least one intermediate representation.

6. The method of claim 1, wherein compiling the representation into machine-executable instructions comprises:

compiling the representation into machine-executable instructions that, in response to application of the value of an input data stream, perform the operations on the value of the input data stream, the operations producing a value of an output stream of the at least one output stream.

7. The method of claim 1, wherein compiling the representation into machine-executable instructions comprises operating a plurality of code generators, each code generator associated an operator of the plurality of operators.

8. The method of claim 1, wherein compiling the representation into machine-executable instructions comprises:

compiling the representation into machine-executable instructions that, when executed, performs to completion, without intervention of an operation scheduler, operations defined by the plurality of operators and the data flow on a value of an input data stream of the at least one input stream to produce a value of an output stream of the at least one output stream.

9. The method of claim 8, wherein:

the method further comprises receiving an indication of processing on the stream to be performed in a plurality of paths and an order associated with the plurality of paths; and

compiling the representation comprises generating the machine-executable instructions adapted to perform processing in each of the plurality of paths, the machine executable instructions executing processing in the order.

10. The method of claim 8, wherein:

receiving the indication of the processing and the order comprises receiving the indication through a graphical user interface, the graphical user interface comprising an icon having a component identifying an input stream and a plurality of components identifying outputs.

11. The method of claim 8, further comprising generating machine-executable instructions to perform at least one operation on the stream prior to processing in the plurality of paths.

12. The method of claim 1, further comprising:

receiving;

an indication of a portion of the plurality of operators to be performed repetitively; and

an indication of an implementation type of the portion;

generating machine-executable instructions to perform processing associated with the portion of the operators, the machine executable code, when executed, for performing steps comprising;

if the implementation type is a first type, recursively executing the operators of the portion; and

if the implementation type is a second type, non-recursively executing the operators of the portion.

13. The method of claim 12, wherein the portion comprises operators interconnected by the data flow in a loop.

14. The method of claim 13, wherein generating machine-executable instructions if the implementation type is a second type comprises:

identifying a stop condition;

generating machine-executable instructions to store values in the stream until the stop condition is met; and

generating machine-executable instructions to execute the indicated processing on the stored values.

15. A computer-readable medium comprising computer program modules that, when executed, perform a method of generating machine-executable instructions from user-specified instructions, the machine-executable instructions for execution in a safe runtime environment that represents objects of a complex type using a header and represents primitive types without a header in a heap, the user specified instructions referencing at least one object of a complex data type having a plurality of elements, the method comprising:

generating machine-executable instructions that cause the safe runtime environment to allocate a storage object identified with a header; and

generating machine-executable instructions in which values for the elements of the at least one object are accessed in a portion of the storage object using values of objects of a primitive type.

16. The method of claim 15, wherein:

generating machine-executable instructions that cause the safe runtime environment to allocate a storage object comprise the safe runtime environment to generate a byte array.

17. The method of claim 16, wherein:

the at least one object comprises a string; and

generating machine-executable instructions in which values for the elements of the at least one object are accessed in a portion of the storage object using values of objects of a primitive type comprises generating machine-executable instructions that store a value indicating a portion of the byte array used to store the string.

18. A method of operating a data processing device processing a stream of data, the stream comprising a plurality of successive values, each value having a plurality of elements associated therewith, the method comprising:

queuing the successive values in the stream in a plurality of queues formed in a machine-readable medium, each queue of the plurality of queues storing, for each of the successive values, an element of the plurality of elements.

19. The method of claim 18, wherein

the method further comprises generating the stream in a first thread;

and

queuing the successive values comprises queuing the successive values in a second thread.

20. The method of claim 19, wherein:

the method further comprises, from within the first thread, storing in a first computer-readable medium successive values of the stream; and

queuing each successive value of the stream comprises storing in a second computer-readable medium a plurality of pointers, each pointer referencing a location in the first computer-readable medium storing an element of the successive value.