Method and system for accelerated stream processing
First Claim
1. A stream processing method:
- streaming a plurality of data events through a member of the group consisting of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), and (3) a chip multi-processor (CMP), wherein the member includes a data processing pipeline, the pipeline including a plurality of parallel processing paths, the parallel processing paths including a first processing path and a second processing path in parallel with the first processing path;
within the first processing path, the member (1) filtering the streaming data events to generate a first reduced stream of the data events, and (2) performing at least one processing operation on the first reduced event stream to generate a first plurality of results for association with the first reduced event stream, wherein the member performs the filtering step and the at least one processing operation within the first processing path in a pipelined manner such that the member performs the at least one processing operation on the first reduced event stream within the first processing path while simultaneously performing the filtering step within the first processing path on new streaming events;
within the second processing path, the member (1) filtering the streaming data events to generate a second reduced stream of the data events, and (2) performing at least one processing operation on the second reduced event stream to generate a second plurality of results for association with the second reduced event stream, wherein the member performs the filtering step and the at least one processing operation within the second processing path in a pipelined manner such that the member performs the at least one processing operation on the second reduced event stream within the second processing path while simultaneously performing the filtering step within the second processing path on new streaming events; and
the member performing the steps within the first processing path in parallel with the steps within the second processing path.
2 Assignments
0 Petitions
Accused Products
Abstract
Disclosed herein are methods and systems for hardware-accelerating various data processing operations in a rule-based decision-making system such as a business rules engine, an event stream processor, and a complex event stream processor. Preferably, incoming data streams are checked against a plurality of rule conditions. Among the data processing operations that are hardware-accelerated include rule condition check operations, filtering operations, and path merging operations. The rule condition check operations generate rule condition check results for the processed data streams, wherein the rule condition check results are indicative of any rule conditions which have been satisfied by the data streams. The generation of such results with a low degree of latency provides enterprises with the ability to perform timely decision-making based on the data present in received data streams.
341 Citations
83 Claims
-
1. A stream processing method:
-
streaming a plurality of data events through a member of the group consisting of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), and (3) a chip multi-processor (CMP), wherein the member includes a data processing pipeline, the pipeline including a plurality of parallel processing paths, the parallel processing paths including a first processing path and a second processing path in parallel with the first processing path; within the first processing path, the member (1) filtering the streaming data events to generate a first reduced stream of the data events, and (2) performing at least one processing operation on the first reduced event stream to generate a first plurality of results for association with the first reduced event stream, wherein the member performs the filtering step and the at least one processing operation within the first processing path in a pipelined manner such that the member performs the at least one processing operation on the first reduced event stream within the first processing path while simultaneously performing the filtering step within the first processing path on new streaming events; within the second processing path, the member (1) filtering the streaming data events to generate a second reduced stream of the data events, and (2) performing at least one processing operation on the second reduced event stream to generate a second plurality of results for association with the second reduced event stream, wherein the member performs the filtering step and the at least one processing operation within the second processing path in a pipelined manner such that the member performs the at least one processing operation on the second reduced event stream within the second processing path while simultaneously performing the filtering step within the second processing path on new streaming events; and the member performing the steps within the first processing path in parallel with the steps within the second processing path. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor; the coprocessor processing at least a portion of the bit stream against at least one rule condition, wherein the processing step comprises; performing a hardware-accelerated rule condition check operation on the bit stream portion that compares the bit stream portion with a plurality of rule conditions; and in response to the performing step resulting in a finding that the bit stream portion satisfies at least one of the rule conditions, generating a rule condition check result that is indicative of a satisfaction of the at least one rule condition; and enhancing the bit stream with at least one bit corresponding to the rule condition checking result. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor; the coprocessor processing at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition checking result for the bit stream portion, the rule condition checking result being indicative of a rule condition being satisfied; the coprocessor enhancing the bit stream with at least one bit corresponding to the rule condition checking result; passing the enhanced bit stream out of the coprocessor for post-processing that is based at least in part on the generated rule condition check result; and performing the post-processing, wherein the post-processing comprises (1) selecting a location within a network for storing at least a portion of the enhanced bit stream based at least in part on the generated rule condition check result and (2) storing that enhanced bit stream portion in the selected location. - View Dependent Claims (29)
-
-
30. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor, wherein the bit stream comprises a plurality of records, each record having at least one data field, the at least one data field having a data value; and the coprocessor processing at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition checking result for the bit stream portion, the rule condition checking result being indicative of a rule condition being satisfied; and wherein the processing step further comprises (1) delivering the received bit stream to a plurality parallel paths, and (2) independently performing the processing step within a plurality of the paths, and wherein the independently performing step comprises; within a first of the parallel paths, (1) filtering the bit stream to pass only data corresponding to at least one pre-selected data field, and (2) processing that passed data against at least one rule condition to thereby generate a rule condition check result for that passed data if that passed data satisfies that at least one rule condition; and within a second of the parallel paths, (1) filtering the bit stream to pass only data corresponding to at least one pre-selected data field that is different than the at least one pre-selected data field of the first path, and (2) processing that passed data against at least one rule condition to thereby generate a rule condition check result for that passed data if that passed data satisfies that at least one rule condition. - View Dependent Claims (31, 32, 33)
-
-
34. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor; and the coprocessor processing at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition checking result for the bit stream portion, the rule condition checking result being indicative of a rule condition being satisfied, wherein the processing step comprises; computing an aggregate value based at least in part upon the bit stream portion; determining whether a rule condition is satisfied based at least in part upon the computed aggregate value; and in response to a determination that that rule condition is satisfied, generating a rule condition check result indicative of that rule condition'"'"'s satisfaction; storing the aggregate value in a memory; and retrieving the aggregate value from the memory as additional bit stream portions are processed by the processing step. - View Dependent Claims (35)
-
-
36. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor, wherein the bit stream comprises a plurality of records, each record having at least one data field, the at least one data field having a data value; and the coprocessor processing at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition checking result for the bit stream portion, the rule condition checking result being indicative of a rule condition being satisfied, and wherein the processing step comprises; filtering the bit stream to pass only data corresponding to at least one pre-selected data field; performing a range check on the each field of the passed data to determine whether the passed data has a data value within a range of acceptable values corresponding to at least one range-based rule condition; and in response to a determination that that at least one range-based rule condition is satisfied, generating a rule condition check result indicative of that range-based rule condition'"'"'s satisfaction. - View Dependent Claims (37, 38, 39, 40, 41, 42)
-
-
43. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor; and the coprocessor processing at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition checking result for the bit stream portion, the rule condition checking result being indicative of a rule condition being satisfied, and wherein the processing step comprises performing an approximate join operation with the coprocessor on data within the received bit stream. - View Dependent Claims (44)
-
-
45. A method for processing a bit stream, the method comprising:
-
receiving a bit stream at a coprocessor; and the coprocessor processing at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition checking result for the bit stream portion, the rule condition checking result being indicative of a rule condition being satisfied, and wherein the processing step comprises performing a join operation with the coprocessor between data within the received bit stream and data stored in a database. - View Dependent Claims (46, 47, 62)
-
-
48. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor, wherein the coprocessor comprises a reconfigurable logic device, wherein the reconfigurable logic device has firmware deployed thereon; wherein the coprocessor is configured to accept a bit stream; wherein the firmware is configured to process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of the at least one rule condition being satisfied; and wherein the coprocessor is further configured to enhance the bit stream with at least one bit corresponding to the rule condition check result. - View Dependent Claims (49)
-
-
50. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor; wherein the coprocessor is configured to (1) accept a bit stream, (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied, and (3) enhance the bit stream with at least one bit corresponding to the rule condition check result; and wherein, as part of the process operation, the coprocessor is further configured to (1) perform a rule condition check operation on the bit stream portion that compares the bit stream portion with a plurality of rule conditions, and (2) in response to finding a match between the bit stream portion and at least one of the rule conditions, generate a rule condition check result that is indicative of a satisfaction of the at least one rule condition. - View Dependent Claims (51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 63, 64, 65, 66)
-
-
67. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor; wherein the coprocessor is configured to (1) accept a bit stream, (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied, and (3) pass the enhanced bit stream to the processor for post-processing that is based at least in part on the generated rule condition check result; and wherein the processor is configured to perform the post-processing, wherein the post-processing comprises (1) an operation to select a location within a network for storing at least a portion of the enhanced bit stream based at least in part on the generated rule condition check result and (2) an operation to store that enhanced bit stream portion in the selected location. - View Dependent Claims (68)
-
-
69. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor; wherein the coprocessor is configured to (1) accept a bit stream, wherein the bit stream comprises a plurality of records, each record having at least one data field, the at least one data field having a data value, and (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied; and wherein, as part of the process operation, the coprocessor is further configured to process bits within the accepted bit stream within a plurality of parallel paths, at least a plurality of the paths being configured to independently process at least a portion of the bit stream against a plurality of rule conditions such that the coprocessor is further configured to; within a first of the parallel paths, (1) filter the bit stream to pass only data corresponding to at least one pre-selected data field, and (2) process that passed data against at least one rule condition to thereby generate a rule condition check result for that passed data if that passed data satisfies that at least one rule condition; and within a second of the parallel paths, (1) filter the bit stream to pass only data corresponding to at least one pre-selected data field that is different than the at least one pre-selected data field of the first path, and (2) process that passed data against at least one rule condition to thereby generate a rule condition check result for that passed data if that passed data satisfies that at least one rule condition. - View Dependent Claims (70, 71, 72)
-
-
73. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor; wherein the coprocessor is configured to (1) accept a bit stream, and (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied; wherein, as part of the process operation, the coprocessor is further configured to compute an aggregate value based at least in part upon the bit stream portion; determine whether a rule condition is satisfied based at least in part upon the computed aggregate value; and in response to a determination that that rule condition is satisfied, generate a rule condition check result indicative of that rule condition'"'"'s satisfaction; and wherein the coprocessor is further configured to; store the aggregate value in memory; and retrieve the aggregate value from memory as additional bit stream portions are processed by the coprocessor. - View Dependent Claims (74, 75, 76, 77)
-
-
78. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor; wherein the coprocessor is configured to (1) accept a bit stream, wherein the bit stream comprises a plurality of records, each record having at least one data field, the at least one data field having a data value, and (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied; wherein, as part of the process operation, the coprocessor is further configured to; filter the bit stream to pass only data corresponding to at least one pre-selected data field; perform a range check on the each field of the passed data to determine whether the passed data has a data value within a range of acceptable values corresponding to at least one range-based rule condition; and in response to a determination that that at least one range-based rule condition is satisfied, generate a rule condition check result indicative of that range-based rule condition'"'"'s satisfaction. - View Dependent Claims (79)
-
-
80. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor; wherein the coprocessor is configured to (1) accept a bit stream, and (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied, and wherein, as part of the process operation, the coprocessor is further configured to perform an approximate join operation on data within the accepted bit stream. - View Dependent Claims (81)
-
-
82. A system for processing a bit stream, the system comprising:
-
a processor; and a coprocessor in communication with the processor and a database; wherein the coprocessor is configured to (1) accept a bit stream, and (2) process at least a portion of the bit stream against at least one rule condition to thereby generate a rule condition check result for the bit stream portion, the rule condition check result being indicative of a rule condition being satisfied; and wherein, as part of the process operation, the coprocessor is further configured to perform a join operation between data within the accepted bit stream and data stored in the database. - View Dependent Claims (83)
-
Specification