METHOD AND APPARATUS FOR PROVIDING A FILTER JOIN ON DATA STREAMS

• US 20110131198A1
• Filed: 11/30/2009
• Published: 06/02/2011
• Est. Priority Date: 11/30/2009
• Status: Abandoned Application

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: JOHNSON, THEODORE, Rozenbaum, Irina
• Application Number:
  US12/627,079
• Publication Number:
  US 20110131198A1
• Time in Patent Office: Days
• US Class Current: 707/714
• CPC Class Codes: G06F 16/2456 Join operations
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries

Abstract

A method and apparatus for processing at least one data stream are disclosed. For example, the method receives at least a join query for the at least one data stream, wherein the join query specifies a lifetime for keeping a tuple as a marker for a beginning of a sequence of interest, and receives a tuple from the at least one data stream. The method marks the tuple as a beginning of a sequence of interest and stores the tuple, if the tuple is the beginning of the sequence of interest. The method applies one or more initial predicates to the tuple, and determines if the tuple matched a marked tuple, if the tuple meets the one or more initial predicates. The method determines if the tuple meets one or more conditions to be outputted, if the tuple meets the one or more initial predicates conditions.

Claims

1. A method for processing at least one data stream, comprising:

receiving at least a join query for the at least one data stream, wherein said join query specifies a lifetime for keeping a tuple as a marker for a beginning of a sequence of interest;

receiving a tuple from the at least one data stream;

marking said tuple as a beginning of a sequence of interest and storing said tuple in accordance with the lifetime specified in said join query, if said tuple is said beginning of said sequence of interest;

applying one or more initial predicates to said tuple;

determining if said tuple matched a marked tuple, if said tuple meets the one or more initial predicates.

determining if said tuple meets one or more conditions to be outputted, if said tuple meets said one or more initial predicates conditions; and

outputting said tuple as a result to said join query if said tuple meets said one or more conditions to be outputted.

2. The method of claim 1, further comprising:

optimizing the join query based on information pertaining to a data source.

3. The method of claim 2, wherein said information pertaining to said data source comprises that two or more data streams of said at least one data stream having a same data source.

4. The method of claim 1, further comprising:

performing one or more query transformations such that said join query is performed by said one or more low level queries, while a complex analysis is performed by said one or more high level queries.

5. The method of claim 1, further comprising:

decomposing said join query if said join query contains a reference to one or more attributes which are not part of a restricted set of attributes of a data stream maintained by a data summary structure.

6. The method of claim 5, wherein said decomposing of said join query is performed to use a filter join at said one or more low level queries and a symmetric hash join at said one or more high level queries.

7. The method of claim 1, wherein each of said one or more low level queries applies a filter join that is implemented using a hash join algorithm or an approximate filter join algorithm.

8. The method of claim 7, wherein said approximate filter join algorithm uses a hash table or a bloom filter as a data summary structure.

9. The method of claim 8, further comprising:

performing collision handling if a slot of said hash table is occupied by a valid tuple.

10. A computer-readable storage medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform steps of a method for processing at least one data stream, comprising:

receiving at least a join query for the at least one data stream, wherein said join query specifies a lifetime for keeping a tuple as a marker for a beginning of a sequence of interest;

receiving a tuple from the at least one data stream;

marking said tuple as a beginning of a sequence of interest and storing said tuple in accordance with the lifetime specified in said join query, if said tuple is said beginning of said sequence of interest;

applying one or more initial predicates to said tuple;

determining if said tuple matched a marked tuple, if said tuple meets the one or more initial predicates.

determining if said tuple meets one or more conditions to be outputted, if said tuple meets said one or more initial predicates conditions; and

outputting said tuple as a result to said join query if said tuple meets said one or more conditions to be outputted.

11. The computer-readable storage medium of claim 10, further comprising:

optimizing the join query based on information pertaining to a data source.

12. The computer-readable storage medium of claim 11, wherein said information pertaining to said data source comprises that two or more data streams of said at least one data stream having a same data source.

13. The computer-readable storage medium of claim 10, further comprising:

performing one or more query transformations such that said join query is performed by said one or more low level queries, while a complex analysis is performed by said one or more high level queries.

14. The computer-readable storage medium of claim 10, further comprising:

decomposing said join query if said join query contains a reference to one or more attributes which are not part of a restricted set of attributes of a data stream maintained by a data summary structure.

15. The computer-readable storage medium of claim 14, wherein said decomposing of said join query is performed to use a filter join at said one or more low level queries and a symmetric hash join at said one or more high level queries.

16. The computer-readable storage medium of claim 10, wherein each of said one or more low level queries applies a filter join that is implemented using a hash join algorithm or an approximate filter join algorithm.

17. The computer-readable storage medium of claim 16, wherein said approximate filter join algorithm uses a hash table or a bloom filter as a data summary structure.

18. The computer-readable storage medium of claim 17, further comprising:

performing collision handling if a slot of said hash table is occupied by a valid tuple.

19. An apparatus for processing at least one data stream, comprising:

means for receiving at least a join query for the at least one data stream, wherein said join query specifies a lifetime for keeping a tuple as a marker for a beginning of a sequence of interest;

means for receiving a tuple from the at least one data stream;

means for marking said tuple as a beginning of a sequence of interest and storing said tuple in accordance with the lifetime specified in said join query, if said tuple is said beginning of said sequence of interest;

means for applying one or more initial predicates to said tuple;

means for determining if said tuple matched a marked tuple, if said tuple meets the one or more initial predicates.

means for determining if said tuple meets one or more conditions to be outputted, if said tuple meets said one or more initial predicates conditions; and

means for outputting said tuple as a result to said join query if said tuple meets said one or more conditions to be outputted.

20. The apparatus of claim 19, further comprising:

means for optimizing the join query based on information pertaining to a data source.