METHOD AND SYSTEM FOR CONTINUOUS QUERY PROCESSING

• US 20130262502A1
• Filed: 03/30/2012
• Published: 10/03/2013
• Est. Priority Date: 03/30/2012
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC), Emirates Telecommunications Corporation, Khalifa University Of Science, Technology & Research
• Original Assignee: British Telecommunications PLC (BT Group PLC), Emirates Telecommunications Corporation, Khalifa University of Science and Technology
• Inventors: Majeed, Basim, Leida, Marcello, Afzal, Ali, Colombo, Maurizio
• Granted Patent: US 9,652,502 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/24565 Triggers; Constraints
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/80 of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML content-based retrieval of web data G06F16/95

Abstract

The invention relates to a method and system that provide a continuous querying mechanism that analyses real-time data streams to generate relevant results. It leverages data grid and event processing technology to provide a high performance and extremely scalable continuous querying solution. An embodiment of the invention provides a flexible querying language and data storage to allow users to register their interest in specific types of data, e.g. processes. The users are notified of matching results using an event-based mechanism. In order to efficiently analyse a large amount of data, the embodiment uses a grid infrastructure that allows storage of the data across many grid nodes and distributes the query execution workload, avoiding the bottleneck represented by constantly querying a database. Continuous queries are stored in a Continuous Query Registry and that Registry is invoked every time an insert or update event occurs on the triple store.

Claims

1. A method of processing a continuous query on a stored data set, the method including the steps of:

receiving a query designated as a continuous query and storing a query model representing that query containing one or more atoms which are individual queries on the data set;

when new data is added to the data set, or the data set is modified, checking the new or modified data against the atoms of the stored query model to determine if any part of the new or modified data matches an atom of the stored query model;

if a match is found, checking whether the new or modified data matches the stored query model as a whole;

if a match is found with the model as a whole, assigning any fixed values in the new or modified data that correspond to free variables in the query model to those free variables;

processing the query resulting from said assignment on the stored data set; and

returning the result of the query.

2. A method according to claim 1 wherein the data set is stored on a plurality of interconnected nodes and the processing of the query is performed by assigning the query to one of said nodes for execution by that node.

3. A method according to claim 2 wherein the node to which the query is assigned is selected at random.

4. A method according to claim 2 wherein the execution of the query by the node is performed asynchronously.

5. A method according to claim 1 wherein the method further includes the step of adding new data to the data set by bulk import.

6. A method according to claim 1 wherein the method further includes the steps of:

monitoring one or more processes;

extracting data from those processes for storage in the data set; and

storing said extracted data in the data set.

7. A method according to claim 6 wherein the processes are business processes and the data is business process data.

8. A method according to claim 1 wherein the received query further contains a designation of an observer, which is an object to which the results of the query are to be reported, and the step of returning the result sends the result of the query to the observer.

9. A method according to claim 1 wherein the received query further contains a time duration for the query, and the method further includes the step of, when the time duration has expired, stopping processing the stored query model.

10. A method according to claim 9 wherein step of stopping processing deletes the stored query model.

11. A method according to claim 1 wherein the queries are specified in SPARQL.

12. A method according to claim 1 wherein if the stored query model has a plurality of query paths, the steps of matching determine whether any of those paths do not match the new or modified data, and if any paths do not match, those paths are not executed in the step of processing.

13. A method according to claim 1 wherein the steps of checking, assigning, processing and returning are executed immediately on receipt of new data in the data set or modification of the data set.

14. A system for processing queries on a data set, the system including:

one or more nodes storing part of the data in the data set and capable of executing queries on the data of the data set;

a query engine which stores a query model representing a continuous query and containing one or more atoms which are individual queries on the data set,

wherein, when new data is added to the data set, or the data set is modified;

the node to which the data is added, or on which the data is modified sends the new or modified data to the query engine;

the query engine;

checks the new or modified data against the atoms of the stored query model to determine if any part of the new or modified data matches an atom of the stored query model;

if a match is found, checks whether the new or modified data matches the stored query model as a whole;

if a match is found with the model as a whole, assigns any fixed values in the new or modified data that correspond to free variables in the query model to those free variables;

processes the query resulting from said assignment on the stored data set; and

returns the result of the query.

15. A system according to claim 14 wherein the query engine processes the query resulting from said assignment by assigning it to one of said nodes.

16. A system according to claim 15 wherein said assignment is random.

17. A system according to claim 14 wherein said assignment is asynchronous.

18. A system according to claim 14 further including a process monitor which monitors one or more processes and extracts data from said processes and stores it in said nodes.

19. A system according to claim 14 further including a data importer for adding new data to said data set by storing it in said nodes.