MANAGING QUERIES

• US 20110153662A1
• Filed: 12/23/2010
• Published: 06/23/2011
• Est. Priority Date: 12/23/2009
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Maclean, John, Stanfill, Craig W.
• Granted Patent: US 10,459,915 B2
• Time in Patent Office: Days
• US Class Current: 707/774
• CPC Class Codes: G06F 16/24532 of parallel queries

Abstract

Managing queries performed on one or more data sources includes: storing at least a first query in a storage medium; selecting the first query for processing; instructing a query engine to process the first query on a first portion of data in the one or more data sources for a first query interval; receiving result data from the query engine based on processing the first query on the first portion of data; saving a state of the first query in the storage medium after the first query interval; instructing the query engine to process a second query during a second query interval after the first query interval; and instructing the query engine to process the first query on a second portion of data in the one or more data sources during a third query interval after the second query interval.

Claims

1. A method for managing queries performed on one or more data sources, including:

storing at least a first query in a storage medium;

selecting the first query for processing;

instructing a query engine to process the first query on a first portion of data in the one or more data sources for a first query interval;

receiving first result data from the query engine based on processing the first query on the first portion of data;

saving a state of the first query in the storage medium after the first query interval;

instructing the query engine to process a second query during a second query interval after the first query interval;

receiving second result data from the query engine based on processing the second query;

determining to suspend the second query based on the second result data; and

instructing the query engine to process the first query on a second portion of data in the one or more data sources during a third query interval after the second query interval.

2. The method of claim 1, further including

storing a priority associated with the first query in the storage medium;

changing the priority associated with the first query prior to selecting the first query for processing;

wherein selecting the first query for processing includes selecting the query based in part on the priority.

3. The method of claim 1, wherein the first query interval is defined by a predetermined amount of time.

4. The method of claim 3, wherein a priority of the first query affects how much of the data in the one or more data sources is included in the first portion of data on which the first query is executed for the first query interval.

5. The method of claim 1, wherein storing the first query includes storing a notification threshold of a quantity of the first result data to be available before a requester that provided the first query is notified.

6. The method of claim 5, further including notifying the requester when the quantity of the first result data exceeds the notification threshold, wherein saving the state of the first query includes storing the quantity of the first result data received from the query engine.

7. The method of claim 6, further including returning the first result data upon request from the requester;

and storing the quantity of the first result data returned to the requester in the storage medium.

8. The method of claim 7, wherein selecting the first query is based on the quantity of the first result data received from the query engine and the quantity of the first result data returned to the requester.

9. The method of claim 1, wherein saving the state of the first query includes:

instructing the query engine to suspend the first query; and

saving a state of the first query after the first query has been suspended.

10. The method of claim 9, wherein instructing the query engine to process the first query on the second portion of data includes:

loading the saved state of the first query; and

instructing the query engine to resume the first query.

11. The method of claim 9, wherein saving the state of the first query includes saving an offset into a secondary index.

12. The method of claim 11, wherein the secondary index is a block compressed indexed file.

13. The method of claim 1, further including dividing the first query into multiple subqueries, and instructing the query engine to process at least some of the subqueries concurrently.

14. The method of claim 1, wherein the second query is received and stored in storage medium after the first query interval begins.

15. The method of claim 1, wherein the second query is received and stored in storage medium before the first query interval begins.

16. A computer-readable medium storing a computer program for managing queries performed on one or more data sources, the computer program including instructions for causing a computer to:

store at least a first query in a storage medium;

select the first query for processing;

instruct a query engine to process the first query on a first portion of data in the one or more data sources for a first query interval;

receive first result data from the query engine based on processing the first query on the first portion of data;

save a state of the first query in the storage medium after the first query interval;

instruct the query engine to process a second query during a second query interval after the first query interval;

receive second result data from the query engine based on processing the second query;

determine to suspend the second query based on the second result data; and

instruct the query engine to process the first query on a second portion of data in the one or more data sources during a third query interval after the second query interval.

17. A system for managing queries performed on one or more data sources, the system including:

a storage medium storing at least a first query;

a query engine configured to process queries on data in the one or more data sources; and

a server configured to;

select the first query for processing;

instruct the query engine to process the first query on a first portion of data in the one or more data sources for a first query interval;

receive first result data from the query engine based on processing the first query on the first portion of data;

save a state of the first query in the storage medium after the first query interval;

instruct the query engine to process a second query during a second query interval after the first query interval;

receive second result data from the query engine based on processing the second query;

determine to suspend the second query based on the second result data; and

instruct the query engine to process the first query on a second portion of data in the one or more data sources during a third query interval after the second query interval.

18. (canceled)