DATABASE PROCESSING SYSTEM AND METHOD

• US 20100211577A1
• Filed: 08/19/2009
• Published: 08/19/2010
• Est. Priority Date: 02/13/2009
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: TOKUDA, Seisuke, USHIJIMA, Kazutomo, TANAKA, Michiko, SHIMIZU, Akira, FUJIWARA, Shinji, KAWAMURA, Nobuo
• Granted Patent: US 8,510,316 B2
• Time in Patent Office: Days
• US Class Current: 707/752
• CPC Class Codes: G06F 16/24537 of operators

Abstract

Provided is a database system in which sorting of query results is sped up. The database system stores storage location information in which storage locations of the pieces of data are recorded in a given order. When there is no second task, which is executed based on data that is fetched in a first task, whether every piece of data requested in a third task, which is executed before the first task, has been fetched is determined. In the case where every piece of data requested in the third task has been fetched, data fetched in the first task is output. In the case where some of the data requested in the third task has not been fetched, data fetched in the first task is kept in a temporary buffer until every piece of data requested in the third task is fetched, and then output.

Claims

1. A database processing system, comprising:

a computer which outputs all query results in response to query requests received from outside; and

a storage system which stores data to be output in accordance with the query requests,

wherein the computer comprises a first interface, which is coupled to the storage system, a processor, which is coupled to the first interface, and a memory, which is coupled to the processor,

wherein the storage system comprises a second interface, which is coupled to the computer, a control unit, which is coupled to the second interface, and storage, which is coupled to the control unit,

wherein the storage stores the data and storage location information in which storage locations of the data are recorded in a given order,

wherein the computer is configured to;

receive query requests for outputting query results in an order of values recorded in the storage location information;

store order relations of the values recorded in the storage location information in association with a plurality of data input requests for fetching the data in accordance with the query requests; and

send the plurality of data input requests to the storage system,

wherein the storage system is configured to send data that is fetched in accordance with the each of the plurality of received data input requests to the computer for each of the plurality of data input requests received from the computer, and

wherein the computer is configured to;

determine, before all of the plurality of data input requests sent are fulfilled, from the order relations whether or not a query result to be output next is found in already fetched partial data, which is only part of the pieces of data to be fetched by the storage system; and

start, in the case where the data to be output next is found, outputting the query result.

2. The database processing system according to claim 1,

wherein the memory includes a temporary buffer, which temporarily holds query results generated from the data that is fetched from the storage system in accordance with the query requests,

wherein the computer uses the processor to execute a database management system, which is stored in the memory and which receives the query requests, issues data input requests to the storage system to generate all query results, and outputs the all query results,

wherein the database management system has a query receiving module, which receives the query requests and creates a query plan as a procedure of generating query results, and a query executing module, which executes the query plan to generate all query results,

wherein the query plan is constituted of operations in which one of the data and the storage location information is accessed,

wherein the query executing module includes:

an operation executing module which executes, for each piece of data, the operations constituting the query plan in parallel, to thereby send the plurality of data input requests for fetching the data to the storage system, and generates query results from the data fetched in response to the plurality of data input requests;

a sort trigger information attaching module which creates sort trigger information indicating order relations that are specified in the query requests, based on the storage location information and an order of processing the operations; and

a partial sort executing module which outputs the query results as instructed by the query requests,

wherein the operation executing module is configured to output the generated query result and its associated sort trigger information to the partial sort executing module, and output sort trigger information for processing that has been ended without generating a query result to the partial sort executing module, and

wherein the partial sort executing module is configured to;

keep, in the memory, output determination information which holds a value of sort trigger information that is to be output next;

set, as an initial value of the output determination information, a value of sort trigger information that is to be output first;

receive one of a pair of the query result and the sort trigger information, and the sort trigger information, which is output from the operation executing module, and compare the sort trigger information against the output determination information;

store, in the case where the sort trigger information and the output determination information do not match, in the temporary buffer a temporary buffer element, which is a pair of the sort trigger information and the query result;

in the case where the sort trigger information and the output determination information match and the query result that is associated with the sort trigger information is found, output the query result, update the output determination information with a value of sort trigger information that is to be output next, and search the temporary buffer for a temporary buffer element that includes sort trigger information matching the updated output determination information; and

in the case where the temporary buffer element is found and the temporary buffer element includes a query result, output the query result, update the output determination information with a value of sort trigger information that is to be output next, search the temporary buffer again for a temporary buffer element that includes sort trigger information matching the updated output determination information, and repeat until a temporary buffer element that includes sort trigger information matching the output determination information is no longer found.

3. The database processing system according to claim 2,

wherein the operation executing module is configured to generate query results by executing operations, for each piece of one of the data and the storage location information independently, through a plurality of tasks that can be run in parallel,

wherein the plurality of tasks fetch, from the sort trigger information attaching module, sort trigger information that is associated with one of the data and the storage location information for which operations are being executed, and the query result and the sort trigger information are output in the case where the query result is to be completed, whereas the sort trigger information is output to the partial sort executing module in the case where the plurality of tasks are to be ended without generating the query result,

wherein the sort trigger information is structured such that values hierarchized based on precedence relations of operations in the query plan and corresponding to an access order in which one of the data and the storage location information is accessed are linked together, the values linking one hierarchy level to another, and

wherein the sort trigger information includes information that indicates last access on each hierarchy level.

4. The database processing system according to claim 2, wherein, in the case where capacity of data stored in the temporary buffer exceeds a first given threshold, the computer is configured to sort the received query results by using a temporary storage space which is allocated to external storage accessible to the computer.

5. The database processing system according to claim 4, wherein the computer is configured to receive setting of the first given threshold.

6. The database processing system according to claim 4, wherein the computer is configured to receive designation of the external storage.

7. The database processing system according to claim 3,

wherein the memory stores task management information for managing a task that is being executed by the processor, and

wherein the computer is configured to:

select, in the case where the capacity of data stored in the temporary buffer exceeds a second given threshold, from the task management information, a task executed prior to a task that holds sort trigger information to be output before sort trigger information that is held in the temporary buffer; and

execute the selected task before executing other tasks that are being executed.

8. The database processing system according to claim 7, wherein the computer is configured to receive setting of the second given threshold.

9. The database processing system according to claim 3,

wherein the memory stores task management information for managing a task that is being executed by the processor, and

wherein the computer is configured to:

select, in the case where the capacity of data stored in the temporary buffer exceeds a third given threshold, from the task management information, a task executed prior to a task that holds sort trigger information to be output before sort trigger information that is held in the temporary buffer; and

reissue data input requests that have been issued in the selected task, at a higher priority level than a priority level of data input requests that are output in other tasks.

10. The database processing system according to claim 9, wherein the computer is configured to receive setting of the third given threshold.

11. A computer which outputs all query results in response to query requests received from outside, the computer comprising:

a first interface coupled via a network to a storage system which stores data to be output in accordance with the query requests;

a processor coupled to the first interface; and

a memory coupled to the processor,

wherein the storage system comprises a second interface, which is coupled to the computer, a control unit, which is coupled to the second interface, and storage, which is coupled to the control unit,

wherein the storage stores the data and storage location information in which storage locations of the data are recorded in a given order,

wherein the processor is configured to;

receive query requests for outputting query results in an order of values recorded in the storage location information;

store order relations of the values recorded in the storage location information in association with a plurality of data input requests for fetching the data in accordance with the query requests;

send the plurality of data input requests to the storage system via the first interface;

receive, from the storage system via the first interface, data that is fetched from among the data stored in the storage system in accordance with each of the plurality of data input requests which are received from the computer;

determine, before all of the plurality of data input requests sent are fulfilled, from the order relations whether or not a query result to be output next is found in already fetched partial data, which is only part of the pieces of data to be fetched by the storage system; and

start, in the case where the data to be output next is found, outputting the query result.

12. The computer according to claim 11,

wherein the memory includes a temporary buffer, which temporarily holds query results generated from the data that is fetched from the storage system in accordance with the query requests,

wherein the processor executes a database management system, which is stored in the memory and which receives the query requests, issues data input requests to the storage system to generate all query results, and outputs the all query results,

wherein the database management system includes a query receiving module, which receives the query requests and creates a query plan as a procedure of generating query results, and a query executing module, which executes the query plan to generate all query results,

wherein the query plan is constituted of operations in which one of the data and the storage location information is accessed,

wherein the query executing module includes:

an operation executing module which executes, for each piece of data, the operations constituting the query plan in parallel, to thereby send the plurality of data input requests for fetching the data to the storage system, and generates query results from the data fetched in response to the plurality of data input requests;

a sort trigger information attaching module which creates sort trigger information indicating order relations that are specified in the query requests, based on the storage location information and an order of processing the operations; and

a partial sort executing module which outputs the query results as instructed by the query requests,

wherein the operation executing module outputs the generated query result and its associated sort trigger information to the partial sort executing module, and outputs sort trigger information for processing that has been ended without generating a query result to the partial sort executing module, and

wherein the partial sort executing module is configured to;

keep, in the memory, output determination information which holds a value of sort trigger information that is to be output next;

set, as an initial value of the output determination information, a value of sort trigger information that is to be output first;

receive one of a pair of the query result and the sort trigger information, and the sort trigger information, which is output from the operation executing module, and compare the sort trigger information against the output determination information;

store, in the case where the sort trigger information and the output determination information do not match, in the temporary buffer a temporary buffer element, which is a pair of the sort trigger information and the query result;

in the case where the sort trigger information and the output determination information match and the query result that is associated with the sort trigger information is found, output the query result, update the output determination information with a value of sort trigger information that is to be output next, and search the temporary buffer for a temporary buffer element that includes sort trigger information matching the updated output determination information; and

in the case where the temporary buffer element is found and the temporary buffer element includes a query result, output the query result, update the output determination information with a value of sort trigger information that is to be output next, search the temporary buffer again for a temporary buffer element that includes sort trigger information matching the updated output determination information, and repeat until a temporary buffer element that includes sort trigger information matching the output determination information is no longer found.

13. The computer according to claim 12,

wherein the operation executing module is configured to generate query results by executing operations, for each piece of one of the data and the storage location information independently, through a plurality of tasks that can be run in parallel,

wherein the plurality of tasks fetch, from the sort trigger information attaching module, sort trigger information that is associated with one of the data and the storage location information for which operations are being executed, and the query result and the sort trigger information are output in the case where the query result is to be completed, whereas the sort trigger information is output to the partial sort executing module in the case where the plurality of tasks are to be ended without generating the query result,

wherein the sort trigger information is structured such that values hierarchized based on precedence relations of operations in the query plan and corresponding to an access order in which one of the data and the storage location information is accessed are linked together, the values linking one hierarchy level to another, and

wherein the sort trigger information includes information that indicates last access on each hierarchy level.

14. The computer according to claim 13,

wherein the memory stores task management information for managing a task that is being executed by the processor, and

wherein the processor is configured to:

select, in the case where capacity of data stored in the temporary buffer exceeds a given threshold, from the task management information, a task that holds sort trigger information to be output before sort trigger information that is held in the temporary buffer; and

reissue data input requests that have been issued in the selected task, at a higher priority level than a priority level of data input requests that are output in other tasks.

15. A database processing method for a system having a computer, which outputs all query results in response to query requests received from outside, and a storage system, which stores data to be output in accordance with the query requests, the database processing method including the steps of:

receiving, by the computer, query requests for outputting query results in an order of values recorded in storage location information in which storage locations of the data are recorded in a given order;

keeping, by the computer, order relations of the values recorded in the storage location information in association with a plurality of data input requests for fetching the data in accordance with the query requests;

sending, by the computer, the plurality of data input requests to the storage system;

sending, by the storage system, for each of the plurality of data input requests received from the computer, data that is fetched in accordance with the received plurality of data input request to the computer;

determining, by the computer, before all of the plurality of data input requests sent are fulfilled, from the order relations whether or not a query result to be output next is found in already fetched partial data, which is only part of the pieces of data to be fetched by the storage system; and

starting, by the computer, in the case where the data to be output next is found, outputting the query result.