SYSTEM AND METHOD FOR AGGREGATING QUERY RESULTS IN A FAULT-TOLERANT DATABASE MANAGEMENT SYSTEM

• US 20120078881A1
• Filed: 09/24/2010
• Published: 03/29/2012
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: Hitachi Vantara, LLC (Hitachi, Ltd.)
• Original Assignee: Hitachi Data Systems Corporation (Hitachi, Ltd.)
• Inventors: CRUMP, Jeffrey M., IOAKIMIDIS, Margie E.
• Granted Patent: US 8,838,624 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A redundant array of independent nodes are networked together. Each node executes an instance of an application that provides object-based storage. Metadata objects are stored in a set of regions distributed among the nodes across the array. A given region is identified by hashing a metadata object attribute and extracting a given set of bits of a resulting hash value. A method of managing query results comprises: receiving, by a first node of the plurality of independent nodes from a client application, a request for a list of objects with a criterion; issuing by the first node a query to all the nodes based on the received request; processing the query by each node over the regions in the node using the metadata objects stored in the regions; aggregating and filtering by the first node results of the query from all the nodes; and returning by the first node the aggregated and filtered results to the client application.

Claims

1. In a redundant array of independent nodes networked together, wherein each node executes an instance of an application that provides object-based storage, wherein metadata objects are stored in a set of regions distributed among the nodes across the array, wherein a given region is identified by hashing a metadata object attribute and extracting a given set of bits of a resulting hash value, a method of managing query results comprising:

receiving, by a first node of the plurality of independent nodes from a client application, a request for a list of objects with a criterion;

issuing by the first node a query to all the nodes based on the received request;

processing the query by each node over the regions in the node using the metadata objects stored in the regions;

aggregating and filtering by the first node results of the query from all the nodes; and

returning by the first node the aggregated and filtered results to the client application.

2. The method according to claim 1,

wherein processing the query by each node comprises providing the results of the query in sets to the first node;

and

wherein the results of the query from all the nodes are aggregated and filtered and returned to the client application in sets.

3. The method according to claim 2, further comprising:

after returning by the first node a current set of the aggregated and filtered results to the client application, awaiting a request from the client application for a next set of results before requesting and retrieving the next set of results from all the nodes.

4. The method according to claim 2,

wherein providing the results of the query in sets to the first node comprises providing a preset number of objects from each region as a result of processing the query.

5. The method according to claim 1, further comprising:

sorting by the first node the aggregated and filtered results to produce a preset ordering.

6. The method according to claim 1,

wherein the request includes one or more of query by change time, query by directory, query by transaction, query by namespace, and page through results.

7. An apparatus for managing query results in a redundant array of independent nodes networked together, wherein each node executes an instance of an application that provides object-based storage, wherein metadata objects are stored in a set of regions distributed among the nodes across the array, wherein a given region is identified by hashing a metadata object attribute and extracting a given set of bits of a resulting hash value, the apparatus comprising a processor, a memory, and a query results management module provided for each of the nodes, the query results management module being configured to:

if the node having the query results management module is a first node which receives from a client application a request for a list of objects with a criterion, issue a query to all the nodes based on the received request;

process the query over the regions in the node using the metadata objects stored in the regions; and

if the node having the query results management module is the first node, aggregate and filter results of the query from all the nodes, the aggregated and filtered results to be returned to the client application.

8. The apparatus according to claim 7,

wherein processing the query comprises providing the results of the query in sets to the first node;

and

wherein the results of the query from all the nodes are aggregated and filtered to be returned to the client application in sets.

9. The apparatus according to claim 8,

wherein if the node having the query results management module is the first node, the query results management module is configured, after returning by the first node a current set of the aggregated and filtered results to the client application, to await a request from the client application for a next set of results before requesting and retrieving the next set of results from all the nodes.

10. The apparatus according to claim 8,

wherein providing the results of the query in sets to the first node comprises providing a preset number of objects from each region as a result of processing the query.

11. The apparatus according to claim 7,

wherein if the node having the query results management module is the first node, the query results management module is configured to sort the aggregated and filtered results to produce a preset ordering.

12. A node in the redundant array of independent nodes according to claim 7, the node comprising a metadata manager to manage the metadata objects in the node which includes organizing and providing access to the metadata objects, wherein the metadata manager includes the query results management module of the node.

13. A computer-readable storage medium storing a plurality of instructions for controlling a data processor to manage query results in a redundant array of independent nodes networked together, wherein each node executes an instance of an application that provides object-based storage, wherein metadata objects are stored in a set of regions distributed among the nodes across the array, wherein a given region is identified by hashing a metadata object attribute and extracting a given set of bits of a resulting hash value, wherein the computer-readable storage medium is provided in each node, the plurality of instructions comprising:

instructions that cause the data processor, if the node having the computer-readable storage medium is a first node which receives from a client application a request for a list of objects with a criterion, to issue a query to all the nodes based on the received request;

instructions that cause the data processor to process the query over the regions in the node using the metadata objects stored in the regions; and

instructions that cause the data processor, if the node having the computer-readable storage medium is the first node, to aggregate and filter by the first node results of the query from all the nodes, the aggregated and filtered results to be returned to the client application.

14. The computer-readable storage medium according to claim 13,

wherein processing the query comprises providing the results of the query in sets to the first node;

and

wherein the results of the query from all the nodes are aggregated and filtered and returned to the client application in sets.

15. The computer-readable storage medium according to claim 14, wherein the plurality of instructions further comprise:

instructions that cause the data processor, if the node having the computer-readable storage medium is the first node, after returning by the first node a current set of the aggregated and filtered results to the client application, to await a request from the client application for a next set of results before requesting and retrieving the next set of results from all the nodes.

16. The computer-readable storage medium according to claim 14,

wherein providing the results of the query in sets to the first node comprises providing a preset number of objects from each region as a result of processing the query.

17. The computer-readable storage medium according to claim 13, wherein the plurality of instructions further comprise:

instructions that cause the data processor, if the node having the computer-readable storage medium is the first node, to sort by the first node the aggregated and filtered results to produce a preset ordering.

18. The computer-readable storage medium according to claim 13,

wherein the request includes one or more of query by change time, query by directory, query by transaction, query by namespace, and page through results.