SEMI-JOIN ACCELERATION

• US 20140095534A1
• Filed: 02/26/2013
• Published: 04/03/2014
• Est. Priority Date: 10/02/2012
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Aingaran, Kathirgamar, Swart, Garret F
• Granted Patent: US 9,292,569 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/082 Associative directories G06F12/0822 takes precedence
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 12/1408 by using cryptography for digital transmission H04L9/00
• CPC Class Codes: G06F 12/1475 in a virtual system, e.g. with translation means
• CPC Class Codes: G06F 13/102 where the programme performs an interfacing function, e.g. device driver G06F13/105 takes precedence; scheduling within device drivers G06F9/52; contention policies within device drivers G06F9/4881
• CPC Class Codes: G06F 13/362 with centralised access control
• CPC Class Codes: G06F 13/4068 Electrical coupling
• CPC Class Codes: G06F 15/17331 Distributed shared memory [DSM], e.g. remote direct memory access [RDMA]
• CPC Class Codes: G06F 15/7889 Reconfigurable logic implemented as a co-processor instruction execution using a coprocessor G06F9/3877
• CPC Class Codes: G06F 16/2237 Vectors, bitmaps or matrices
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24553 of query operations
• CPC Class Codes: G06F 16/24558 Binary matching operations
• CPC Class Codes: G06F 16/2456 Join operations
• CPC Class Codes: G06F 16/24561 Intermediate data storage techniques for performance improvement
• CPC Class Codes: G06F 16/24569 Query processing with adaptation to specific hardware, e.g. adapted for using GPUs or SSDs
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 21/6209 : to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 9/30065 : Loop control instructions; iterative instructions, e.g. LOOP, REPEAT
• CPC Class Codes: G06F 9/381 : Loop buffering
• CPC Class Codes: G06F 9/3851 : from multiple instruction streams, e.g. multistreaming
• CPC Class Codes: G06F 9/3867 : using instruction pipelines
• CPC Class Codes: G06F 9/546 : Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 69/14 : Multichannel or multilink protocols
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

Techniques for processing a query are provided. One or more operations that are required to process a query are performed by a coprocessor that is separate from a general purpose microprocessor that executes query processing software. The query processing software receives a query, determines one or more operations that are required to be executed to fully process the query, and issues one or more commands to one or more coprocessors that are programmed to perform one of the operations, such as a table scan operation and/or a lookup operation. The query processing software obtains results from the coprocessor(s) and performs one or more additional operations thereon to generate a final result of the query.

Claims

1. A method for processing a query, comprising:

determining that execution of the query involves a particular operation;

in response to determining that execution of the query involves a particular operation, generating a command that includes, as parameters of the command, first address data that is used to identify input data to be read by a coprocessor, and second address data that is used to identify a lookup vector;

causing the command to be stored in memory;

processing, by the coprocessor, the command by;

reading the command from the memory;

causing the input data to be read from a location indicated by the first address data, wherein the input data comprises a plurality of values;

causing the lookup vector to be read from a location that is indicated by the second address data;

for each value of the plurality of values, identifying, within the lookup vector, a result that corresponds to said each value;

generating a result data based on the identifying for each value of the plurality of values;

causing the result data to be stored;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, wherein:

the command further includes operation type data that indicates a type of operation to perform;

the coprocessor processing the command further by first identifying the operation type data to determine the type of operation.

3. The method of claim 2, further comprising:

using the operation type data to determine logic that is used to interpret the input data;

converting the input data from a first data type to a second data type that is different than the first data type.

4. The method of claim 1, wherein:

the input data comprises value data and count data;

an input value in the value data corresponds to data elements, a number of which is indicated by a count value in the count data;

identifying a result within the lookup vector comprises identifying a result that corresponds to the input value;

generating the result data comprises generating a result indication that indicates the result that corresponds to the input value;

generating the result data is performed without identifying, said number of times, a result that corresponds to the input value within the lookup vector.

5. The method of claim 1, wherein the command further includes destination data that indicates where the result data is to be stored.

6. The method of claim 1, wherein the storage is cache of a microprocessor.

7. The method of claim 1, wherein causing the result data to be stored further comprises causing, to be stored, a completion status that indicates that the scan operation command has been performed.

8. The method of claim 1, wherein:

determining that execution of the query involves a plurality of operations that includes the particular operation and one or more other operations;

the method further comprising;

retrieving the result data from storage;

after retrieving the result data from the storage, processing the one or more other operations that require the result data as input to the one or more other operations.

9. The method of claim 1, wherein:

the coprocessor is a first coprocessor of a plurality of coprocessors that are connected in a series;

causing the result data to be stored comprises causing the result data to be sent to a buffer of a second coprocessor of the plurality of coprocessors;

the method further comprising;

reading, by the second coprocessor, the result data from the buffer while the first coprocessor is executing a portion of the query, and

based on the result data, generating, by the second coprocessor, second result data.

10. The method of claim 1, wherein the steps of determining, generating, and causing are performed by a general purpose microprocessor executing a query execution engine, wherein the general purpose microprocessor is separate from the coprocessor.

11. The method of claim 1, wherein:

generating the command comprises generating a plurality of commands that includes the command;

causing the command to be stored in memory comprises, for each command of the plurality of commands, causing said each command to be stored in the memory;

each coprocessor of a plurality of coprocessors selects a command of the plurality of commands.

12. The method of claim 1, wherein:

the first address data that is included in the command includes one or more virtual addresses;

the method further comprising causing the one or more virtual addresses to be replaced with one or more physical addresses that the coprocessor uses to read the input data.

13. The method of claim 1, further comprising determining whether one or more criteria are satisfied, wherein causing the command to be sent to the coprocessor is only performed if the one or more criteria are satisfied.

14. The method of claim 13, wherein the one or more criteria is based on a size of the lookup vector or an amount of data that needs to be read in by the coprocessor to perform the particular operation.

15. The method of claim 1, wherein generating the result data based on the identifying comprises generating a bit vector, wherein each bit in the bit vector indicates whether a value of the plurality of values is found in the lookup vector.

16. A coprocessor that is configured to perform the steps of:

receiving a command (1) that was generated by a microprocessor that executes instructions related to query processing and (2) that includes, as parameters of the command, first address data that is used to identify input data to be read by a coprocessor, and second address data that is used to identify a lookup vector;

reading the command from the memory;

causing the input data to be read from a location indicated by the first address data, wherein the input data comprises a plurality of values;

causing the lookup vector to be read from a location that is indicated by the second address data;

for each value of the plurality of values, identifying, within the lookup vector, a result that corresponds to said each value;

generating a result data based on the identifying for each value of the plurality of values;

causing the microprocessor to be informed of the result data.

17. The coprocessor of claim 16, wherein:

the command further includes operation type data that indicates a type of operation to perform;

the coprocessor is further configured to perform the step of identifying the operation type data to determine the type of operation.

18. The coprocessor of claim 17, wherein the coprocessor is further configured to perform the step of:

using the operation type data to determine logic that is used to interpret the input data;

converting the input data from a first data type to a second data type that is different than the first data type.

19. The coprocessor of claim 16, wherein the command further includes destination data that indicates where the result data is to be stored.

20. The coprocessor of claim 16, wherein the storage is cache of the microprocessor.

21. The coprocessor of claim 16, wherein:

the input data comprises value data and count data;

an input value in the value data corresponds to data elements, a number of which is indicated by a count value in the count data;

identifying a result within the lookup vector comprises identifying a result that corresponds to the input value;

generating the result data comprises generating a result indication that indicates the result that corresponds to the input value;

generating the result data is performed without identifying, said number of times, a result that corresponds to the input value within the lookup vector.

22. The coprocessor of claim 16, wherein:

the coprocessor is a first coprocessor of a plurality of coprocessors that are connected in a series;

causing the result data to be stored comprises causing the result data to be sent to a buffer of a second coprocessor of the plurality of coprocessors.

23. The coprocessor of claim 16, wherein:

the first address data that is included in the command includes one or more virtual addresses;

the coprocessor is further configured to perform the step of causing the one or more virtual addresses to be replaced with one or more physical addresses that the coprocessor uses to read the input data.

24. The coprocessor of claim 16, wherein generating the result data based on the identifying comprises generating a bit vector, wherein each bit in the bit vector indicates whether a value of the plurality of values is found in the lookup vector.