Unified table query processing

• US 9,465,844 B2
• Filed: 03/15/2013
• Issued: 10/11/2016
• Est. Priority Date: 04/30/2012
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Faerber, Franz, Lee, Juchang, Schreter, Ivan
• Primary Examiner(s): Gofman, Alex
• Assistant Examiner(s): Gurmu, Muluemebet
• Application Number:
  US13/843,898
• Publication Number:
  US 20140122452A1
• Time in Patent Office: 1,306 Days
• Field of Search: 707/693
• US Class Current: 1/1
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2456 Join operations
• CPC Class Codes: G06F 16/24561 Intermediate data storage techniques for performance improvement

Abstract

A system and method of query processing in a multi-level storage system having a unified table architecture. A query is received by a common query execution engine connected with the unified table architecture, the query specifying a data record. The common query execution engine performs a look-up for the data record based on the query at the first level storage structure. If the data record is not present at the first level storage structure, the common query execution engine performs separate look-ups in each of the second level storage structure and the main store.

Claims

1. A method comprising:

storing incoming data requests in a logical row format as data records in a first level storage structure in an in-memory computing system having a multi-level storage architecture;

transforming the data records from the logical row format into a logical column format;

storing, in the in-memory computing system, a encoded data records in the logical column format in a second level storage structure in the multi-level storage architecture, the second level storage structure separate from the first level storage structure;

compressing the encoded data records for long-term storage;

storing, in the in-memory computing system, the compressed data records in the logical column format in a main store in the multi-level storage architecture, the main store separate from the first level storage structure and the second level storage structure;

receiving a query by a common query execution engine connected with a unified table architecture, the query specifying a data record;

performing, by the common query execution engine, a look-up for the data record based on the query at the first level storage structure; and

when the data record is not present at the first level storage structure, performing, by the common query execution engine, separate look-ups in each of the second level storage structure and the main store.

2. The method in accordance with claim 1, wherein the separate look-ups are performed simultaneously to produce two intermediate results.

3. The method in accordance with claim 2, further comprising executing a union of the two intermediate results.

4. The method in accordance with claim 3, further comprising triggering an on-demand mini-merge or an on-demand dictionary creation for the first level storage structure.

5. A computer implemented method comprising:

storing incoming data requests in a logical row format as data records in a first level storage structure in an in-memory computing system having a multi-level storage architecture;

transforming the data records from the logical row format into a logical column format;

storing, in the in-memory computing system, a encoded data records in the logical column format in a second level storage structure in the multi-level storage architecture, the second level storage structure separate from the first level storage structure;

compressing the encoded data records for long-term storage;

storing, in the in-memory computing system, the compressed data records in the logical column format in a main store in the multi-level storage architecture, the main store separate from the first level storage structure and the second level storage structure;

receiving a query by a common query execution engine connected with a unified table architecture, the query specifying a data record;

performing, by the common query execution engine, a look-up for the data record based on the query at the first level storage structure; and

when the data record is not present at the first level storage structure, performing, by the common query execution engine, separate look-ups in each of the second level storage structure and the main store.

6. The computer implemented method in accordance with claim 5, wherein the separate look-ups are performed simultaneously to produce two intermediate results.

7. The computer implemented method in accordance with claim 6, further comprising executing a union of the two intermediate results.

8. The computer implemented method in accordance with claim 7, further comprising triggering an on-demand mini-merge or an on-demand dictionary creation for the first level storage structure.

9. A system comprising:

at least one programmable processor;

a multi-level storage architecture, the multi-level storage architecture having a first level storage structure to store incoming data requests in a logical row format as data records, a second level storage structure to encode and store the data records in a logical column format, and a main store to compress and store the encoded data records for long-term storage; and

a machine-readable medium storing instructions that, when executed by the at least one processor, cause the at least one programmable processor to perform operations comprising;

storing incoming data requests in a logical row format as data records in the first level storage structure in an in-memory computing system;

transforming the data records from the logical row format into a logical column format;

storing, in the in-memory computing system, a encoded data records in the logical column format in the second level storage structure, the second level storage structure separate from the first level storage structure;

compressing the encoded data records for long-term storage;

storing, in the in-memory computing system, the compressed data records in the logical column format in the main store in the multi-level storage architecture, the main store separate from the first level storage structure and the second level storage structure;

receiving a query by a common query execution engine connected with a unified table architecture, the query specifying a data record;

performing, by the common query execution engine, a look-up for the data record based on the query at the first level storage structure; and

when the data record is not present at the first level storage structure, performing, by the common query execution engine, separate look-ups in each of the second level storage structure and the main store.

10. The system in accordance with claim 9, wherein the separate look-ups are performed simultaneously to produce two intermediate results.

11. The system in accordance with claim 10, wherein the operations further comprise executing a union of the two intermediate results.

12. The system in accordance with claim 11, wherein the operations further comprise triggering an on-demand mini-merge or an on-demand dictionary creation for the first level storage structure.