Dual data storage using an in-memory array and an on-disk page structure

• US 9,891,831 B2
• Filed: 11/25/2014
• Issued: 02/13/2018
• Est. Priority Date: 11/25/2014
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Legler, Thomas, Wein, David, Schreter, Ivan, Andrei, Mihnea, Goel, Anil Kumar, Radestock, Guenter
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): TESSEMA, AIDA Z
• Application Number:
  US14/553,859
• Publication Number:
  US 20160147457A1
• Time in Patent Office: 1,176 Days
• Field of Search: None
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/90339 by using parallel associative memories or content-addressable memories
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The subject matter disclosed herein provides methods for the dual storage of data using an in-memory array and an on-disk page structure. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each of the one or more pages can have one or more rows for storing the column of data. Random access can be provided to a subset of the one or more rows by at least loading the subset of rows from the one or more pages to the in-memory array without loading all of the rows from the one or more pages. Related apparatus, systems, techniques, and articles are also described.

Claims

1. A method for implementation by one or more data processors forming part of at least one computing system, the method comprising:

storing an in-memory array to disk, the in-memory array including a portion of data from a column of data, and the in-memory array being stored to disk by at least;

determining, based at least on a content of the portion of data, a page on disk for storing the portion of data;

storing, in the page, the portion of data from the column of data, the portion of data being stored in a row in the page; and

providing random access to the portion of data from the column of data, the providing of the random access comprising;

identifying, based at least on the content of the portion of data, the page storing the portion of data; and

loading, from disk and into the in-memory array, the row from the page storing the portion of data, the row being loaded without loading another row from the page.

2. The method of claim 1, wherein the in-memory array has a contiguous block of memory addresses, and wherein the contiguous block of memory addresses supports random access of the data when all of the data is loaded into the in-memory array.

3. The method of claim 1, wherein the column of data in the in-memory array includes one or more value identifiers.

4. The method of claim 3, wherein the one or more value identifiers are encoded in a native format, and wherein the row and/or the other row from the page store the one or more value identifiers in the native format.

5. The method of claim 4, wherein the native format uses an N-bit encoding scheme.

6. The method of claim 3, wherein the row and/or the other row from the page do not store any data for the column when the one or more value identifiers are null values.

7. The method of claim 3, wherein the row and/or the other row from the page do not store any data for the column when an identity property applies to the column, and wherein the identity property applies to the column when each value identifier is equal in value to a row position associated with the value identifier.

8. The method of claim 1, wherein the loading is performed when the computing system is being restored.

21. The method of claim 1, wherein the page for storing the portion of data is determined and/or identified based at least on a hash of the content of the portion of data.

9. A non-transitory computer-readable medium containing instructions to configure a processor to perform operations comprising:

storing an in-memory array to disk, the in-memory array including a portion of data from a column of data, and the in-memory array being stored to disk by at least;

determining, based at least on a content of the portion of data, a page on disk for storing the portion of data;

storing, in the page, the portion of data from the column of data, the portion of data being stored in a row in the page; and

providing random access to the portion of data from the column of data, the providing of the random access comprising;

identifying, based at least on the content of the portion of data, the page storing the portion of data; and

loading, from disk and into the in-memory array, the row from the page storing the portion of data, the row being loaded without loading another row from the page.

10. The non-transitory computer-readable medium of claim 9, wherein the in-memory array has a contiguous block of memory addresses, and wherein the contiguous block of memory addresses supports random access of the data when all of the data is loaded into the in-memory array.

11. The non-transitory computer-readable medium of claim 9, wherein the column of data in the in-memory array includes one or more value identifiers.

12. The non-transitory computer-readable medium of claim 11, wherein the one or more value identifiers are encoded in a native format, and wherein the row and/or the other row from the page store the one or more value identifiers in the native format.

13. The non-transitory computer-readable medium of claim 11, the row and/or the other row from the page do not store any data for the column when the one or more value identifiers are null values.

14. The non-transitory computer-readable medium of claim 11, wherein the row and/or the other row from the page do not store any data for the column when an identity property applies to the column, and wherein the identity property applies to the column when each value identifier is equal in value to a row position associated with the value identifier.

15. A system comprising:

at least one processor; and

at least one memory, wherein the at least one processor and the at least one memory are configured to perform operations comprising;

storing an in-memory array to a disk, the in-memory array including a portion of data from a column of data, and the in-memory array being stored to disk by at least;

determining, based at least on a content of the portion of data, a page on disk for storing the portion of data;

storing, in the page, the portion of data from the column of data, the portion of data being stored in a row in the page; and

providing random access to the portion of data from the column of data, the providing of the random access comprising;

identifying, based at least on the content of the portion of data, the page storing the portion of data; and

loading, from the disk and into the in-memory array, the row from the page storing the portion of data, the row being loaded without loading another row from the page.

16. The system of claim 15, wherein the in-memory array has a contiguous block of memory addresses, and wherein the contiguous block of memory addresses supports random access of the data when all of the data is loaded into the in-memory array.

17. The system of claim 15, wherein the column of data in the in-memory array includes one or more value identifiers.

18. The system of claim 17, wherein the one or more value identifiers are encoded in a native format, and wherein the row and/or the other row from the page store the one or more value identifiers in the native format.

19. The system of claim 17, wherein the row and/or the other row from the page do not store any data for the column when the one or more value identifiers are null values.

20. The system of claim 17, wherein the row and/or the other row from the page do not store any data for the column when an identity property applies to the column, and wherein the identity property applies to the column when each value identifier is equal in value to a row position associated with the value identifier.