Optimized rollover processes to accommodate a change in value identifier bit size and related system reload processes

• US 10,296,611 B2
• Filed: 11/25/2014
• Issued: 05/21/2019
• Est. Priority Date: 11/25/2014
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP SE
• Inventors: Wein, David, Andrei, Mihnea, Schreter, Ivan, Blanco, Rolando, Legler, Thomas
• Primary Examiner(s): Savla, Arpan P.
• Assistant Examiner(s): Vo, Cecile H
• Application Number:
  US14/553,844
• Publication Number:
  US 20160147801A1
• Time in Patent Office: 1,638 Days
• Field of Search: 707E17014, 707693, 707812, 707E17044, 707E17002, 707736, 707999101, 707E17045, 707E17055, 707E17089, 707E17117, 707E17125, 707E17141, 707600, 707741, 707E17005, 7079992, 707E1701, 707E17116, 707639, 707705, 707825, 707E17001, 707E17009, 707999201
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

The subject matter disclosed herein provides methods for reformatting a page due to a rollover. An in-memory array holding a column of data can be maintained. One or more pages can be maintained. Each page can have one or more rows for storing the column of data. The column of data in the in-memory array can be monitored for a change. A rollover can be performed on at least one of the pages based on the change. The rollover can reformat the at least one page by rewriting metadata associated with the at least one page. Related apparatus, systems, techniques, and articles are also described.

Claims

1. A method for implementation by one or more data processors forming part of at least one computing system, the method comprising:

maintaining an in-memory array holding a column of data;

maintaining one or more pages, each of the one or more pages having one or more rows for storing the column of data;

monitoring a change in the column of data in the in-memory array, the change including an increase in a quantity of bits forming each of a plurality of values in the column of data; and

in response to detecting the change in the column of data, performing a rollover on at least one of the one or more pages, the performing of the rollover comprising;

determining additional memory resources required for the rollover;

reformatting, based at least on the additional resources required for the rollover, the at least one page by at least rewriting metadata associated with the at least one page, the at least one page being reformatted to accommodate the increase in the quantity of bits forming each of the plurality of values in the column of data; and

materializing data from the in-memory array to the reformatted at least one page, the materialization of the data comprising copying, to the reformatted at least one page, at least a portion of the plurality of values in the column of data.

2. The method of claim 1, wherein the performing of the rollover further comprises:

determining that the at least page has not been materialized; and

performing the rollover on the at least one page based at least on the at least one page not having been materialized.

3. The method of claim 1, wherein the performing of the rollover further comprises:

determining, based at least on the additional resources required for the rollover, that the rollover requires one or more additional pages to accommodate a spillover from the at least one page, the spillover resulting from the increase in the quantity of bits forming each of the plurality of values in the column of data; and

allocating the one or more additional pages prior to the performing of the rollover.

4. The method of claim 3, further comprising:

determining, based at least on the additional resources required for the rollover, that the spillover from the at least one page impacts a succeeding page that has already been materialized;

in response to determining that the spillover from the at least one page impacts a succeeding page that has already been materialized,

creating a new set of pages,

formatting the new set of pages to accommodate the increase in the quantity of bits forming each of the plurality of values in the column of data,

materializing, to the new set of pages, at least the portion of the plurality of values from the column of data,

creating a link between a last non-rolled over page and the new set of pages, and

deleting the at least one page.

5. The method of claim 1, further comprising loading the at least one page to the in-memory array during a system restart.

6. The method of claim 5, wherein the loading comprises:

adding a new column to the in-memory array, a plurality of values in the new column populated with at least one of a null value and a default value.

7. The method of claim 5, wherein the loading comprises:

determining whether the column in the in-memory array has been deleted; and

not copying data from the at least one page based on the determining.

8. A non-transitory computer-readable medium containing instructions to configure a processor to perform operations comprising:

maintaining an in-memory array holding a column of data;

maintaining one or more pages, each of the one or more pages having one or more rows for storing the column of data;

monitoring a change in the column of data in the in-memory array, the change including an increase in a quantity of bits forming each of a plurality of values in the column of data; and

in response to detecting the change in the column of data, performing a rollover on at least one of the one or more pages, the performing of the rollover comprising;

determining additional memory resources required for the rollover;

reformatting, based at least on the additional resources required for the rollover, the at least one page by at least rewriting metadata associated with the at least one page, the at least one page being reformatted to accommodate the increase in the quantity of bits forming each of the plurality of values in the column of data; and

materializing data from the in-memory array to the reformatted at least one page, the materialization of the data comprising copying, to the reformatted at least one page, at least a portion of the plurality of values in the column of data.

9. The non-transitory computer-readable medium of claim 8, wherein the performing of the rollover further comprises:

determining that the at least page has not been materialized; and

performing the rollover on the at least one page based at least on the at least one page not having been materialized.

10. The non-transitory computer-readable medium of claim 8, the operations further comprising:

determining, based at least on the additional resources required for the rollover, that the rollover requires one or more additional pages to accommodate a spillover from the at least one page, the spillover resulting from the increase in the quantity of bits forming each of the plurality of values in the column of data; and

allocating the one or more additional pages prior to the performing of the rollover.

11. The non-transitory computer-readable medium of claim 10, the operations further comprising:

determining, based at least on the additional resources required for the rollover, that the spillover from the at least one page impacts a succeeding page that has already been materialized;

in response to determining that the spillover from the at least one page impacts a succeeding page that has already been materialized,

creating a new set of pages,

formatting the new set of pages to accommodate the increase in the quantity of bits forming each of the plurality of values in the column of data,

materializing, to the new set of pages, at least the portion of the plurality of values from the column of data,

creating a link between a last non-rolled over page and the new set of pages, and

deleting the at least one page.

12. The non-transitory computer-readable medium of claim 8, the operations further comprising loading the at least one page to the in-memory array during a system restart,

wherein the loading comprises one or more of the following:

adding a new column to the in-memory array, a plurality of values in the new column populated with at least one of a null value and a default value, and

determining whether the column in the in-memory array has been deleted, wherein data is not copied from the at least one page based on the determining.

13. A system comprising:

at least one processor; and

at least one memory, wherein the at least one processor and the at least one memory are configured to perform operations comprising;

maintaining an in-memory array holding a column of data;

maintaining one or more pages, each of the one or more pages having one or more rows for storing the column of data;

monitoring a change in the column of data in the in-memory array, the change including an increase in a quantity of bits forming each of a plurality of values in the column of data; and

in response to detecting the change in the column of data, performing a rollover on at least one of the one or more pages, the performing of the rollover comprising;

determining additional memory resources required for the rollover;

reformatting, based at least on the additional resources required for the rollover, the at least one page by at least rewriting metadata associated with the at least one page, the at least one page being reformatted to accommodate the increase in the quantity of bits forming each of the plurality of values in the column of data; and

materializing data from the in-memory array to the reformatted at least one page, the materialization of the data comprising copying, to the reformatted at least one page, at least a portion of the plurality of values in the column of data.

14. The system of claim 13, wherein the performing of the rollover further comprises:

determining that the at least pages has not been materialized; and

performing the rollover on the at least one page based at least on the at least one page not having been materialized.

15. The system of claim 13, the operations further comprising:

determining, based at least on the additional resources required for the rollover, that the rollover requires one or more additional pages to accommodate a spillover from the at least one page, the spillover resulting from the increase in the quantity of bits forming each of the plurality of values in the column of data; and

allocating the one or more additional pages prior to the performing of the rollover.

16. The system of claim 13, the operations further comprising:

determining, based at least on the additional resources required for the rollover, that the spillover from the at least one page impacts a succeeding page that has already been materialized;

in response to determining that the spillover from the at least one page impacts a succeeding page that has already been materialized,

creating a new set of pages,

formatting the new set of pages to accommodate the increase in the quantity of bits forming each of the plurality of values in the column of data,

materializing, to the new set of pages, at least the portion of the plurality of values from the column of data,

creating a link between a last non-rolled over page and the new set of pages, and

deleting the at least one page.

17. The system of claim 13, the operations further comprising loading the at least one page to the in-memory array during a system restart,

wherein the loading comprises one or more of the following:

adding a new column to the in-memory array, a plurality of values in the new column populated with at least one of a null value and a default value, and

determining whether the column in the in-memory array has been deleted, wherein data is not copied from the at least one page based on the determining.