SELECTIVE DATA MIRRORING FOR IN-MEMORY DATABASES

• US 20180260338A1
• Filed: 05/14/2018
• Published: 09/13/2018
• Est. Priority Date: 08/31/2015
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: KRISHNAPPA, CHINMAYI, MARWAH, VINEET, GANESH, AMIT
• Granted Patent: US 10,331,572 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 12/0802 Addressing of a memory level in which the access to the desired data or data block requires associative addressing means, e.g. caches
• CPC Class Codes: G06F 12/12 Replacement control
• CPC Class Codes: G06F 16/24552 Database cache management
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: H03M 7/30 Compression speech analysis-synthesis for redundancy reduction G10L19/00; for image communication H04N; Expansion; Suppression of unnecessary data, e.g. redundancy reduction

Abstract

Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data. Selection of data to be maintained in the volatile memory may be based on various factors. Once selected the data may also be compressed to save space in the volatile memory. The compression level may depend on one or more factors that are evaluated for the selected data. The factors for the selection and compression level of data may be periodically evaluated, and based on the evaluation, the selected data may be removed from the volatile memory or its compression level changed accordingly.

Claims

1. A computer-implemented method comprising:

maintaining, by a database server, a database on persistent storage;

wherein the database includes a data portion that has been designated as eligible for mirroring within volatile memory;

performing an evaluation, by the database server, of one or more factors relating to the data portion;

based on the evaluation, generating eligibility data that corresponds to the data portion;

wherein the eligibility data (a) reflects an estimated benefit of having the data portion mirrored in the volatile memory and (b) is determined by the database server based on the one or more factors;

based on the eligibility data, automatically changing a mirror status of the data portion;

wherein automatically changing the mirror status of the data portion includes one of;

when the data portion is not currently mirrored in the volatile memory, causing the data portion to be mirrored within the volatile memory, or

when the data portion is currently mirrored in the volatile memory, causing the data portion to be removed from the volatile memory.

2. The method of claim 1, wherein the one or more factors include at least one of:

access statistics associated with the data portion,

a datatype of the data portion,

operation statistics associated with the data portion, or

data statistics associated with the data portion.

3. The method of claim 1,

wherein the one or more factors include access statistics associated with the data portion;

wherein the access statistics associated with the data portion comprise heat map statistics of the data portion;

wherein the evaluation includes evaluating the heat map statistics of the data portion; and

the method further comprising;

if the heat map statistics indicate frequent read accesses to the data portion, then modifying the eligibility data to increase likelihood of the data portion to be loaded into the volatile memory,

if the heat map statistics indicate infrequent read accesses to the data portion, then modifying the eligibility data to decrease the likelihood of the data portion to be loaded into the volatile memory.

4. The method of claim 1, wherein:

the one or more factors include access statistics associated with the data portion;

the access statistics associated with the data portion comprise heat map statistics of the data portion;

the evaluation includes evaluating the heat map statistics of the data portion; and

the method further comprising, if the heat map statistics indicate frequent write accesses to the data portion, then modifying the eligibility data to decrease likelihood of the data portion to be loaded into the volatile memory.

5. The method of claim 1, wherein:

the one or more factors include a datatype of the data portion, and

the method further comprising, if the datatype is a large object type, modifying the eligibility data to decrease likelihood of the data portion to be loaded into the volatile memory.

6. The method of claim 1, wherein:

automatically changing the mirror status of the data portion includes, when the data portion is currently mirrored in the volatile memory, causing the data portion to be removed from the volatile memory; and

wherein causing the data portion to be removed from the volatile memory is performed in response to a purge event that causes one or more copies of mirrored data portions to be removed from the volatile memory.

6-1. The method of claim 6, further comprising:

performing a comparison between the eligibility data and a threshold value, and

based on the comparison, automatically determining that the data from the data portion is to be removed from the volatile memory.

8. The method of claim 1 wherein:

the data portion is a first data portion;

the database includes a second data portion, separate from the first data portion, that has been designated as eligible for mirroring within the volatile memory; and

the method further comprises;

storing particular eligibility data that corresponds to the second data portion, wherein the particular eligibility data reflects an estimated benefit of having the second data portion mirrored in the volatile memory;

based on the particular eligibility data, automatically determining that the data from the second data portion should be loaded into the volatile memory;

while data from the first data portion is mirrored in the volatile memory and the data from the second data portion is not mirrored in the volatile memory, the database server performing the steps of;

performing a comparison between the particular eligibility data and the eligibility data, and

based on the comparison, the database server determining that the data from the first data portion is to be removed from the volatile memory;

in response to a purge event that causes copies of mirrored data portions to be removed from the volatile memory, removing from the volatile memory, the data from the first data portion.

9. The method of claim 1, wherein:

automatically changing the mirror status of the data portion includes, when the data portion is not currently mirrored in the volatile memory, causing the data portion to be mirrored within the volatile memory; and

wherein causing the data portion to be mirrored within the volatile memory is performed in response to a load event that causes data portions to be mirrored within the volatile memory.

10. The method of claim 9, wherein loading, into the volatile memory, the data from the data portion comprises constructing an in-memory compression unit using the data from the data portion.

11. The method of claim 10, wherein, in the persistent storage, the data from the data portion is organized in a row-major format, and, within the in-memory compression unit, the data from the data portion is organized in a column-major format.

12. The method of claim 10, wherein:

the evaluation includes evaluating, for a certain data portion of the data portion, at least one of the following;

access statistics of the certain data portion,

a data type of the certain data portion, or

performance statistics for one or more compressed data portions that are similar to the certain data portion; and

the method further comprises;

based on the evaluation, modifying a compression level for the certain data portion;

compressing data in the certain data portion at the modified compression level; and

loading the compressed data into the volatile memory.

13. The method of claim 12, wherein the performance statistics, for the one or more compressed data portions that are similar to the certain data portion, are evaluated to indicate that a different compression level consumes less resources of the database server;

and modifying the compression level for the certain data portion to approach the different compression level.

14. The method of claim 1, wherein the database includes a particular data portion that has been also designated as eligible for mirroring within the volatile memory, and the method further comprises:

storing particular eligibility data that corresponds to the particular data portion, wherein the particular eligibility data reflects an estimated benefit of having the particular data portion mirrored in the volatile memory;

wherein the particular eligibility data are determined by the database server based on the one or more factors;

based on the particular eligibility data, automatically determining that the particular data portion should not be loaded into the volatile memory; and

in response to a load event that causes data portions to be mirrored within the volatile memory, loading, into the volatile memory, the data from the data portion without loading into volatile memory data from the particular data portion.

15. One or more non-transitory storage media storing instructions, which when executed by one or more hardware processors, cause:

maintaining, by a database server, a database on persistent storage;

wherein the database includes a data portion that has been designated as eligible for mirroring within volatile memory;

performing an evaluation, by the database server, of one or more factors relating to the data portion;

based on the evaluation, generating eligibility data that corresponds to the data portion;

wherein the eligibility data (a) reflects an estimated benefit of having the data portion mirrored in the volatile memory and (b) is determined by the database server based on the one or more factors;

based on the eligibility data, automatically changing a mirror status of the data portion;

wherein automatically changing the mirror status of the data portion includes one of;

when the data portion is not currently mirrored in the volatile memory, causing the data portion to be mirrored within the volatile memory, or

when the data portion is currently mirrored in the volatile memory, causing the data portion to be removed from the volatile memory.

16. The one or more non-transitory storage media of claim 15, wherein the one or more factors include at least one of:

access statistics associated with the data portion,

a datatype of the data portion,

operation statistics associated with the data portion, or

data statistics associated with the data portion.

17. The one or more non-transitory storage media of claim 15, wherein:

the one or more factors include access statistics associated with the data portion;

the access statistics associated with the data portion comprise heat map statistics of the data portion;

the evaluation includes evaluating the heat map statistics of the data portion; and

the instructions include instructions, which when executed by the one or more hardware processors, further cause;

if the heat map statistics indicate frequent read accesses to the data portion, then modifying the eligibility data to increase likelihood of the data portion to be loaded into the volatile memory,

if the heat map statistics indicate infrequent read accesses to the data portion, then modifying the eligibility data to decrease the likelihood of the data portion to be loaded into the volatile memory.

18. The one or more non-transitory storage media of claim 15, wherein:

the one or more factors include a datatype of the data portion, and

the instructions include instructions, which when executed by the one or more hardware processors, further cause, if the datatype is a large object type, modifying the eligibility data to decrease likelihood of the data portion to be loaded into the volatile memory.

19. The one or more non-transitory storage media of claim 15, wherein instructions, which when executed by the one or more hardware processors, cause automatically changing the mirror status of the data portion include instructions, which when executed by the one or more hardware processors, further cause:

when the data portion is currently mirrored in the volatile memory, causing the data portion to be removed from the volatile memory; and

wherein causing the data portion to be removed from the volatile memory is performed in response to a purge event that causes one or more copies of mirrored data portions to be removed from the volatile memory.

20. The one or more non-transitory storage media of claim 15, wherein instructions, which when executed by the one or more hardware processors, cause automatically changing the mirror status of the data portion include instructions, which when executed by the one or more hardware processors, further cause:

automatically changing the mirror status of the data portion includes, when the data portion is not currently mirrored in the volatile memory, causing the data portion to be mirrored within the volatile memory; and

wherein causing the data portion to be mirrored within the volatile memory is performed in response to a load event that causes data portions to be mirrored within the volatile memory.