Efficient data storage and retrieval using a heterogeneous main memory

• US 10,133,667 B2
• Filed: 09/06/2016
• Issued: 11/20/2018
• Est. Priority Date: 09/06/2016
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: ORCLE INTERNATIONAL CORPORATION
• Inventors: Mukherjee, Niloy, Lahiri, Tirthankar, Loaiza, Juan R., Kamp, Jesse, Gaharwar, Prashant, Lakshmanan, Hariharan, Shah, Dhruvil
• Primary Examiner(s): Nguyen, Hiep
• Application Number:
  US15/257,754
• Publication Number:
  US 20180067853A1
• Time in Patent Office: 805 Days
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Techniques related to efficient data storage and retrieval using a heterogeneous main memory are disclosed. A database includes a set of persistent format (PF) data that is stored on persistent storage in a persistent format. The database is maintained on the persistent storage and is accessible to a database server. The database server converts the set of PF data to sets of mirror format (MF) data and stores the MF data in a hierarchy of random-access memories (RAMs). Each RAM in the hierarchy has an associated latency that is different from a latency associated with any other RAM in the hierarchy. Storing the sets of MF data in the hierarchy of RAMs includes (1) selecting, based on one or more criteria, a respective RAM in the hierarchy to store each set of MF data and (2) storing said each set of MF data in the respective RAM.

Claims

1. A method comprising:

a database server maintaining a database, said database server comprising one or more central processing units (CPUs);

wherein said database includes a plurality of database tables stored in a plurality of data blocks on block-addressable persistent storage devices;

said database server storing, in a hierarchy of random-access memories (RAMs), sets of mirror format (MF) data produced by converting data in said plurality of data blocks to a mirror format, each RAM in said hierarchy of RAMs being byte addressable memory having an address space referenceable by software instructions executed by said one or more CPUs, each RAM in said hierarchy of RAMs having an associated latency different from a latency associated with any other RAM in said hierarchy of RAMs;

wherein each set of MF data of said sets of MF data store, in column-major form, data from a plurality of columns of said plurality of database tables;

wherein, for each set of MF data of said sets of MF data, storing includes;

selecting based on one or more criteria, by said database server, a respective RAM from said hierarchy to store said each set of MF data;

storing, by said database server, said each set of MF data in said respective RAM.

2. The method of claim 1, further comprising, after storing said each set of MF data in said respective RAM, automatically moving a particular set of MF data from a first RAM to a second RAM if one or more policy conditions are satisfied, said first RAM and said second RAM being different RAMs in said hierarchy of RAMs.

3. The method of claim 2, wherein satisfaction of said one or more policy conditions is indicated by one or more bitmaps.

4. The method of claim 2, wherein said one or more policy conditions comprise a predetermined amount of change in an access frequency of said particular set of MF data.

5. The method of claim 2, wherein said one or more policy conditions comprise a predetermined amount of change in an age of said particular set of MF data.

6. The method of claim 2, wherein said one or more policy conditions comprise a predetermined amount of change in an amount of available memory in said second RAM.

7. The method of claim 1, wherein said one or more criteria comprise a user-specified compression level for a particular set of MF data of said sets of MF data.

8. The method of claim 1, wherein said one or more criteria comprise a user-specified priority level for a particular set of MF data of said sets of MF data.

9. The method of claim 1, wherein said one or more criteria comprise a user-specified percentage of said sets of MF data to be stored in a particular RAM in said hierarchy of RAMs.

10. The method of claim 1, wherein said hierarchy of RAMs comprises a volatile RAM and a non-volatile RAM.

11. One or more non-transitory storage media storing a set of instructions which, when executed by one or more computing devices, cause:

a database server maintaining a database, said database server comprising one or more central processing units (CPUs);

wherein said database includes a plurality of database tables stored in a plurality of data blocks on block-addressable persistent storage devices;

said database server storing, in a hierarchy of random-access memories (RAMs), sets of mirror format (MF) data produced by converting data in said plurality of data blocks to a mirror format, each RAM in said hierarchy of RAMs being byte addressable memory having an address space referenceable by software instructions executed by said one or more CPUs, each RAM in said hierarchy of RAMs having an associated latency different from a latency associated with any other RAM in said hierarchy of RAMs;

wherein each set of MF data of said sets of MF data store, in column-major form, data from a plurality of columns of said plurality of database tables;

wherein, for each set of MF data of said sets of MF data, storing includes;

selecting based on one or more criteria, by said database server, a respective RAM from said hierarchy to store said each set of MF data;

storing, by said database server, said each set of MF data in said respective RAM.

12. The one or more non-transitory storage media of claim 11, wherein said set of instructions further comprises instructions which, when executed by said one or more computing devices, cause:

after storing said each set of MF data in said respective RAM, automatically moving a particular set of MF data from a first RAM to a second RAM if one or more policy conditions are satisfied, said first RAM and said second RAM being different RAMs in said hierarchy of RAMs.

13. The one or more non-transitory storage media of claim 12, wherein satisfaction of said one or more policy conditions is indicated by one or more bitmaps.

14. The one or more non-transitory storage media of claim 12, wherein said one or more policy conditions comprise a predetermined amount of change in an access frequency of said particular set of MF data.

15. The one or more non-transitory storage media of claim 12, wherein said one or more policy conditions comprise a predetermined amount of change in an age of said particular set of MF data.

16. The one or more non-transitory storage media of claim 12, wherein said one or more policy conditions comprise a predetermined amount of change in an amount of available memory in said second RAM.

17. The one or more non-transitory storage media of claim 11, wherein said one or more criteria comprise a user-specified compression level for a particular set of MF data of said sets of MF data.

18. The one or more non-transitory storage media of claim 11, wherein said one or more criteria comprise a user-specified priority level for a particular set of MF data of said sets of MF data.

19. The one or more non-transitory storage media of claim 11, wherein said one or more criteria comprise a user-specified percentage of said sets of MF data to be stored in a particular RAM in said hierarchy of RAMs.

20. The one or more non-transitory storage media of claim 11, wherein said hierarchy of RAMs comprises a volatile RAM and a non-volatile RAM.