Disk optimized paging for column oriented databases

• US 9,430,508 B2
• Filed: 12/30/2013
• Issued: 08/30/2016
• Est. Priority Date: 12/30/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Petculescu, Cristian, Netz, Amir
• Primary Examiner(s): Woo, Isaac M
• Application Number:
  US14/144,374
• Publication Number:
  US 20150186410A1
• Time in Patent Office: 974 Days
• Field of Search: 707600-899
• US Class Current: 1/1
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof

Abstract

Implementing a database system using a plurality of sequentially ordered drives to store sequential portions of columns of the database, but where the database system is usable by a system configured for use with in-memory database systems. The method includes loading one or more columns of a database into main memory of a computing system as a table based database. The method further includes dividing a column of the database into sequential portions. Each of the sequential portions is of sufficient size to create efficient transfers using hard disk controllers to transfer an entire sequential portion. The method further includes writing each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion such that sequential portions of the column are on sequential drives.

Claims

1. In a computing environment, a method of implementing a database system using a plurality of sequentially ordered drives to store sequential portions of columns of the database, but where the database system is usable by a system configured for use with in-memory database systems, the method comprising:

loading one or more columns of a database into main memory of a computing system as a table based database;

dividing a column of the database into sequential portions, wherein each of the sequential portions is of sufficient size to create efficient transfers using hard disk controllers to transfer an entire sequential portion, and wherein the sufficient size of each of the sequential portions is configurable such that a size of a first sequential portion is different than a size of a second sequential portion; and

writing each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion such that sequential portions of the column are on sequential drives.

2. The method of claim 1, wherein each of the sequentially ordered drives is an SSD drive.

3. The method of claim 1, wherein each of the sequential portions is a segment.

4. The method of claim 1, wherein each of the drives stores database data on a file per column basis such that each drive has a single data file for a given column of the database irrespective of how many portions of a column are stored at a given drive.

5. The method of claim 1, further comprising, for one or more other columns, repeating the acts of dividing a column of the database into sequential portions and writing each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion.

6. The method of claim 1, further comprising, reading sequential portions from the drives into a ring buffer, the ring buffer being implemented in the main memory of the computing system.

7. The method of claim 6, further comprising determining that certain data in the ring buffer is hot data in that the data has a predetermined frequency of reuse, and as a result, transferring the certain data to another buffer implemented in the main memory of the computing system.

8. In a computing environment, a database system using a plurality of sequentially ordered drives to store sequential portions of columns of the database, but where the database system is usable by a system configured for use with in-memory database systems, the system comprising:

one or more processors;

a plurality of sequentially ordered drives;

main physical memory; and

one or more computer readable media, wherein the one or more computer readable media comprise computer executable instructions that are executable by at least one of the one or more processors to cause at least one of the one or more processors to perform the following;

load one or more columns of a database into the main memory of a database system as a table based database;

divide a column of the database into sequential portions, wherein each of the sequential portions is of sufficient size to create efficient transfers using hard disk controllers to transfer an entire sequential portion, and wherein the sufficient size of each of the sequential portions is configurable such that a size of a first sequential portion is different than a size of a second sequential portion; and

write each of the sequential portions sequentially onto the sequentially ordered drives in a round robin fashion such that sequential portions of the column are on sequential drives.

9. The system of claim 8, wherein each of the sequentially ordered drives is an SSD drive.

10. The system of claim 8, wherein each of the sequential portions is a segment.

11. The system of claim 8, wherein each of the drives stores database data on a file per column basis such that each drive has a single data file for a given column of the database irrespective of how many portions of a column are stored at a given drive.

12. The system of claim 8, wherein the one or more computer readable media further comprise computer executable instructions that are executable by the one or more processors to cause the following to be performed, for one or more other columns, repeat the acts of dividing a column of the database into sequential portions and write each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion.

13. The system of claim 8, further comprising, reading sequential portions from the drives into a ring buffer, the ring buffer being implemented in the main memory of the computing system.

14. The system of claim 13, further comprising determining that certain data in the ring buffer is hot data in that the data has a predetermined frequency of reuse, and as a result, transferring the certain data to another buffer implemented in the main memory of the computing system.

15. In a computing environment, a physical computer readable storage medium comprising computer executable instructions that are executable by one or more processors to cause the following to be performed to implement a database system using a plurality of sequentially ordered drives to store sequential portions of columns of the database, but where the database system is usable by a system configured for use with in-memory database systems:

load one or more columns of a database into main memory of a computing system as a table based database;

divide a column of the database into sequential portions, wherein each of the sequential portions is of sufficient size to create efficient transfers using hard disk controllers to transfer an entire sequential portion, and wherein the sufficient size of each of the sequential portions is configurable such that a size of a first sequential portion is different than a size of a second sequential portion; and

write each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion such that sequential portions of the column are on sequential drives.

16. The physical computer readable storage medium of claim 15, wherein each of the sequentially ordered drives is an SSD drive.

17. The physical computer readable storage medium of claim 15, wherein each of the drives stores database data on a file per column basis such that each drive has a single data file for a given column of the database irrespective of how many portions of a column are stored at a given drive.

18. The physical computer readable storage medium of claim 15, further comprising computer executable instructions that are executable by the one or more processors to cause the following to be performed, for one or more other columns, repeat the acts of dividing a column of the database into sequential portions and write each of the sequential portions sequentially onto sequentially ordered drives in a round robin fashion.

19. The physical computer readable storage medium of claim 15, further comprising computer executable instructions that are executable by the one or more processors to cause the following to be performed, read sequential portions from the drives into a ring buffer, the ring buffer being implemented in the main memory of the computing system.

20. The physical computer readable storage medium of claim 19, further comprising computer executable instructions that are executable by the one or more processors to cause the following to be performed, determine that certain data in the ring buffer is hot data in that the data has a predetermined frequency of reuse, and as a result, transfer the certain data to another buffer implemented in the main memory of the computing system.