Buffer pool extension for database server

• US 9,069,484 B2
• Filed: 03/13/2014
• Issued: 06/30/2015
• Est. Priority Date: 03/04/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Zhang, Chengliang, Krishnamurthy, Sadashivan, Reynya, Georgiy I., Verbitski, Alexandre, Celis, Pedro, Bradshaw, Dexter Paul
• Primary Examiner(s): NGUYEN, KIM T
• Application Number:
  US14/208,268
• Publication Number:
  US 20140195750A1
• Time in Patent Office: 474 Days
• Field of Search: 707/705, 707/748
• US Class Current: 1/1
• CPC Class Codes: G06F 12/121 using replacement algorithms
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

Aspects of the subject matter described herein relate to a buffer pool for a database system. In aspects, secondary memory such as solid state storage is used to extend the buffer pool of a database system. Thresholds such as hot, warm, and cold for classifying pages based on access history of the pages may be determined via a sampling algorithm. When a database system needs to free space in a buffer pool in main memory, a page may be evicted to the buffer pool in secondary memory or other storage based on how the page is classified and conditions of the secondary memory or other storage.

Claims

1. A method, implemented at a computer system that includes one or more processors, a first memory, a second memory that is slower than the first memory, and a third memory that is slower than the second memory, for associating pages of a buffer pool with different memories, the method comprising:

determining a first page access frequency threshold and a second page access frequency threshold for classifying a plurality of pages of the buffer pool; and

de-allocating a portion of the first memory by re-assigning pages from the first memory to both the second memory and the third memory, including;

preserving one or more first pages having corresponding page access frequencies greater than the first page access threshold within the first memory;

re-assigning one or more second pages from the first memory to the second memory based on the one or more second pages having corresponding page access frequencies that are less than the first page access threshold, but greater than the second page access threshold; and

re-assigning one or more third pages from the first memory to the third memory based on the one or more third pages having corresponding page access frequencies that are less than the second page access threshold.

2. The method of claim 1, wherein the first memory comprises volatile memory, and the second memory comprises solid-state storage.

3. The method of claim 2, wherein the first memory and the second memory form the buffer pool.

4. The method of claim 1, wherein when an input/output (I/O) threshold of the second memory is reached, a new I/O request is redirected to the third memory.

5. The method of claim 4, wherein the I/O threshold of the second memory is determined to have been reached when accesses to the second memory approach a read/write bandwidth of the second memory, or when accesses to the second memory approach a pre-defined threshold.

6. The method of claim 4, wherein the I/O threshold of the second memory is determined to have been reached based on one or more of an I/O response time, a number of I/Os, a pattern of accesses, or a frequency of accesses.

7. The method of claim 1, wherein a throughput of the third memory is less than a throughput of one or more of the first memory and the second memory.

8. The method of claim 1, wherein a throughput of the third memory is at least as large as a throughput of one or more of the first memory and the second memory.

9. The method of claim 8, wherein a response time of the third memory slower than a response time of one or more of the first memory and the second memory.

10. The method of claim 1, wherein metadata for each of the plurality of pages specifies one or more of (i) a timestamp that indicates a most recent time that data on the page was accessed, (ii) a first flag that indicates whether the page has been modified, (iii) a second flag that indicates whether a copy of the page is stored in the second memory, (iv) a counter that indicates a number of times the page has been accessed, or (v) a weighted value that indicates a frequency of access to the page with more weighting for more recent access.

11. A computer system, comprising:

one or more hardware processors;

a first memory, a second memory that is slower than the first memory, and a third memory that is slower than the second memory; and

one or more computer-readable media having stored thereon computer-executable instruction that, when executed by the one or more hardware processors, cause the computer system to associate pages of a buffer pool with different memories, including the following;

determining a first page access frequency threshold and a second page access frequency threshold for classifying a plurality of pages of the buffer pool; and

de-allocating a portion of the first memory by re-assigning pages from the first memory to both the second memory and the third memory, including;

preserving one or more first pages having corresponding page access frequencies greater than the first page access threshold within the first memory;

re-assigning one or more second pages from the first memory to the second memory based on the one or more second pages having corresponding page access frequencies that are less than the first page access threshold, but greater than the second page access threshold; and

re-assigning one or more third pages from the first memory to the third memory based on the one or more third pages having corresponding page access frequencies that are less than the second page access threshold.

12. The system of claim 11, wherein the first memory comprises volatile memory, and the second memory comprises solid-state storage.

13. The system of claim 12, wherein the first memory and the second memory form the buffer pool.

14. The system of claim 11, wherein when an input/output (I/O) threshold of the second memory is reached, a new I/O request is redirected to the third memory.

15. The system of claim 14, wherein the I/O threshold of the second memory is determined to have been reached when accesses to the second memory approach a read/write bandwidth of the second memory, or when accesses to the second memory approach a pre-defined threshold.

16. The system of claim 14, wherein the I/O threshold of the second memory is determined to have been reached based on one or more of an I/O response time, a number of I/Os, a pattern of accesses, or a frequency of accesses.

17. The system of claim 11, wherein a throughput of the third memory is less than a throughput of one or more of the first memory and the second memory.

18. The system of claim 11, wherein a throughput of the third memory is at least as large as a throughput of one or more of the first memory and the second memory.

19. The system of claim 18, wherein a response time of the third memory slower than a response time of one or more of the first memory and the second memory.

20. A computer program product comprising one or more hardware storage devices having stored thereon computer-executable instructions that, when executed by one or more processors of a computer system, cause the computer system to associate pages of a buffer pool with different memories, the method comprising:

determining a first page access frequency threshold and a second page access frequency threshold for classifying a plurality of pages of the buffer pool; and

de-allocating a portion of a first memory by re-assigning pages from the first memory to both a second memory that is slower than the first memory and a third memory that is slower than the second memory, including;

preserving one or more first pages having corresponding page access frequencies greater than the first page access threshold within the first memory;

re-assigning one or more second pages from the first memory to the second memory based on the one or more second pages having corresponding page access frequencies that are less than the first page access threshold, but greater than the second page access threshold; and

re-assigning one or more third pages from the first memory to the third memory based on the one or more third pages having corresponding access frequencies that are less than the second page access threshold.