Method, apparatus and program storage device for managing buffers during online reorganization

• US 7,774,304 B2
• Filed: 01/31/2005
• Issued: 08/10/2010
• Est. Priority Date: 01/31/2005
• Status: Expired due to Fees

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Friske, Craig A., Banzon, Arnold T., Ruddy, James A., Ng, Ka C., Vizconde, Bituin B., Garth, John M.
• Primary Examiner(s): Pham; Khanh B
• Assistant Examiner(s): CHEEMA, AZAM M
• Application Number:
  US11/048,341
• Publication Number:
  US 20060173922A1
• Time in Patent Office: 2,017 Days
• Field of Search: None
• US Class Current: 707/609
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof

Abstract

A method, apparatus and program storage device for managing buffers during online reorganization. An adaptive buffer is provided having a dynamically adjustable boundary, the adaptive buffer processes log records and pointers associated with the log records during online reorganization of a database. Adaptive switching is provided between a first and a second task during the processing of data log records and index log records during the online reorganization of the database.

Claims

1. A program storage device, comprising:

program instructions executable by a processing device to perform operations for managing buffers during online reorganization, the operations comprising;

providing an adaptive buffer having a dynamically adjustable boundary between log records and pointers associated with the log records, the adaptive buffer for processing the log records and pointers associated with the log records during online reorganization of a database; and

adaptive switching between a first and a second task during the processing of the log records comprising data log records and index log records during the online reorganization of the database, further comprising;

begin processing, by a first task, records located in first buffer associated with first task set and processing, by a second task, records located in first buffer associated with second task set;

completing by first task the processing of records in first buffer associated with first task;

begin processing by first task, records located in second buffer associated with first task set without waiting for second task to complete processing records of first buffer associated with second task set;

completing processing records located in second buffer associated with first task by first task;

waiting, by first task, until second task completes the processing of records located in first buffer associated with second task set;

refilling, by first task, the first buffer associated with first task set and the first buffer associated with second task set;

determining whether the first task or the second task completes processing records in its first buffer next;

beginning to refill second buffer for both tasks by the task determined to next complete processing of its first buffer; and

processing, by task determined to next complete processing its first buffer, second buffer of task determined to next complete processing its first buffer.

2. The program storage device of claim 1, wherein providing an adaptive buffer having a dynamically adjustable boundary, the adaptive buffer for processing log records and pointers associated with the log records during online reorganization of a database further comprises dynamically positioning a boundary in the buffer, the boundary separating the adaptive buffer into a pointer area and a log record area.

3. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises filling the log record area from a single direction.

4. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises filling the log record area beginning at a first location in the log record area using table space log records and at a second location in the log record area using index log records.

5. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises filling the log record area with table space log records and index log records.

6. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises filling the pointer area with table space log record pointers and index log record pointers.

7. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises estimating a ratio of anticipated buffer space for pointers versus buffer space actually used for log records.

8. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises processing log apply functions using concurrently running tasks without contention on the same space in the buffer for each log apply function.

9. The program storage device of claim 2, wherein dynamically positioning a boundary in the buffer further comprises combining log apply tasks into a common task.

10. The program storage device of claim 9, wherein combining log apply tasks into a common task further comprises combining log reading, sorting, and log translation into a single task and providing separate tasks for a log apply for each object.

11. A program storage device comprising:

program instructions executable by a processing device to perform operations for managing buffers during online reorganization, the operations comprising;

providing an adaptive buffer having a dynamically adjustable boundary between log records and pointers associated with the log records, the adaptive buffer for processing the log records and pointers associated with the log records during online reorganization of a database; and

adaptive switching between a first and a second task during the processing of the log records comprising data log records and index log records during the online reorganization of the database, further comprising;

begin processing, by a first task, records located in first buffer associated with first task set and processing, by a second task, records located in first buffer associated with second task set;

completing by first task the processing of records in first buffer associated with first task;

begin processing by first task, records located in second buffer associated with first task set without waiting for second task to complete processing records of first buffer associated with second task set;

completing processing records located in second buffer associated with first task by first task;

waiting, by first task, until second task completes the processing of records located in first buffer associated with second task set;

refilling, by first task, the first buffer associated with first task set and the first buffer associated with second task set;

determining whether the first task or the second task completes processing records in its second buffer next;

beginning to refill first buffer for both tasks by the task determined to first complete processing of its second buffer; and

processing, by task determined to next complete processing its second buffer, first buffer of task determined to next complete processing its second buffer.

12. A program storage device, comprising:

program instructions executable by a processing device to perform operations for managing buffers during online reorganization, the operations comprising;

begin processing, by a first task, records located in first buffer associated with first task set and processing, by a second task, records located in first buffer associated with second task set;

completing by first task the processing of records in first buffer associated with first task;

begin processing by first task, records located in second buffer associated with first task set without waiting for second task to complete processing records of first buffer associated with second task set;

completing processing records located in second buffer associated with first task by first task;

waiting, by first task, until second task completes the processing of records located in first buffer associated with second task set;

refilling, by first task, the first buffer associated with first task set and the first buffer associated with second task set;

determining whether the first task or the second task completes processing records in its first buffer next;

beginning to refill second buffer for both tasks by the task determined to next complete processing of its first buffer; and

processing, by task determined to next complete processing its first buffer, second buffer of task determined to next complete processing its first buffer.

13. The program storage device of claim 12, further comprising:

determining whether the first task or the second task completes processing records in its second buffer next;

beginning to refill first buffer for both tasks by the task determined to first complete processing of its second buffer; and

processing, by task determined to next complete processing its second buffer, first buffer of task determined to next complete processing its second buffer.

14. The program storage device of claim 13, wherein first task comprises processing data log records.

15. The program storage device of claim 13, wherein first task comprises processing index log records.

16. The program storage device of claim 13, wherein first buffer comprises an index log record portion.

17. The program storage device of claim 13, wherein second buffer comprises a data log record portion.

18. An apparatus for managing buffers during online reorganization, comprising:

a computer having a data store coupled thereto, wherein the data store stores data; and

one or more computer programs, performed by the computer, for beginning processing, by a first task, records located in first buffer associated with first task set and processing, by a second task, records located in first buffer associated with second task set, completing by first task the processing of records in first buffer associated with first task, beginning processing by first task records located in second buffer associated with first task set without waiting for second task to complete processing records of first buffer associated with second task set, completing processing records located in second buffer associated with first task by first task, waiting, by first task, until second task completes the processing of records located in first buffer associated with second task set and refilling, by first task, the first buffer associated with first task set and the first buffer associated with second task set, determining whether the first task or the second task completes processing records in its first buffer next, beginning to refill second buffer for both tasks by the task determined to next complete processing of its first buffer; and

processing, by task determined to next complete processing its first buffer, second buffer of task determined to next complete processing its first buffer.

19. An apparatus for managing buffers during online reorganization, comprising:

a data storage device for storing data in a database;

a computer, coupled to the data storage device, for executing instructions for reorganizing the database, the computer including;

first task means for beginning to process records located in first buffer associated with first task means; and

second task means for processing records located in first buffer associated with second task means;

wherein the first task means completes the processing of records in a first buffer associated with first task means, processes records located in second buffer associated with first task means without waiting for second task means to complete processing records of first buffer associated with second task means, completes processing records located in second buffer associated with first task means, wait until second task completes the processing of records located in first buffer associated with second task means, refills the first buffer associated with first task means and the first buffer associated with second task means, determines whether the first task or the second task completes processing records in its first buffer next, begins to refill second buffer for both tasks by the task determined to next complete processing of its first buffer; and

processes, by task determined to next complete processing its first buffer, second buffer of task determined to next complete processing its first buffer.

20. A method of adaptive switching between tasks during online reorganization of a database, comprising:

begin processing, by a first task, records located in first buffer associated with first task set and processing, by a second task, records located in first buffer associated with second task set;

completing by first task the processing of records in first buffer associated with first task;

begin processing by first task, records located in second buffer associated with first task set without waiting for second task to complete processing records of first buffer associated with second task set;

completing processing records located in second buffer associated with first task by first task;

waiting, by first task, until second task completes the processing of records located in first buffer associated with second task set; and

refilling, by first task, the first buffer associated with first task set and the first buffer associated with second task set,

determining whether the first task or the second task completes processing records in its first buffer next,

beginning to refill second buffer for both tasks by the task determined to next complete processing of its first buffer; and

processing, by task determined to next complete processing its first buffer, second buffer of task determined to next complete processing its first buffer.