Virtual logging system and method

• US 20030226058A1
• Filed: 05/31/2002
• Published: 12/04/2003
• Est. Priority Date: 05/31/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bradshaw, Dexter Paul, Miller, Thomas Jay
• Granted Patent: US 7,007,197 B2
• Time in Patent Office: Days
• US Class Current: 714/15
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A common transactional logging system (a “virtual logging system”) that presents to one or more log clients the appearance that each log client is interacting with a dedicated logging system. In reality, the virtual logging system is multiplexing multiple virtual log streams, including log records, for each log client into a single transactional log.

Claims

1. A transactional logging system comprising:

a common logging engine configured to receive a plurality of virtual log streams, each virtual log stream having log data associated with one of a plurality of computer application programs, the common logging engine being further configured to multiplex the virtual log streams into a single physical log stream.

2. The transactional logging system of claim 1, wherein at least one of the virtual log streams is received from a remote computer application program over a network.

3. The computer-readable medium of claim 1, further comprising a plurality of distributed logging modules, each distributed logging module being associated with one of the computer application programs and being configured to marshal the log data of the computer application program into at least one virtual log stream, the distributed logging module being further configured to send the virtual log stream to the common logging engine.

4. The computer-readable medium of claim 3, wherein at least one of the distributed logging modules is associated with a remote computer application program.

5. The computer-readable medium of claim 3, wherein each distributed logging module further being configured to provide an illusion to the associated computer application program that a dedicated log is maintained exclusively for the computer application program.

6. The computer-readable medium of claim 1, wherein the physical log stream is appended to a physical log stored in a storage unit.

7. The computer-readable medium of claim 1, wherein the common logging engine is implemented in a kernel mode of a computer.

8. The computer-readable medium of claim 1, wherein at least one of the application programs is implemented in a kernel mode of a computer.

9. The computer-readable medium of claim 1, wherein at least one of the application programs is implemented in a user mode of a computer.

10. A computer-readable medium encoded with computer-executable components for performing transactional logging, comprising:

a plurality of distributed logging modules, a first one of the distributed logging modules being associated with a first computer application program and a second of the distributed logging modules being associated with a second computer application program, each of the distributed logging modules being further configured to receive log data from the distributed logging module'"'"'s associated computer application program, each of the distributed logging modules being programmed to marshal the log data sent by the associated computer application program; and

a common logging engine in communication with each of the distributed logging modules and configured to receive log data from each of the distributed logging modules and to multiplex the received log data into a single physical log.

11. The computer-readable medium of claim 10, further comprising a plurality of buffers, each buffer being associated with at least one of the distributed logging modules and being configured to temporarily store the log data until the log data are sent to the common logging engine.

12. The computer-readable medium of claim 10, further comprising a plurality of distributed logging module data stores (DLM data stores), each of DLM data stores being associated with one of the distributed logging modules and containing information about the physical log for use by the marshaling object in sending the log data to the common logging engine.

13. The computer-readable medium of claim 12, wherein the information about the physical log comprises an indication of an amount of pre-reserved space in the physical log available to the distributed logging module.

14. The computer-readable medium of claim 12, wherein the information about the physical log comprises an indication of an amount of pre-reserved space that the distributed logging module prefers to have pre-reserved.

15. The computer-readable medium of claim 12, wherein the information about the physical log comprises an indication of an amount of space required to store log data buffered by the distributed logging module.

16. The computer-readable medium of claim 10, wherein the common logging engine is implemented in a kernel mode of a computer system.

17. The computer-readable medium of claim 10, further comprising a flush queue, the flush queue being associated with the common logging engine and being configured to temporarily defer the multiplexed log data until the multiplexed log data is appended to the physical log.

18. The computer-readable medium of claim 10, further comprising a common logging engine data store (CLE data store), the CLE data store being associated with the common logging engine and containing information about the physical log.

19. The computer-readable medium of claim 18, wherein information about the common log file comprises an indication of an amount of pre-reserved space in the physical log available to the plurality of distributed logging modules.

20. The computer-readable medium of claim 18, wherein information about the physical log comprises an indication of an amount of space in the physical log allocated to log records.

21. The computer-readable medium of claim 10, further comprising an owner page cache associated with the common logging engine and containing data for retrieving log data.

22. The computer-readable medium of claim 21, wherein the data for retrieving log data comprises owner pages identifying a particular computer application program that owns a log record within the physical log.

23. A computer-readable medium encoded with a data structure, comprising:

a first data field including a plurality of data blocks, each of the data blocks containing log data related to a computer application program in a plurality of computer application programs; and

a second data field associating each of the data blocks with a computer application program to which the log data in the data block relate.

24. The computer-readable medium of claim 23, further comprising a third data field, the third data field specifying a range of data blocks in the first data field associated with the computer program.

25. A computer-implemented method of performing transactional logging for a computer application program, comprising:

receiving log records from each of a plurality of computer application programs;

marshalling the log records into a plurality of buffers; and

sending the marshaled log records to a common logging engine in a plurality of virtual log streams, each of the virtual log streams being associated with a computer application program in the plurality of computer application programs.

26. The computer-implemented method of claim 25, further comprising storing the marshaled log records in a buffer before sending the virtual log streams.

27. The computer-implemented method of claim 25, further comprising:

receiving at least one instruction to retrieve a log record from at least one of the computer application programs;

if the log record is in a buffer, retrieving the log record in the buffer; and

transmitting the log record to the computer application program.

28. The computer-implemented method of claim 27, further comprising if the log record is in a flush queue, retrieving the log record in the flush queue.

29. The computer-implemented method of claim 28, further comprising if the log record is not in the buffer or the flush queue, sending an instruction for retrieving the log record to a common logging engine.

30. A computer-implemented method for maintaining a log for a plurality of computer application programs, comprising:

receiving a plurality of virtual log streams, each virtual log stream comprising log data associated with one of the computer application programs;

multiplexing the virtual log streams into a single physical log stream; and

appending the multiplexed physical log stream to a physical log.

31. The computer-implemented method of claim 30, wherein at least one of the virtual log streams is received from a remote computer application program over a network.

32. The computer-implemented method of claim 30, deferring the multiplexed physical log stream in a flush queue before appending the multiplexed physical log stream to the physical log.

33. The computer-implemented method of claim 30, wherein multiplexing the virtual log streams further comprises:

parceling the log data into a plurality of log blocks, each of the log blocks being associated with one of a plurality of regions in the physical log stream; and

assigning an owner array to each of the regions, wherein for each of the log blocks in the region, the owner array identifies one of the computer application programs that is associated with the log block.

34. The computer-implemented method of claim 30, further comprising assigning an owner referral to each of the regions, wherein the owner referral identifies a range of log blocks associated with each of the computer application programs.

35. The computer-implemented method of claim 30, further comprising:

receiving at least one instruction from at least one of a plurality of distributed logging components to retrieve a log record, the distributed logging components being associated with the computer application programs;

mapping a virtual log sequence number (LSN) associated with the log record to a physical LSN;

if the log record is in a memory cache, retrieving the log record from the memory cache; and

transmitting the retrieved log record to the distributed logging component.

36. The computer-implemented method of claim 35, further comprising if the log record is in a storage unit, retrieving the log record from the storage unit.

37. A computer-implemented method for communicating between computer-executable components, comprising:

sending, by a distributed logging module, logging instructions to a common logging engine;

sending, by the distributed logging module, a first piggy back data along with the logging instructions;

receiving, by a common logging engine, the logging instructions and piggy back data;

sending, by the common logging engine, a response and acknowledgment message to the distributed logging module in response to the logging instructions;

sending, by the common logging engine, a second piggy back data along with the response and acknowledgment message.

38. The computer-implemented method of claim 37, wherein the logging instructions include a virtual log stream.

39. The computer-implemented method of claim 38, wherein the response and acknowledgement message include a pointer to a memory location where the virtual log stream is stored in a physical log.

40. The computer-implemented method of claim 37, further comprising sending, by the common logging engine a next physical log sequence number to the distributed logging module in response to the logging instructions.

41. The computer-implemented method of claim 37, wherein the first piggy back data include accounting information associated with pre-reservation.

42. The computer-implemented method of claim 37, wherein the first piggy back data and the second piggy back data form a feedback loop for updating accounting information.