Buffered message queue architecture for database management systems with memory optimizations and "zero copy" buffered message queue

• US 20040034664A1
• Filed: 05/21/2003
• Published: 02/19/2004
• Est. Priority Date: 08/01/2002
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Meiyyappan, Krishnan, Mishra, Shailendra, Surlaker, Kapil, Jain, Namit, Bhatt, Neerja
• Granted Patent: US 7,203,706 B2
• Time in Patent Office: Days
• US Class Current: 707/104.1
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 707/99948 Application of database or data structure, e.g. distributed, multimedia, or image

Abstract

A buffered message queue architecture for managing messages in a database management system is disclosed. A “buffered message queue” refers to a message queue implemented in a volatile memory, such as a RAM. The volatile memory may be a shared volatile memory that is accessible by a plurality of processes. The buffered message queue architecture supports a publish and subscribe communication mechanism, where the message producers and message consumers may be decoupled from and independent of each other. The buffered message queue architecture provides all the functionality of a persistent publish-subscriber messaging system, without ever having to store the messages in persistent storage. The buffered message queue architecture provides better performance and scalability since no persistent operations are needed and no UNDO/REDO logs need to be maintained. Messages published to the buffered message queue are delivered to all eligible subscribers at least once, even in the event of failures, as long as the application is “repeatable.” The buffered message queue architecture also includes management mechanisms for performing buffered message queue cleanup and also for providing unlimited size buffered message queues when limited amounts of shared memory are available. The architecture also includes “zero copy” buffered message queues and provides for transaction-based enqueue of messages.

Claims

1. A method for managing messages in a database system, the method comprising the computer-implemented steps of:

a first process that is executing in the database system, creating, in a first portion of a volatile memory in the database system, a message in a native format of the first process, wherein the first portion of the volatile memory is associated with the first process;

the first process causing a reference to the message to be enqueued into a message queue that is disposed in the volatile memory separate from the first portion of the volatile memory; and

the first process causing the message to be enqueued directly from the first portion of the shared memory into the message queue.

2. The method as recited in claim 1, wherein the step of causing the message to be enqueued directly from the first portion of the shared memory into a message queue includes copying the message directly from the first portion of the volatile memory into the message queue.

3. The method as recited in claim 1, further comprising the computer-implemented steps of:

a second process that is executing in the database system, requesting that a message be dequeued from the message queue; and

in response to the second process requesting that a message be dequeued from the message queue, dequeuing the reference to the message to the second process.

4. The method as recited in claim 3, wherein the step of dequeuing the reference to the message to the second process includes providing to the second process, an address where the message is stored in the message queue.

5. The method as recited in claim 1, further comprising the computer-implemented step of deleting the message from the first portion of the volatile memory.

6. The method as recited in claim 1, wherein the step of causing the message to be enqueued directly from the first portion of the shared memory into the message queue is performed without formatting data into a format required by a database in the database system.

7. A computer-readable medium for managing messages in a database system, the computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

a first process that is executing in the database system, creating, in a first portion of a volatile memory in the database system, a message in a native format of the first process, wherein the first portion of the volatile memory is associated with the first process;

the first process causing a reference to the message to be enqueued into a message queue that is disposed in the volatile memory separate from the first portion of the volatile memory; and

the first process causing the message to be enqueued directly from the first portion of the shared memory into the message queue.

8. The computer-readable medium as recited in claim 7, wherein the step of causing the message to be enqueued directly from the first portion of the shared memory into a message queue includes copying the message directly from the first portion of the volatile memory into the message queue.

9. The computer-readable medium as recited in claim 7, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

a second process that is executing in the database system, requesting that a message be dequeued from the message queue; and

in response to the second process requesting that a message be dequeued from the message queue, dequeuing the reference to the message to the second process.

10. The computer-readable medium as recited in claim 9, wherein the step of dequeuing the reference to the message to the second process includes providing to the second process, an address where the message is stored in the message queue.

11. The computer-readable medium as recited in claim 7, further comprising one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of deleting the message from the first portion of the volatile memory.

12. The computer-readable medium as recited in claim 7, wherein the step of causing the message to be enqueued directly from the first portion of the shared memory into the message queue is performed without formatting data into a format required by a database in the database system.

13. An apparatus for managing messages in a database system, the apparatus comprising a memory carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

a first process that is executing in the database system, creating, in a first portion of a volatile memory in the database system, a message in a native format of the first process, wherein the first portion of the volatile memory is associated with the first process;

the first process causing a reference to the message to be enqueued into a message queue that is disposed in the volatile memory separate from the first portion of the volatile memory; and

the first process causing the message to be enqueued directly from the first portion of the shared memory into the message queue.

14. The apparatus as recited in claim 13, wherein the step of causing the message to be enqueued directly from the first portion of the shared memory into a message queue includes copying the message directly from the first portion of the volatile memory into the message queue.

15. The apparatus as recited in claim 13, wherein the memory further comprises one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the steps of:

a second process that is executing in the database system, requesting that a message be dequeued from the message queue; and

in response to the second process requesting that a message be dequeued from the message queue, dequeuing the reference to the message to the second process.

16. The apparatus as recited in claim 15, wherein the step of dequeuing the reference to the message to the second process includes providing to the second process, an address where the message is stored in the message queue.

17. The apparatus as recited in claim 13, wherein the memory further comprises one or more additional instructions which, when executed by the one or more processors, cause the one or more processors to perform the step of deleting the message from the first portion of the volatile memory.

18. The apparatus as recited in claim 13, wherein the step of causing the message to be enqueued directly from the first portion of the shared memory into the message queue is performed without formatting data into a format required by a database in the database system.