Commit-time ordered message queue supporting arbitrary read and dequeue patterns from multiple subscribers

• US 20070083569A1
• Filed: 10/07/2005
• Published: 04/12/2007
• Est. Priority Date: 10/07/2005
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Wong, Lik, Tran, Hung, Stamos, James
• Granted Patent: US 7,680,793 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database

Abstract

Data consistency in the context of information sharing requires maintenance of dependencies among information being shared. Transactional dependency ordering is implemented in a database system message queue, by associating a unique system commit time with each transactional message group. Read consistency is implemented in such a queue by allowing only messages with a fully determined order to be visible. A fully determined order is implemented through use of a high watermark, which guarantees that future transactions, for which messages are entering the queue, have commit times that are greater than the current high watermark. Therefore, only messages below the current high watermark are visible and can be dequeued, with no chance of other new messages enqueuing below the current high watermark.

Claims

1. A method comprising performing a machine-executed operation involving instructions, wherein the machine-executed operation is at least one of:

A) sending said instructions over transmission media;

B) receiving said instructions over transmission media;

C) storing said instructions onto a machine-readable storage medium; and

D) executing the instructions;

wherein said instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of;

in response to a request, from a client of a database system, to enqueue one or more first messages, the database system adding said one or more first messages to a message queue, said one or more first messages associated with a first database transaction being executed for said client,

committing the first database transaction;

wherein committing the first database transaction includes determining a unique system commit time corresponding to the first database transaction, wherein system commit times corresponding to respective database transactions are based on transactional dependencies among database transactions;

associating the unique system commit time with the one or more first messages; and

ordering messages in the message queue based on the unique system commit time.

2. The method of claim 1, wherein the instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

in response to a request to dequeue one or more messages from the message queue,

determining whether any message of the one or more messages is associated with a system commit time that is less than a first high watermark that is associated with the message queue; and

if there are no messages, of the one or more messages, that are associated with a system commit time that is less than the first high watermark, then generating a second high watermark for association with the message queue, wherein the second high watermark is greater than the first high watermark.

3. The method of claim 2, wherein the instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

after generating the second high watermark, receiving a second request to commit a database transaction with which one or more second messages are associated;

in response to the second request, determining a unique system commit time corresponding to the database transaction with which the one or more second messages are associated;

wherein the unique system commit time corresponding to the database transaction with which the one or more second messages are associated must be greater than the second high watermark.

4. The method of claim 2, wherein the instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

if, at the time of the request to dequeue one or more messages from the message queue, a database transaction holds a shared write lock on the message queue, then

the step of generating the second high watermark includes awaiting release of the shared write lock before generating the second high watermark.

5. The method of claim 2, wherein the database system message queue is a first message queue and the unique system commit time is a first unique system commit time, and wherein said instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

in response to a request, from a client of a database system, to enqueue one or more second messages, the database system adding said one or more second messages to a second message queue that is a different queue than the first message queue, said one or more second messages associated with a second database transaction being executed for said client that requested enqueue of the one or more messages,

committing the second database transaction;

wherein committing the second database transaction includes determining a second unique system commit time corresponding to the second database transaction;

associating the second unique system commit time with the one or more second messages;

ordering messages in the second message queue based on the second unique system commit time; and

in response to a request to dequeue one or more messages from the second message queue, determining whether any message of the one or more messages from the second message queue is associated with a system commit time that is less than a high watermark that is associated with the second message queue but not the first message queue.

6. The method of claim 5, wherein the request to enqueue the one or more first messages is from the same client as the request to enqueue the one or more second messages.

7. The method of claim 5, wherein the request to enqueue the one or more first messages is from a different client than the request to enqueue the one or more second messages.

8. The method of claim 1, wherein the instructions for associating the unique system commit time with the one or more first messages include instructions for performing the steps of:

storing, in a database table separate from the message queue, records that include an identifier of the database transaction in association with the unique system commit time.

9. The method of claim 8, wherein the database table is an index-only table.

10. The method of claim 8, wherein the step of ordering messages in the message queue comprises ordering the records in the database table that is separate from the message queue.

11. The method of claim 1, wherein the instructions for ordering messages in the message queue include instructions for performing the steps of:

ordering messages in the message queue based on their respective commit times as a primary ordering element and priority as a secondary ordering element.

12. The method of claim 1, wherein said instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

receiving a request to dequeue from the message queue a sub-set of the one or more first messages;

determining, based on the unique system commit time corresponding to the database transaction and stored in association with the message queue, that the database transaction has committed; and

in response to determining that the database transaction has committed, dequeuing from the message queue only the sub-set of the one or more first messages.

13. The method of claim 1, wherein the first database transaction is a database transaction on which a second database transaction depends, and wherein said instructions are instructions which, when executed by one or more processors, cause the one or more processors to perform the steps of:

before receiving the one or more first messages, receiving at the message queue one or more second messages associated with the second database transaction;

before the first transaction is committed,

receiving a request to commit the second database transaction, and

receiving a request to dequeue from the message queue a particular second message;

determining that the particular second message is not associated with a unique system commit time; and

in response to determining that the particular second message is not associated with a unique system commit time, refusing to dequeue the particular second message.