Multi-datacenter message queue

• US 10,476,982 B2
• Filed: 05/13/2016
• Issued: 11/12/2019
• Est. Priority Date: 05/15/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Inventors: Tarre, Marc Solanas, Rantzau, Ralf, Dutta, Debojyoti, Sharma, Manoj
• Primary Examiner(s): Joo, Joshua
• Application Number:
  US15/154,141
• Publication Number:
  US 20160337465A1
• Time in Patent Office: 1,278 Days
• Field of Search: None
• CPC Class Codes: G06F 9/00 Arrangements for program control, e.g. control units program control for peripheral devices G06F13/10
• CPC Class Codes: H04L 49/9005 using dynamic buffer space allocation
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561

Abstract

Approaches are disclosed for distributing messages across multiple data centers where the data centers do not store messages using a same message queue protocol. In some embodiment, a network element translates messages from a message queue protocol (e.g., Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ) to an application layer messaging protocol (e.g., XMPP, MQTT, WebSocket protocol, or other application layer messaging protocols). In other embodiments, a network element translates messages from an application layer messaging protocol to a message queue protocol. Using the new approaches disclosed herein, data centers communicate using, at least in part, application layer messaging protocols to disconnect the message queue protocols used by the data centers and enable sharing messages between messages queues in the data centers. Consequently, the data centers can share messages regardless of whether the underlying message queue protocols used by the data centers (and the network devices therein) are compatible with one another.

Claims

1. A method comprising:

receiving from a first message queue, a message encoded in a first message queue protocol associated with the first message queue;

generating a translated message by translating the message from the first message queue protocol to an application layer messaging protocol;

transmitting, utilizing the application layer messaging protocol, the translated message to a server, the translated message updating a second message queue, the second message queue being encoded in a second message queue protocol; and

determining whether a connection to the server is active,

wherein,

the message queue protocol is a first message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ, and

the second message queue protocol is a different message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ.

2. The method of claim 1, further comprising:

dynamically adjusting a size of a buffer for the first message queue based on whether the connection to the server is active.

3. The method of claim 1, further comprising:

determining the connection to the server is active; and

increasing a size of a buffer for the first message queue.

4. The method of claim 1, further comprising:

receiving, from the server, an instruction to stop transmitting messages associated with the second message queue.

5. The method of claim 1, further comprising:

determining the connection to the server is not active;

stopping transmission of messages to the server, the messages associated with the second message queue; and

storing the messages in a buffer until the connection to the server is active.

6. The method of claim 1, wherein the translated message by translating the message from the first message queue protocol to the application layer messaging protocol comprises:

translating the message from an APACHE Kafka message encoding to an Extensible Messaging and Presence Protocol (XMPP) message encoding.

7. The method of claim 1, further comprising:

determining the connection to the server is not active; and

decreasing a size of a buffer for the first message queue.

8. An apparatus comprising:

a network interface, the network interface being configured to receive, from a first message queue, a message encoded in a first message queue protocol associated with the first message queue;

a memory element to store code; and

at least one processor coupled to the memory element and network interface, wherein the at least one processor is to execute the code to perform operations comprising;

generating a translated message by translating the message from the first message queue protocol to an application layer messaging protocol;

transmitting, utilizing the application layer messaging protocol and the network interface, the translated message to a server, wherein the translated message is to update a second message queue encoded in a second message queue protocol; and

determining whether a connection to the server is active, wherein,

the message queue protocol is a first message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ, and

the second message queue protocol is a different message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ.

9. The apparatus of claim 8, further comprising:

a buffer configured to store messages, and wherein the operations further comprise;

dynamically adjusting a size of a buffer for the first message queue based on whether the connection to the server is active.

10. The apparatus of claim 8, wherein the operations further comprise:

determining the connection to the server is active; and

increasing a size of a buffer for the first message queue.

11. The apparatus of claim 8, wherein the operations further comprise:

receiving, from the server, an instruction to stop transmitting messages associated with the second message queue.

12. The apparatus of claim 8, wherein the operations further comprise:

determining the connection to the server is not active;

stopping transmission of messages to the server, the messages associated with the second message queue; and

storing the messages in a buffer until the connection to the server is active.

13. The apparatus of claim 8, wherein the generating the translated message by translating the message from the first message queue protocol to the application layer messaging protocol comprises:

translating the message from an APACHE Kafka message encoding to an Extensible Messaging and Presence Protocol (XMPP) message encoding.

14. The apparatus of claim 8, wherein the operations further comprise:

determining the connection to the server is not active; and

decreasing a size of a buffer for the first message queue.

15. A computer-readable non-transitory medium comprising instructions, that when executed by at least one processor configure the at least one processor to perform operations comprising:

receiving from a first message queue, a message encoded in a first message queue protocol associated with the first message queue;

generating a translated message by translating the message from the first message queue protocol to an application layer messaging protocol;

transmitting, utilizing the application layer messaging protocol, the translated message to a server, wherein the translated message update a second message queue, the second message queue being encoded in a second message queue protocol; and

determining whether a connection to the server is active,

wherein,

the message queue protocol is a first message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ, and

the second message queue protocol is a different message queue protocol selected from the group consisting of Kestrel, RABBITMQ, APACHE Kafka, and ACTIVEMQ.

16. The computer-readable non-transitory medium of claim 15, wherein the instructions, when executed by the at least one processor further configure the at least one processor to perform further operations comprising:

dynamically adjusting a size of a buffer for the first message queue based on whether the connection to the server is active.

17. The computer-readable non-transitory medium of claim 16, wherein the dynamically adjusting the size of the buffer for the first message queue based on whether the connection to the server is active comprises:

determining the connection to the server is active; and

increasing the size of the buffer for the first message queue.

18. The computer-readable non-transitory medium of claim 15, wherein the instructions, when executed by the at least one processor further configure the at least one processor to perform further operations comprising:

receiving, from the server, an instruction to stop transmitting messages associated with the second message queue.

19. The computer-readable non-transitory medium of claim 16, wherein the instructions, when executed by the at least one processor further configure the at least one processor to perform additional operations comprising:

determining the connection to the server is not active;

stopping transmission of messages to the server, the messages associated with the second message queue; and

storing the messages in a buffer until the connection to the server is active.

20. The computer-readable non-transitory medium of claim 15, wherein the translated message by translating the message from the first message queue protocol to the application layer messaging protocol comprises:

translating the message from an APACHE Kafka message encoding to an Extensible Messaging and Presence Protocol (XMPP) message encoding.