Apparatus and method for processing data streams

• US 8,930,754 B2
• Filed: 12/08/2009
• Issued: 01/06/2015
• Est. Priority Date: 12/12/2008
• Status: Active Grant

• Current Assignee: BAE Systems Plc
• Original Assignee: BAE Systems Plc
• Inventors: Nussbaum, Ian, Grover, Ian, Gray, Michael
• Primary Examiner(s): GUYTON, PHILIP A
• Application Number:
  US13/139,443
• Publication Number:
  US 20110258414A1
• Time in Patent Office: 1,855 Days
• Field of Search: 714/4.11, 714/12, 714/13
• US Class Current: 714/13
• CPC Class Codes: G06F 11/1443 Transmit or communication errors
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

A distributed architecture and method for maintaining the integrity of data streams within a multi-pipelined processing environment. The architecture comprising a communications network for carrying a plurality of data streams and a master processor adapted to process one or more messages in at least one of the data streams, the message processing including the creation of one or more data packets within the stream, each packet encapsulating at least a transaction summary of the data that has been processed. The architecture further comprising at least one slave processor per master processor adapted to emulate the transactional state of the master processor by regenerating the data stream as a result of processing the one or more data packets, whereupon in response to an error event on the master processor, the slave processor acts to avoid interrupting the data stream by generating one or more successive data packet(s). Hence, the architecture and method serve as a high availability, robust fault tolerant system, mitigating against the loss of data within data streams.

Claims

1. A distributed architecture configured to maintain the integrity of data streams within a multi-pipelined processing environment, the architecture comprising:

a communications network for carrying a plurality of data streams;

a master processor adapted to process one or more messages in at least one of the data streams, the message processing including the creation of one or more data packets within the at least one data stream, each packet encapsulating a transaction unit for the message processing, the transaction unit including the message that triggered the message processing, a description of changes or updates to data made as a result of the message processing and a message configured to trigger subsequent processing; and

at least one slave processor adapted to emulate the transactional state of the master processor by regenerating the at least one data stream as a result of processing the one or more data packets by storing the description of changes or updates to the data made as a result of the message processing and storing the message configured to trigger subsequent processing in a processing queue, whereupon in response to an error event on the master processor, the slave processor acts to avoid interrupting the at least one data stream by generating one or more successive data packet(s) using only data previously received from the master processor.

2. The distributed architecture as in claim 1, wherein the slave processor is configured to regenerate the at least one data stream by restoring the one or more messages.

3. The distributed architecture as in claim 2, wherein the slave processor is configured to restore the one or more messages in the same order in which they were originally created and/or processed.

4. The distributed architecture as in claim 1, wherein the regeneration of the at least one data stream preserves the dynamic state of the one or more messages.

5. The distributed architecture as in claim 1, wherein the slave processor is configured to asynchronously process the one or more data packets.

6. The distributed architecture as in claim 1, wherein the slave processor is further configured to make any successive data packets available on the communications network as part of the at least one data stream.

7. The distributed architecture as in claim 1, wherein the master processor is configured to automatically shut-down in response to the error event.

8. The distributed architecture as in claim 1, wherein the slave processor is configured to assume control over the master processor in response to detecting the error event.

9. The distributed architecture as in claim 8, wherein the slave processor is arranged to assume control over the master processor within substantially several tens of milliseconds of the error event being detected.

10. The distributed architecture as in claim 1, wherein the slave processor is configured to be logically inactive until the error event is detected.

11. The distributed architecture as in claim 1, wherein the message and/or packet handling is implemented within the middleware of the multi-pipelined processing environment.

12. The distributed architecture as in claim 1, wherein the communications network has an inter-meshed topology.

13. The distributed architecture as in claim 1, wherein the at least one data stream includes information from one or more data sources.

14. The distributed architecture as in claim 1, wherein the architecture further comprises a plurality of processors, with a portion acting as master processors and another portion acting as slave processors, with at least one slave processor associated with each master processor.

15. The distributed architecture as in claim 1, wherein the slave processor is configured to delete a stored message configured to trigger subsequent processing from the processing queue after a subsequent transaction unit is received from the master processor.

16. A method of maintaining the integrity of data streams within a multi-pipelined processing environment, implemented across a distributed architecture, the method comprising the steps of:

carrying a plurality of data streams on a communications network;

processing, on a master processor, one or more messages in at least one of the data streams, the processing including the creation of one or more data packets within the at least one of the data streams, each packet encapsulating a transaction unit for the message processing, the transaction unit including the message that triggered the message processing, a description of changes or updates to data made as a result of the message processing and a message configured to trigger subsequent processing; and

emulating, on at least one slave processor, the transactional state of the master processor by regenerating the at least one of the data streams as a result of processing the one or more data packets storing the description of changes or updates to the data made as a result of the message processing and storing the message configured to trigger subsequent processing in a processing queue, whereupon in response to an error event on the master processor, the slave processor acts to avoid interrupting the at least one of the data streams by generating one or more successive data packet(s) using only data previously received from the master processor.

17. The method as in claim 16, wherein regenerating the at least one of the data streams comprises the step of restoring the one or more messages.

18. The method as in claim 17, wherein the step of restoring restores the one or more messages in the same order in which they were originally created and/or processed.

19. The method as in claim 16, wherein regenerating the at least one of the data streams preserves the dynamic state of the one or more messages.

20. The method as in claim 16, wherein the one or more data packets are processed asynchronously.

21. The method as in claim 16, further comprising the step of making any successive data packets available on the communications network as part of the at least one of the data streams.

22. The method as in claim 16, further comprising the step of transferring control from the master processor to the slave processor in response to detecting the error event.

23. The method as in claim 16, further comprising the step of receiving information from one or more data sources as one or more data streams.