APPARATUS AND METHOD FOR PROCESSING DATA STREAMS

• US 20110258414A1
• Filed: 12/08/2009
• Published: 10/20/2011
• Est. Priority Date: 12/12/2008
• Status: Active Grant

• Current Assignee: BAE Systems Plc
• Original Assignee: BAE Systems Plc
• Inventors: Gray, Michael, Nussbaum, Ian, Grover, Ian
• Granted Patent: US 8,930,754 B2
• Time in Patent Office: Days
• US Class Current: 712/31
• CPC Class Codes: G06F 11/1443 Transmit or communication errors
• CPC Class Codes: G06F 11/2028 eliminating a faulty processor or activating a spare
• CPC Class Codes: G06F 11/203 using migration
• CPC Class Codes: G06F 11/2038 with a single idle spare processing component
• CPC Class Codes: G06F 11/2048 where the redundant components share neither address space nor persistent storage
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups

Abstract

A distributed architecture and method for maintaining the integrity of data streams within a multi-pipelined processing environment. The architecture comprising a communications network for carrying a plurality of data streams and a master processor adapted to process one or more messages in at least one of the data streams, the message processing including the creation of one or more data packets within the stream, each packet encapsulating at least a transaction summary of the data that has been processed. The architecture further comprising at least one slave processor per master processor adapted to emulate the transactional state of the master processor by regenerating the data stream as a result of processing the one or more data packets, whereupon in response to an error event on the master processor, the slave processor acts to avoid interrupting the data stream by generating one or more successive data packet(s). Hence, the architecture and method serve as a high availability, robust fault tolerant system, mitigating against the loss of data within data streams.

Claims

1. A distributed architecture configured to maintain the integrity of data streams within a multi-pipelined processing environment, the architecture comprising:

a communications network for carrying a plurality of data streams;

a master processor adapted to process one or more messages in at least one of the data streams, the message processing including the creation of one or more data packets within the stream, each packet encapsulating at least a transaction summary of the data that has been processed; and

at least one slave processor adapted to emulate the transactional state of the master processor by regenerating the data stream as a result of processing the one or more data packets, whereupon in response to an error event on the master processor, the slave processor acts to avoid interrupting the data stream by generating one or more successive data packet(s).

2. The distributed architecture as in claim 1, wherein the slave processor is configured to regenerate the data stream by restoring the one or more messages.

3. The distributed architecture as in claim 2, wherein the slave processor is configured to restore the one or more messages in the same order in which they were originally created and/or processed.

4. The distributed architecture as in claim 1, wherein the regeneration of the data stream preserves the dynamic state of the one or more messages.

5. The distributed architecture as in claim 1, wherein the slave processor is configured to asynchronously process the one or more data packets.

6. The distributed architecture as in claim 1, wherein the slave processor comprises a buffer for receiving and storing the one or more data packets.

7. The distributed architecture as in claim 1, wherein the slave processor is further configured to make any successive data packets available on the communications network as part of the data stream.

8. The distributed architecture as in claim 1, wherein the master processor is configured to automatically shut-down in response to the error event.

9. The distributed architecture as in claim 1, wherein the slave processor is configured to assume control over the master processor in response to detecting the error event.

10. The distributed architecture as in claim 9, wherein the slave processor is arranged to assume control over the master processor within substantially several tens of milliseconds of the error event being detected.

11. The distributed architecture as in claim 1, wherein the slave processor is configured to be logically inactive until the error event is detected.

12. The distributed architecture as in claim 1, wherein the message and/or packet handling is implemented within the middleware of the multi-pipelined processing environment.

13. The distributed architecture as in claim 1, wherein the communications network has an inter-meshed topology.

14. The distributed architecture as in claim 1, wherein the data stream includes information from one or more data sources.

15. The distributed architecture as in claim 1, wherein the architecture further comprises a plurality of processors, with a portion acting as master processors and another portion acting as slave processors, with at least one slave processor associated with each master processor.

16. A method of maintaining the integrity of data streams within a multi-pipelined processing environment, implemented across a distributed architecture, the method comprising the steps of:

carrying a plurality of data streams on a communications network;

processing, on a master processor, one or more messages in at least one of the data streams, the processing including the creation of one or more data packets within the stream, each packet encapsulating at least a transaction summary of the data that has been processed; and

emulating, on at least one slave processor, the transactional state of the master processor by regenerating the data stream as a result of processing the one or more data packets, whereupon in response to an error event on the master processor, the slave processor acts to avoid interrupting the data stream by generating one or more successive data packet(s).

17. The method as in claim 16, wherein regenerating the data stream comprises the step of restoring the one or more messages.

18. The method as in claim 17, wherein the step of restoring restores the one or more messages in the same order in which they were originally created and/or processed.

19. The method as in claim 16, wherein regenerating the data stream preserves the dynamic state of the one or more messages.

20. The method as in claim 16, wherein the one or more data packets are processed asynchronously.

21. The method as in claim 16, further comprising the step of receiving and storing the one or more data packets in a buffer associated with the slave processor.

22. The method as in claim 16, further comprising the step of making any successive data packets available on the communications network as part of the data stream.

23. The method as in claim 16, further comprising the step of transferring control from the master processor to the slave processor in response to detecting the error event.

24. The method as in claim 16, further comprising the step of receiving information from one or more data sources as one or more data streams.