RECOVERING A FAILURE IN A DATA PROCESSING SYSTEM

• US 20140304545A1
• Filed: 04/05/2013
• Published: 10/09/2014
• Est. Priority Date: 04/05/2013
• Status: Abandoned Application

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chen, Qiming, Hsu, Meichun
• Application Number:
  US13/857,885
• Publication Number:
  US 20140304545A1
• Time in Patent Office: Days
• US Class Current: 714/4.3
• CPC Class Codes: G06F 11/1438 Restarting or rejuvenating
• CPC Class Codes: G06F 9/46 Multiprogramming arrangements
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues

Abstract

A technique of recovering a failure in a data processing system comprises restoring a checkpointed state in a last window, and resending all the input messages received at the second node during the failed window boundary.

Claims

1. A method of recovering a failure in a data processing system comprising:

at a source node, recording input/output channels and segment numbers for all tuples received in a window;

processing each input tuple to derive a number of output tuples, each output tuple comprising the recorded input/output channels and segment numbers; and

if a failure occurs at a target node;

restore a last window state of the target node; and

request a number of tuples from a current window of the target node up to a current tuple to be resent from a source node based on the input/output channels and segment numbers recorded at the source node.

2. The method of claim 1, further comprising checkpointing the states and a number of output messages once per window.

3. The method of claim 2, in which checkpointing the states and a number of output messages once per window comprises checkpointing the states and a number of output messages once per window after processing a last input tuple within the window.

4. The method of claim 1, in which checkpointing the execution state and the output message for each output task is performed asynchronously with respect to the derivation of the output tuples.

5. The method of claim 1, in which recording input/output channels and segment numbers is performed before emitting the output tuples.

6. The method of claim 1, further comprising:

at the target node, recording input/output channels and segment numbers for all tuples received in a window; and

processing each input tuple to derive a number of output tuples, each output tuple comprising the recorded input/output channels and segment numbers,

in which the checkpoint interval for a target task is an integer of the checkpoint interval of the source task.

7. The method of claim 6, in which the method is implemented on top of an existing distributed stream processing infrastructure.

8. The method of claim 6, in which the input tasks and output tasks are communicated through messaging.

9. The method of claim 1, in which the method is performed while continuously processing a stream per-tuple,

10. A system for processing data, comprising:

a processor; and

a memory communicatively coupled to the processor, in which the processor, executing computer usable program code;

checkpoints a number of states and a number of output messages once per window;

emits the output tasks to a second node; and

if one of the output tasks fails at the second node;

restores the checkpointed state in a last window; and

resends all the input messages received at the second node during the failed window boundary based on input/output channels and segment numbers recorded at the first node.

11. The system of claim 10, in which the window is defined by a number of messages sent.

12. The system of claim 11, in which the window defined by the number of messages sent is user-definable.

13. The system of claim 10, in which the system is provided as a service over a network.

14. A computer program product for recovering a failure in a data processing system, the computer program product comprising:

a computer readable storage medium comprising computer usable program code embodied therewith, the computer usable program code comprising;

computer usable program code to, when executed by a processor, receive a number of input tasks at a first node of a data processing system, the input tasks comprising a number of input tuples;

computer usable program code to, when executed by the processor, for each of the number of input tuples, derive a number of output tuples for a number of output tasks;

computer usable program code to, when executed by the processor, generate a number of states for a number of the output tasks;

computer usable program code to, when executed by the processor, checkpoint the states and a number of output messages once per window;

computer usable program code to, when executed by the processor, emit the output tasks to a second node; and

if one of the output tasks fails;

computer usable program code to, when executed by the processor, restore a checkpointed state in a last window boundary; and

computer usable program code to, when executed by the processor, resend all the input messages received at the second node during the failed window boundary based on input/output channels and segment numbers recorded at the source node and appended to the emitted output tasks.

15. The computer program product of claim 14, further comprising computer usable program code to, when executed by the processor, store data associated with window boundaries to synchronize the checkpoints of the tasks.

16. The computer program product of claim 14, in which the window is a time window.