Method and apparatus for detecting a multi-stage event

• US 9,870,470 B2
• Filed: 03/31/2014
• Issued: 01/16/2018
• Est. Priority Date: 03/29/2013
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Herwono, Ian, Cui, Zhan, Azvine, Ben, Brown, Martin, Smith, Karl
• Primary Examiner(s): Lewis, Lisa
• Assistant Examiner(s): Bucknor, Olanrewaju
• Application Number:
  US14/781,185
• Publication Number:
  US 20160055335A1
• Time in Patent Office: 1,387 Days
• Field of Search: None
• CPC Class Codes: G06F 21/554 involving event detection and direct action
• CPC Class Codes: G06F 2221/034 Test or assess a computer or a system
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: H04L 63/1408 by monitoring network traffic monitoring network traffic per se H04L43/00
• CPC Class Codes: H04L 63/1416 Event detection, e.g. attack signature detection
• CPC Class Codes: H04L 63/1458 Denial of Service

Abstract

A multi-stage event detector for monitoring a system to detect the occurrence of multistage events in the monitored system, the multi-stage event detector includes: one or more event detecting detector units (142, 144) for detecting observable events occurring on the monitored system; one or more parameter generating detector units (152, 154) for generating parameter values which vary over time dependent on the behavior of the monitored system; a hidden state determiner (120) for determining a likely sequence of states of interest of the system based on the outputs of the one or more event detecting detector units; and a transition determiner (130) for determining a likely transition occurrence based on a comparison of a set of values of a parameter or set of parameters generated by one or more of the one or more parameter generating detector units with a plurality of pre-specified functions or sets of values of a corresponding parameter or set of parameters associated with different transition occurrences.

Claims

1. A multi-stage event detector for monitoring a system to detect the occurrence of multi-stage events in the monitored system, the multi-stage event detector including:

a data store storing a plurality of models of multi-stage events, each model specifying a sequence of stages and a set of transitions from one stage to another; and

processing resources including at least one processor and a memory comprising instructions that, when executed, control the multi-stage event detector to at least;

(a) detect observable events occurring on the monitored system;

(b) generate parameter values that vary over time dependent on behavior of the monitored system;

(c) determine a likely sequence of hidden states of interest of the monitored system based on the detected observable events occurring on the monitored system, wherein the hidden states of interest of the monitored system correspond to stages in the stored models of the multi-stage events;

(d) determine a likely transition occurrence, corresponding to a transition from

one stage to another in the stored models of the multi-stage events, based on a comparison of a set of values of a parameter or set of parameters generated in (b) with a plurality of pre-specified functions or sets of values of a corresponding parameter or set of parameters associated with different transitions from one stage to another in the stored models of the multi-stage events; and

(e) determine an estimated identity of a multi-stage event and an estimated current stage of the multi-stage event in dependence upon a comparison of both the determination in (c) of a likely sequence of hidden states of interest of the monitored system and the determination in (d) of a likely transition occurrence with the stored models of multi-stage events specifying both sequences of stages and transitions between stages,

wherein (c) is performed in connection with a model of the plurality of the models of multi-stage events, the model further specifying a plurality of distinct hidden states that the monitored system can occupy, and a plurality of visible states that can be determined in (a).

2. The multi-stage event detector according to claim 1, wherein (c) is performed in connection with a set of probabilities including at least a plurality of transition probabilities expressing the probability of the monitored system transitioning from one hidden state to another and a plurality of production probabilities expressing the probability of a particular visible state being determined in (a) for a particular given hidden state occupancy of the monitored system.

3. The multi-stage event detector according to claim 2, wherein the model of the system being monitored is a Hidden Markov Model.

4. The multi-stage event detector according to claim 2, wherein values for the probabilities contained within the model of the system being monitored are generated using an automated training procedure.

5. A method of detecting the occurrence of a multi-stage event within a system being monitored, the method comprising:

having stored a plurality of models of multi-stage events, each model specifying a sequence of stages and a set of transitions from one stage to another;

(a) detecting, using processing resources including at least one processor and a memory, observable events occurring on the monitored system;

(b) generating, using the processing resources, parameter values that vary over time dependent on behavior of the monitored system;

(c) determining, using the processing resources, a likely sequence of hidden states of interest of the monitored system based on the detected observable events, wherein the hidden states of interest of the monitored system correspond to stages in the stored models of the multistage events;

(d) determining, using the processing resources, at least one likely transition occurrence, corresponding to a transition from one stage to another in the stored models of the multi-stage events, based on a comparison of the generated parameter values with a plurality of pre-specified functions or sets of values of a corresponding parameter or set of parameters associated with different transitions from one stage to another in the stored models of the multi-stage events; and

(e) determining, using the processing resources, an estimated identity of a multi-stage event and an estimated current stage of the multi-stage event in dependence upon a comparison of both the determination of a likely sequence of hidden states of interest of the monitored system and the determination of a likely transition occurrence or occurrences with the stored models of multi-stage events specifying both sequences of stages and transitions between stages,

wherein (c) is performed in connection with a model of the plurality of the models of multi-stage events, the model further specifying a plurality of distinct hidden states that the monitored system can occupy, and a plurality of visible states that can be determined in (a).

6. The method according to claim 5, wherein the determining of the likely sequence of hidden states of interest of the monitored system is performed in connection with a set of probabilities including at least a plurality of transition probabilities expressing the probability of the monitored system transitioning from one hidden state to another and a plurality of production probabilities expressing the probability of a particular visible state being determined in (a) for a particular given hidden state occupancy of the monitored system.

7. The method according to claim 6, wherein the model of the system being monitored is a Hidden Markov Model.

8. The method according to claim 6, wherein values for the probabilities contained within the model of the system being monitored are generated using an automated training procedure.

9. A non-transitory computer readable storage medium including instructions for detecting the occurrence of a multi-stage event within a system being monitored, the instructions being executable to perform functionality comprising:

having stored a plurality of models of multi-stage events, each model specifying a sequence of stages and a set of transitions from one stage to another;

(a) detecting observable events occurring on the monitored system;

(b) generating parameter values that vary over time dependent on behavior of the monitored system;

(c) determining a likely sequence of hidden states of interest of the monitored system based on the detected observable events, wherein the hidden states of interest of the monitored system correspond to stages in the stored models of the multi-stage events;

(d) determining at least one likely transition occurrence, corresponding to a transition from one stage to another in the stored models of the multi-stage events, based on a comparison of the generated parameter values with a plurality of pre-specified functions or sets of values of a corresponding parameter or set of parameters associated with different transitions from one stage to another in the stored models of the multi-stage events; and

(e) determining an estimated identity of a multi-stage event and an estimated current stage of the multi-stage event in dependence upon a comparison of both the determination of a likely sequence of hidden states of interest of the monitored system and the determination of a likely transition occurrence or occurrences with the stored models of multi-stage events specifying both sequences of stages and transitions between stages,

wherein (c) is performed in connection with a model of the plurality of the models of multi-stage events, the model further specifying a plurality of distinct hidden states that the monitored system can occupy, and a plurality of visible states that can be determined in (a).

10. The non-transitory computer readable storage medium according to claim 9, wherein the determining of the likely sequence of hidden states of interest of the monitored system is performed in connection with a set of probabilities including at least a plurality of transition probabilities expressing the probability of the monitored system transitioning from one hidden state to another and a plurality of production probabilities expressing the probability of a particular visible state being determined in (a) for a particular given hidden state occupancy of the monitored system.

11. The method according to claim 10, wherein the model of the system being monitored is a Hidden Markov Model.

12. The method according to claim 10, wherein values for the probabilities contained within the model of the system being monitored are generated using an automated training procedure.