Methods and apparatus for monitoring abnormalities in data stream

• US 8,165,975 B2
• Filed: 07/23/2009
• Issued: 04/24/2012
• Est. Priority Date: 09/17/2004
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aggarwal, Charu C.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): Wong, Lut
• Application Number:
  US12/507,876
• Publication Number:
  US 20090292979A1
• Time in Patent Office: 1,006 Days
• Field of Search: 706/12
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/045 Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence

Abstract

A technique for monitoring a primary data stream comprising a plurality of secondary data streams for abnormalities is provided. A deviation value for each of two or more of the plurality of secondary data streams is determined. The two or more deviation values of the two or more secondary data streams are combined to form a combined deviation value. An abnormality signal is generated based at least in part on the combined deviation value.

Claims

1. A computer-implemented method for monitoring a primary data stream, comprising a plurality of secondary data streams, for abnormalities, comprising the steps of:

determining a deviation value for each of two or more of the plurality of secondary data streams;

combining the two or more deviation values of the two or more secondary data streams according to statistical weights associated with the secondary data streams to foam a combined deviation value; and

generating an abnormality signal based at least in part on the combined deviation value such that a ratio of a value of the abnormality signal at times of target abnormalities to a value of the abnormality signal at times of non-target abnormalities is maximized.

2. The method of claim 1, wherein the step of determining a deviation value further comprises the steps of:

computing a polynomial approximation;

computing a predicted value from the polynomial approximation; and

computing a deviation value using an actual value and the predicted value.

3. The method of claim 1, wherein, in the step of combining the two or more deviation values, the two or more secondary data streams are associated with one or more channels of the primary data stream.

4. The method of claim 3, wherein the step of combining the two or more deviation values further comprises the steps of:

determining one or more relevant channels from the one or more channels of the primary data stream;

computing a statistical weight for each of the one or more relevant channels; and

combining two or more deviation values of the two or more secondary data streams associated with the one or more relevant channels in accordance with one or more corresponding statistical weights.

5. The method of claim 1, wherein, in the step of combining the two or more deviation values, the abnormality signal comprises a value relating to a likelihood of an abnormality in the primary data stream.

6. The method of claim 1, wherein the steps of determining a deviation value and combining the two or more deviation values are repeated for each point in time at which the primary data stream is monitored.

7. The method of claim 1, wherein the step of determining a deviation value for each of two or more of the plurality of secondary data streams comprises the steps of:

determining which two or more of the plurality of secondary data streams correlate to target abnormalities; and

determining a deviation value for each of the two or more secondary data streams determined to correlate to target abnormalities.

8. The method of claim 1, wherein each of the two or more secondary data streams comprises a set of two or more dimensions.

9. The method of claim 8, wherein the set of dimensions of at least a first one of the two or more secondary data streams differs from the set of dimensions of at least a second one of the two or more secondary data streams.

10. Apparatus for monitoring a primary data stream, comprising a plurality of secondary data streams, for abnormalities, comprising:

a memory; and

at least one processor coupled to the memory and operative to perform the operations of;

determining a deviation value for each of two or more of the plurality of secondary data streams;

combining the two or more deviation values of the two or more secondary data streams according to statistical weights associated with the secondary data streams to form a combined deviation value; and

generating an abnormality signal based at least in part on the combined deviation value such that a ratio of a value of the abnormality signal at times of target abnormalities to a value of the abnormality signal at times of non-target abnormalities is maximized.

11. The apparatus of claim 10, wherein the operation of determining a deviation value further comprises the operations of:

computing a polynomial approximation;

computing a predicted value from the polynomial approximation; and

computing a deviation value using an actual value and the predicted value.

12. The apparatus of claim 10, wherein, in the operation of combining the two or more deviation values, the two or more secondary data streams are associated with one or more channels of the primary data stream.

13. The apparatus of claim 12, wherein the operation of combining the two or more deviation values further comprises the operations of:

determining one or more relevant channels from the one or more channels of the primary data stream;

computing a statistical weight for each of the one or more relevant channels; and

combining two or more deviation values of the two or more secondary data streams associated with the one or more relevant channels in accordance with one or more corresponding statistical weights.

14. The apparatus of claim 10, wherein, in the operation of combining the two or more deviation values, the abnormality signal comprises a value relating to a likelihood of an abnormality in the primary data stream.

15. The apparatus of claim 10, wherein the operations of determining a deviation value and combining the two or more deviation values are repeated for each point in time at which the primary data stream is monitored.

16. The apparatus of claim 10, wherein the operation of determining a deviation value for each of two or more of the plurality of secondary data streams comprises the operations of:

determining which two or more of the plurality of secondary data streams correlate to target abnormalities; and

determining a deviation value for each of the two or more secondary data streams determined to correlate to target abnormalities.

17. The apparatus of claim 10, wherein each of the two or more secondary data streams comprises a set of two or more dimensions.

18. The apparatus of claim 14, wherein the set of dimensions of at least a first one of the two or more secondary data streams differs from the set of dimensions of at least a second one of the two or more secondary data streams.

19. An article of manufacture for monitoring a primary data stream, comprising a plurality of secondary data streams, for abnormalities, comprising a machine readable tangible storage medium containing one or more programs which when executed implement the steps of:

determining a deviation value for each of two or more of the plurality of secondary data streams;

combining the two or more deviation values of the two or more secondary data streams according to statistical weights associated with the secondary data streams to form a combined deviation value; and

generating an abnormality signal based at least in part on the combined deviation value such that a ratio of a value of the abnormality signal at times of target abnormalities to a value of the abnormality signal at times of non-target abnormalities is maximized.