Methods and Apparatus for Monitoring Abnormalities in Data Stream

• US 20090292979A1
• Filed: 07/23/2009
• Published: 11/26/2009
• Est. Priority Date: 09/17/2004
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aggarwal, Charu C.
• Granted Patent: US 8,165,975 B2
• Time in Patent Office: Days
• US Class Current: 714/819
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/045 Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence

Abstract

A technique for monitoring a primary data stream comprising a plurality of secondary data streams for abnormalities is provided. A deviation value for each of two or more of the plurality of secondary data streams is determined. The two or more deviation values of the two or more secondary data streams are combined to form a combined deviation value. An abnormality signal is generated based at least in part on the combined deviation value.

Claims

1. A method for monitoring a primary data stream, comprising a plurality of secondary data streams, for abnormalities, comprising the steps of:

determining a deviation value for each of two or more of the plurality of secondary data streams;

combining the two or more deviation values of the two or more secondary data streams to form a combined deviation value; and

generating an abnormality signal based at least in part on the combined deviation value.

2. The method of claim 1, wherein the step of determining a deviation value further comprises the steps of:

computing a polynomial approximation;

computing a predicted value from the polynomial approximation; and

computing a deviation value using an actual value and the predicted value.

3. The method of claim 1, wherein, in the step of combining the two or more deviation values, the two or more secondary data streams are associated with one or more channels of the primary data stream.

4. The method of claim 3, wherein the step of combining the two or more deviation values further comprises the steps of:

determining one or more relevant channels from the one or more channels of the primary data stream;

computing a statistical weight for each of the one or more relevant channels; and

combining two or more deviation values of the two or more secondary data streams associated with the one or more relevant channels in accordance with one or more corresponding statistical weights.

5. The method of claim 1, wherein, in the step of combining the two or more deviation values, the abnormality signal comprises a value relating to a likelihood of an abnormality in the primary data stream.

6. The method of claim 1, wherein the steps of determining a deviation value and combining the two or more deviation values are repeated for each point in time at which the primary data stream is monitored.

7. The method of claim 1, wherein the step of determining a deviation value for each of two or more of the plurality of secondary data streams comprises the steps of:

determining which two or more of the plurality of secondary data streams correlate to primary abnormalities; and

determining a deviation value for each of the two or more secondary data streams determined to correlate to primary abnormalities.

8. The method of claim 1, wherein a ratio of the value of the signal at primary abnormalities to a ratio of the value of the signal at secondary abnormalities is maximized.

9. The method of claim 1, wherein each of the two or more secondary data streams comprises a set of two or more dimensions.

10. The method of claim 9, wherein the set of dimensions of at least a first one of the two or more secondary data streams differs from the set of dimensions of at least a second one of the two or more secondary data streams.

11. Apparatus for monitoring a primary data stream, comprising a plurality of secondary data streams, for abnormalities, comprising:

a memory; and

at least one processor coupled to the memory and operative to perform the operations of;

determining a deviation value for each of two or more of the plurality of secondary data streams;

combining the two or more deviation values of the two or more secondary data streams to form a combined deviation value; and

generating an abnormality signal based at least in part on the combined deviation value.

12. The apparatus of claim 11, wherein the operation of determining a deviation value further comprises the operations of:

computing a polynomial approximation;

computing a predicted value from the polynomial approximation; and

computing a deviation value using an actual value and the predicted value.

13. The apparatus of claim 11, wherein, in the operation of combining the two or more deviation values, the two or more secondary data streams are associated with one or more channels of the primary data stream.

14. The apparatus of claim 13, wherein the operation of combining the two or more deviation values further comprises the operations of:

determining one or more relevant channels from the one or more channels of the primary data stream;

computing a statistical weight for each of the one or more relevant channels; and

combining two or more deviation values of the two or more secondary data streams associated with the one or more relevant channels in accordance with one or more corresponding statistical weights.

15. The apparatus of claim 11, wherein, in the operation of combining the two or more deviation values, the abnormality signal comprises a value relating to a likelihood of an abnormality in the primary data stream.

16. The apparatus of claim 11, wherein the operations of determining a deviation value and combining the two or more deviation values are repeated for each point in time at which the primary data stream is monitored.

17. The apparatus of claim 11, wherein the operation of determining a deviation value for each of two or more of the plurality of secondary data streams comprises the operations of:

determining which two or more of the plurality of secondary data streams correlate to primary abnormalities; and

determining a deviation value for each of the two or more secondary data streams determined to correlate to primary abnormalities.

18. The apparatus of claim 11, wherein a ratio of the value of the signal at primary abnormalities to a ratio of the value of the signal at secondary abnormalities is maximized.

19. The apparatus of claim 11, wherein each of the two or more secondary data streams comprises a set of two or more dimensions.

20. The apparatus of claim 19, wherein the set of dimensions of at least a first one of the two or more secondary data streams differs from the set of dimensions of at least a second one of the two or more secondary data streams.

21. An article of manufacture for monitoring a primary data stream, comprising a plurality of secondary data streams, for abnormalities, comprising a machine readable storage medium containing one or more programs which when executed implement the steps of:

determining a deviation value for each of two or more of the plurality of secondary data streams;

combining the two or more deviation values of the two or more secondary data streams to form a combined deviation value; and

generating an abnormality signal based at least in part on the combined deviation value.