System failure prediction using long short-term memory neural networks

• US 10,289,509 B2
• Filed: 04/04/2017
• Issued: 05/14/2019
• Est. Priority Date: 04/06/2016
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Xu, Jianwu, Zhang, Ke, Zhang, Hui, Min, Renqiang, Jiang, Guofei
• Primary Examiner(s): Kudirka, Joseph R
• Application Number:
  US15/478,714
• Publication Number:
  US 20170293542A1
• Time in Patent Office: 770 Days
• Field of Search: 714 26
• CPC Class Codes: G06F 11/008 Reliability or availability analysis
• CPC Class Codes: G06F 11/2263 using neural networks
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent

Abstract

Methods for system failure prediction include clustering log files according to structural log patterns. Feature representations of the log files are determined based on the log clusters. A likelihood of a system failure is determined based on the feature representations using a neural network. An automatic system control action is performed if the likelihood of system failure exceeds a threshold.

Claims

1. A method for system failure prediction, comprising:

clustering a plurality of log files according to structural log patterns to form log clusters;

determining feature representations of the log files based on the log clusters;

training a long short-term memory recurrent neural network from training data and user labels on failure time periods;

determining a likelihood of a system failure based on the feature representations using a neural network by processing the feature representations using the long short-term memory recurrent neural network; and

performing an automatic system control action if the likelihood of the system failure exceeds a threshold.

2. The method of claim 1, wherein determining the feature representations comprises determining two, and only two, feature representations.

3. The method of claim 2, wherein the two feature representations consist of pattern distribution among clustered logs and term frequency-inverse document frequency.

4. The method of claim 1, wherein clustering the plurality of log files comprises performing unsupervised, hierarchical clustering.

5. The method of claim 1, wherein determining the likelihood of the system failure comprises determining a binomial distribution based on an output of the neural network and a binary target vector with two complementary classes.

6. The method of claim 1, wherein the plurality of log files comprise a plurality of different log formats.

7. The method of claim 6, wherein the plurality of different log formats comprise different delimiter formats.

8. A method for system failure prediction, comprising:

clustering a plurality of log files having heterogeneous log formats according to structural log patterns using unsupervised, hierarchical clustering to form log clusters;

determining feature representations of the log files based on the log clusters, wherein the feature representations consist of pattern distribution among clustered logs and term frequency-inverse document frequency;

training a long short-term memory recurrent neural network from training data and user labels on failure time periods;

determining a likelihood of a system failure based on the feature representations using the long short-term memory neural network, a binomial distribution based on an output of the long short-term memory neural network, and a binary target vector with two complementary classes by processing the feature representations using the long short-term memory recurrent neural network; and

performing an automatic system control action if the likelihood of the system failure exceeds a threshold.

9. A system for system failure prediction, comprising:

a long-short term memory recurrent neural network that is trained from training data and user labels on failure time periods;

a pattern learning module comprising a processor configured to cluster a plurality of log files according to structural log patterns to form log clusters;

a feature extraction module configured to determine feature representations of the log files based on the log clusters;

a failure prediction module configured to determine a likelihood of a system failure based on the feature representations by processing the feature representations using the long short-term memory recurrent neural network; and

a system control module configured to perform an automatic system control action if the likelihood of the system failure exceeds a threshold.

10. The system of claim 9, wherein the feature extraction module is further configured to determine two, and only two, feature representations.

11. The system of claim 10, wherein the two feature representations consist of pattern distribution among clustered logs and term frequency-inverse document frequency.

12. The system of claim 9, wherein the pattern learning module is further configured to perform unsupervised, hierarchical clustering.

13. The system of claim 9, wherein the failure prediction module is further configured to determine a binomial distribution based on an output of the neural network and a binary target vector with two complementary classes.

14. The system of claim 9, wherein the plurality of log files comprise a plurality of different log formats.

15. The system of claim 14, wherein the plurality of different log formats comprise different delimiter formats.