Hierarchical machine learning system for lifelong learning

• US 10,162,794 B1
• Filed: 03/07/2018
• Issued: 12/25/2018
• Est. Priority Date: 03/07/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: Apprente, Inc. (International Business Machines Corporation)
• Inventors: Arel, Itamar, Looks, Joshua Benjamin
• Primary Examiner(s): Chen, Alan
• Application Number:
  US15/914,955
• Time in Patent Office: 293 Days
• Field of Search: None
• CPC Class Codes: G06F 18/21 Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
• CPC Class Codes: G06F 18/2136 based on sparsity criteria, e.g. with an overcomplete basis
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

Embodiments described herein cover a hierarchical machine learning system with a separated perception subsystem (that includes a hierarchy of nodes having at least a first layer and a second layer) and application subsystem. In one example embodiment a first node in the first layer processes a first input and processes at least a portion of the first input to generate a first feature vector. A second node in the second layer processes a second input comprising at least a portion of the first feature vector to generate a second feature vector. The first node generates a first sparse feature vector from the first feature vector and/or the second node generates a second sparse feature vector from the second feature vector. A third node of the perception subsystem then processes at least one of the first sparse feature vector or the second sparse feature vector to determine an output.

Claims

1. A computer-implemented method for a machine learning system that mitigates catastrophic forgetting, comprising:

receiving a first input at a first node in a first layer of a hierarchy of nodes, wherein the first input comprises a) a first previous feature vector that was generated by the first node based on a first previous input and b) a data item;

processing at least a portion of the first input by the first node to generate a first feature vector, the first feature vector comprising a first plurality of feature elements;

processing a second input by a second node in a second layer of the hierarchy of nodes to generate a second feature vector, wherein the second input comprises a) at least a portion of the first feature vector and b) a second previous feature vector that was generated by the second node based on a second previous input, and wherein the second feature vector comprises a second plurality of feature elements;

generating at least one of a first sparse feature vector from the first feature vector or a second sparse feature vector from the second feature vector, wherein a majority of feature elements in the first sparse feature vector and the second sparse feature vector have a value of zero; and

processing at least one of the first sparse feature vector or the second sparse feature vector by a third node to determine a first output.

2. The computer-implemented method of claim 1, wherein the first input further comprises at least a portion of the second previous feature vector.

3. The computer-implemented method of claim 1, wherein the data item comprises a target, the method further comprising:

determining an error associated with the first output based at least in part on a function of the first output and the target;

determining a relevancy rating based at least in part on the error, wherein the relevancy rating applies to the first node and the second node;

determining whether the relevancy rating satisfies a relevancy criterion;

determining a first novelty rating for the first node and a second novelty rating for the second node;

determining whether the first novelty rating satisfies a novelty criterion; and

determining whether the second novelty rating satisfies the novelty criterion.

4. The computer-implemented method of claim 3, wherein the novelty criterion comprises a novelty threshold, the relevancy rating comprises a relevancy threshold, and the second node comprises a plurality of centroids, the method further comprising:

determining that the second novelty rating is below the novelty threshold and that the relevancy rating is below the relevancy threshold; and

updating a first centroid of the plurality of centroids in the second node, wherein the first centroid is associated with a first feature element of the second feature vector having a highest value.

5. The computer-implemented method of claim 4, further comprising:

updating a second centroid of the plurality of centroids in the second node, wherein the second centroid is associated with a second feature element of the second previous feature vector having a highest value.

6. The computer-implemented method of claim 3, wherein the novelty criterion comprises a novelty threshold, the relevancy rating comprises a relevancy threshold, and the second node comprises a plurality of centroids, the method further comprising:

determining that the novelty rating is above the novelty threshold and that the relevancy rating is above the relevancy threshold; and

allocating a first new centroid for the second node, the first new centroid having values based on the second input.

7. The computer-implemented method of claim 6, further comprising:

allocating a second new centroid for the second node, the second new centroid having values based at least in part on the second previous input.

8. The computer-implemented method of claim 1, further comprising:

processing a third input comprising at least a portion of the second feature vector by a third node in a third layer of the hierarchy of nodes to generate a third feature vector, the third feature vector comprising a third plurality of feature elements; and

generating a third sparse feature vector from the third feature vector;

wherein processing at least one of the first sparse feature vector or the second sparse feature vector by the third node comprises processing the second sparse feature vector and the third sparse feature vector by the third node to determine the first output.

9. The computer-implemented method of claim 1, further comprising:

determining whether update criteria are satisfied for the first node;

separately determining whether the update criteria are satisfied for the second node; and

updating at least one of the first node or the second node.

10. The computer-implemented method of claim 9, further comprising updating the second node without updating the first node.

11. The computer-implemented method of claim 1, wherein the first node comprises a first plurality of centroids and wherein the second node comprises a second plurality of centroids, the method further comprising:

determining a relevancy rating associated with the first output;

determining a first novelty rating by the first node based on feature element values of the first plurality of feature elements in the first feature vector, wherein each feature element value represents a similarity of the input to a centroid of the first plurality of centroids;

determining a second novelty rating by the second node based on feature element values of the second plurality of feature elements in the second feature vector;

determining whether to update the first node based on a comparison of the first novelty rating to a novelty criterion and a comparison of the relevancy rating to a relevancy criterion; and

determining whether to update the second node based on a comparison of the second novelty rating to the novelty criterion and a comparison of the relevancy rating to the relevancy criterion.

16. The computer-implemented method of claim 1, wherein the first node and the second nodes are components of a hierarchical perception subsystem of a machine learning architecture and wherein the third node is a component of an application subsystem of the machine learning architecture, the method further comprising:

co-training the perception subsystem and the application subsystem based on labeled data items and unlabeled data items, wherein a first function is used to train nodes in the perception subsystem and a second function is used to train nodes in the application subsystem.

17. The computer-implemented method of claim 1, further comprising:

receiving a third input at a fourth node in the first layer of the hierarchy of nodes;

processing at least a portion of the third input by the fourth node to generate a third feature vector, wherein the second input processed by the second node further comprises at least a portion of the third feature vector; and

generating at least a third sparse feature vector from the third feature vector; and

wherein the third node processes the first sparse feature vector, the second sparse feature vector and the third sparse feature vector to determine the first output.

18. The computer-implemented method of claim 17, wherein the first node and the fourth node share a same plurality of centroids, the method further comprising:

determining that update criteria are satisfied for allocation of a new centroid for the first node;

allocating a new centroid for the first node; and

allocating the new centroid for the fourth node.

12. A computer-implemented method for a machine learning system that mitigates catastrophic forgetting, comprising:

receiving a first input at a first node in a first layer of a hierarchy of nodes;

processing at least a portion of the first input by the first node to generate a first feature vector, the first feature vector comprising a first plurality of feature elements;

processing a second input comprising at least a portion of the first feature vector by a second node in a second layer of the hierarchy of nodes to generate a second feature vector, the second feature vector comprising a second plurality of feature elements;

generating at least one of a first sparse feature vector from the first feature vector or a second sparse feature vector from the second feature vector, wherein a majority of feature elements in the first sparse feature vector and the second sparse feature vector have a value of zero;

processing at least one of the first sparse feature vector or the second sparse feature vector by a third node to determine a first output;

receiving a new input at the first node;

processing at least a portion of the new input by the first node to generate a first new feature vector; and

determining, by the second node, whether a second new input comprising at least a portion of the first new feature vector satisfies a processing criterion.

13. The computer-implemented method of claim 12, further comprising:

determining that the second new input satisfies the processing criterion;

processing the second new input by the second node to generate a second new feature vector;

generating a first new sparse feature vector from the first new feature vector;

generating a second new sparse feature vector from the second new feature vector; and

processing the first new sparse feature vector and the second new sparse feature vector by the second node to determine a second output.

14. The computer-implemented method of claim 12, further comprising:

determining a first feature element of the first new feature vector having a highest feature element value;

determining a first centroid of a plurality of centroids in the first node that is associated with the first feature element; and

resetting an update multiplier associated with the first centroid.

15. The computer-implemented method of claim 12, further comprising:

prior to receiving the new input, performing the following comprising;

determining that the first output fails to satisfy update criteria;

determining a first feature element of the first new feature vector having a highest feature element value;

determining a first centroid of a plurality of centroids in the first node that is associated with the first feature element; and

retaining the first input for the first centroid; and

after receiving the new input, applying a decay factor to an update multiplier associated with the first centroid.

19. A system comprising:

a memory to store instructions for a machine learning system that mitigates catastrophic forgetting; and

a processing device, operatively coupled to the memory, to execute the instructions, wherein the instructions cause the processing device to instantiate the machine learning system, the machine learning system comprising;

a hierarchy of nodes, the hierarchy of nodes comprising;

a first layer of nodes comprising a first plurality of nodes, wherein a first node of the first plurality of nodes is to;

process a first input to produce a first feature vector; and

generate a first sparse feature vector from the first feature vector; and

a second layer of nodes comprising a second plurality of nodes, wherein a second node of the second plurality of nodes is to;

process a second input comprising the first feature vector to produce a second feature vector; and

generate a second sparse feature vector from the second feature vector; and

an additional node to;

receive a first plurality of sparse feature vectors from the first plurality of nodes, the first plurality of sparse feature vectors comprising the first sparse feature vector;

receive a second plurality of sparse feature vectors from the second plurality of nodes, the second plurality of sparse feature vectors comprising the second sparse feature vector; and

determine an output based on the first plurality of sparse feature vectors and the second plurality of sparse feature vectors.

20. The system of claim 19, wherein the processing device comprises a plurality of processors, wherein a first processor of the plurality of processors is to execute one or more nodes of the first plurality of nodes and a second processor of the plurality of processors is to execute one or more additional nodes of the second plurality of nodes.

21. The system of claim 19, wherein the first layer of nodes is pipelined with the second layer of nodes.

22. The system of claim 19, wherein the first plurality of nodes comprise a first plurality of recurrent nodes and the second plurality of nodes comprise a second plurality of recurrent nodes, wherein a recurrent node of at least one of the first plurality of recurrent nodes or the second plurality of recurrent nodes is to generate a current feature vector based at least in part on a combination of an external input and a past feature vector that was previously generated by the recurrent node.

23. The system of claim 19, wherein the first plurality of nodes comprise a first plurality of recurrent nodes and the second plurality of nodes comprise a second plurality of recurrent nodes, wherein a recurrent node of the first plurality of recurrent nodes is to generate a current feature vector based at least in part on a combination of an external input, a past feature vector that was previously generated by the recurrent node and a second past feature vector that was previously generated by a second recurrent node of the second plurality of recurrent nodes.

24. The system of claim 19, wherein the hierarchy of nodes further comprises:

a third layer comprising a third plurality of nodes, wherein a third node of the third plurality of nodes is to;

process a third input to produce a third feature vector; and

provide the third feature vector to the first node, wherein the first input processed by the first node comprises at least the third feature vector;

wherein neither the third feature vector or a sparse feature vector based on the third feature vector are provided to the additional node.

25. The system of claim 19, wherein the hierarchy of nodes are components of a hierarchical perception subsystem of a machine learning architecture and wherein the additional node is a component of an application subsystem of the machine learning architecture, wherein the processing device is to:

co-train the perception subsystem and the application subsystem based on labeled data items and unlabeled data items, wherein a first function is used to train nodes in the perception subsystem and a second function is used to train nodes in the application subsystem.

26. A non-transitory computer readable medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations for a machine learning system that mitigates catastrophic forgetting, the operations comprising:

receiving a first input at a first node in a first layer of a hierarchy of nodes;

processing at least a portion of the first input by the first node to generate a first feature vector, the first feature vector comprising a first plurality of feature elements;

processing a second input comprising at least a portion of the first feature vector by a second node in a second layer of the hierarchy of nodes to generate a second feature vector, the second feature vector comprising a second plurality of feature elements;

generating at least one of a first sparse feature vector from the first feature vector or a second sparse feature vector from the second feature vector, wherein a majority of feature elements in the first sparse feature vector and the second sparse feature vector have a value of zero; and

processing at least one of the first sparse feature vector or the second sparse feature vector by a third node to determine a first output;

determining whether update criteria are satisfied for the first node;

separately determining whether the update criteria are satisfied for the second node; and

updating at least one of the first node or the second node.

27. The non-transitory computer readable medium of claim 26, the operations further comprising:

processing a third input comprising at least a portion of the second feature vector by a third node in a third layer of the hierarchy of nodes to generate a third feature vector, the third feature vector comprising a third plurality of feature elements; and

generating a third sparse feature vector from the third feature vector;

wherein processing at least one of the first sparse feature vector or the second sparse feature vector by the third node comprises processing the second sparse feature vector and the third sparse feature vector by the third node to determine the first output.