SELF-ATTENTIVE ATTRIBUTED NETWORK EMBEDDING

• US 20200134428A1
• Filed: 10/24/2019
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Cheng, Wei, Yu, Wenchao, Chen, Haifeng
• Granted Patent: US 11,544,530 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/21355 nonlinear criteria, e.g. embedding a manifold in a Euclidean space
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Methods and systems for determining a network embedding include training a network embedding model using training data that includes topology information for networks and attribute information relating to vertices of the networks. An embedded representation is generated using the trained network embedding model to represent an input network, with associated attribute information, in a network topology space. A machine learning task is performed using the embedded representation as input to a machine learning model.

Claims

1. A method for determining a network embedding, comprising:

training a network embedding model using a processor, based on training data that includes topology information for networks and attribute information relating to vertices of the networks;

generating an embedded representation using the trained network embedding model to represent an input network, with associated attribute information, in a network topology space; and

performing a machine learning task using the embedded representation as input to a machine learning model.

2. The method of claim 1, wherein training the network embedding model includes adversarial training of a first autoencoder configured to perform topology embedding and a second autoencoder configured to perform attribute embedding.

3. The method of claim 2, wherein the first autoencoder is a recurrent neural network autoencoder.

4. The method of claim 2, wherein the second autoencoder is a multilayer perceptron autoencoder.

5. The method of claim 2, wherein training the network embedding model comprises solving a joint optimization problem that includes a loss term for a topology reconstruction autoencoder, a loss term for an attribute reconstruction autoencoder, and a locality-preserving loss term.

6. The method of claim 5, wherein the joint optimization problem further includes a distance metric between a distribution in topology space and a distribution in attribute space.

7. The method of claim 6, wherein the distance metric is an earth mover distance.

8. The method of claim 1, wherein training the network embedding model includes generating random walk sequences on a network to encode in a network topology space and in a network attribute space.

9. The method of claim 8, further comprising training the machine learning model using a set of training data that includes network walk sequences that are embedded using the network embedding model.

10. The method of claim 1, wherein the machine learning task includes a task selected from the group consisting of network reconstruction, multi-label classification, and link prediction.

11. A system for determining a network embedding, comprising:

a model trainer configured to train a network embedding model using training data that includes topology information for networks and attribute information relating to vertices of the networks, wherein the network embedding model is configured to generate an embedded representation to represent an input network, with associated attribute information, in a network topology space; and

a machine learning model configured to perform a machine learning task using the embedded representation output by the network embedding model as input.

12. The system of claim 11, wherein the model trainer is further configured to perform adversarial training of a first autoencoder of the network embedding model, configured to perform topology embedding, and a second autoencoder of the network embedding model, configured to perform attribute embedding.

13. The system of claim 12, wherein the first autoencoder is a recurrent neural network autoencoder.

14. The system of claim 12, wherein the second autoencoder is a multilayer perceptron autoencoder.

15. The system of claim 12, wherein the model trainer is further configured to solve a joint optimization problem that includes a loss term for a topology reconstruction autoencoder, a loss term for an attribute reconstruction autoencoder, and a locality-preserving loss term.

16. The system of claim 15, wherein the joint optimization problem further includes a distance metric between a distribution in topology space and a distribution in attribute space.

17. The system of claim 16, wherein the distance metric is an earth mover distance.

18. The system of claim 11, wherein the model trainer is further configured to generate random walk sequences on a network to encode in a network topology space and in a network attribute space.

19. The system of claim 18, wherein the model trainer is further configured to train the machine learning model using a set of training data that includes network walk sequences that are embedded using the network embedding model.

20. The system of claim 11, wherein the machine learning task includes a task selected from the group consisting of network reconstruction, multi-label classification, and link prediction.