Training encoder model and/or using trained encoder model to determine responsive action(s) for natural language input

• US 10,783,456 B2
• Filed: 12/14/2018
• Issued: 09/22/2020
• Est. Priority Date: 12/15/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Strope, Brian, Sung, Yun-hsuan, Yuan, Wangqing
• Primary Examiner(s): Misir, Dave
• Application Number:
  US16/611,725
• Publication Number:
  US 20200104746A1
• Time in Patent Office: 648 Days
• Field of Search: 706 12
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/3346 using probabilistic model
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Systems, methods, and computer readable media related to: training an encoder model that can be utilized to determine semantic similarity of a natural language textual string to each of one or more additional natural language textual strings (directly and/or indirectly); and/or using a trained encoder model to determine one or more responsive actions to perform in response to a natural language query. The encoder model is a machine learning model, such as a neural network model. In some implementations of training the encoder model, the encoder model is trained as part of a larger network architecture trained based on one or more tasks that are distinct from a “semantic textual similarity” task for which the encoder model can be used.

Claims

1. A method implemented by one or more processors, comprising:

identifying a plurality of positive training instances that each include an input and a response, wherein for each of the positive training instances;

the input is based on content of a corresponding electronic communication, and

the response is based on a corresponding responsive electronic communication that is responsive to the corresponding electronic communication;

training an encoder model based on the positive training instances, wherein training the encoder model based on a given instance of the positive training instances comprises;

generating an input encoding based on processing the input using the encoder model;

generating a response encoding based on processing the response using the encoder model;

generating a final response encoding based on processing the response encoding using a reasoning model;

determining a value based on comparison of the input encoding and the final response encoding; and

updating both the reasoning model and the encoder model based on comparison of the value to a given value indicated by the given instance; and

after training the encoder model;

using the trained encoder model, independent of the reasoning model, to determine a similarity value of two textual segments, wherein the similarity value indicates semantic similarity of the two textual segments, and wherein using the trained encoder model to determine the similarity value of the two textual segments comprises;

receiving a query directed to an automated assistant;

generating a query encoding based on processing the query using the trained encoder model;

comparing the query encoding to a plurality of pre-determined query encodings each stored in association with one or more corresponding actions;

determining, based on the comparing, a given predetermined query encoding to which the query encoding is most similar; and

in response to the query and based on the given predetermined query encoding being most similar to the query encoding, causing the automated assistant to perform the one or more corresponding actions that are stored in association with the given predetermined query encoding.

2. The method of claim 1, further comprising:

training the encoder model based on a plurality of distinct additional training instances, wherein the plurality of distinct additional training instances are for a task that is distinct from the task of the plurality of positive training instances.

3. The method of claim 2, wherein training the encoder model based on a given distinct instance of the distinct additional training instances comprises:

generating a first encoding based on processing a first input of the given distinct instance using the encoder model;

generating a second encoding based on processing a second input of the given distinct instance using the encoder model;

generating a prediction based on processing of the first encoding and the second encoding using an additional model, wherein the additional model is not utilized in training the encoder model based on the positive training instances; and

updating both the additional model and the encoder model based on comparison of the prediction to a labeled output of the given distinct instance.

4. The method of claim 3, wherein the labeled output indicates a particular category, of a plurality of potential categories, for a natural language inference task.

5. The method of claim 3, wherein training the encoder model based on the plurality of distinct additional training instances occurs simultaneously with training the encoder model based on the positive training instances.

6. The method of claim 5, wherein training the encoder model based on the plurality of distinct additional training instances is by one or more first worker threads and wherein training the encoder model based on the positive training instances is by one or more second worker threads.

7. The method of claim 1, further comprising

determining that a distance, between the query encoding and the given predetermined query encoding, satisfies a closeness threshold;

wherein causing the automated assistant to perform the one or more corresponding actions is further in response to determining that the distance satisfies the closeness threshold.

8. The method of claim 1, wherein comparing the query encoding to the plurality of pre-determined query encodings comprises:

generating a plurality of scalar values, each based on a corresponding dot product of the query encoding and a corresponding one of the plurality of pre-determined query encodings; and

wherein determining, based on the comparing, the given predetermined query encoding to which the query encoding is most similar comprises;

selecting the given predetermined query encoding based on a scalar value, that is based on the dot product of the query encoding and the given predetermined query encoding, being the minimal of the generated plurality of scalar values.

9. The method of claim 1, wherein the query is not explicitly mapped, by the automated assistant, to the one or more corresponding actions.

10. The method of claim 1, wherein the query is based on user input received at a first computing device, and wherein the one or more corresponding actions comprise controlling one or more additional devices.

11. The method of claim 1, wherein the query is received as a voice input, wherein the method further comprises performing a voice-to-text conversion process on the voice input to generate text, and wherein generating the query encoding based on processing the query using the trained encoder model comprises processing the text using the trained encoder model.

12. A method implemented by one or more processors, comprising:

identifying a plurality of positive training instances that each include an input and a response, wherein for each of the positive training instances;

the input is based on content of a corresponding electronic communication, and

the response is based on a corresponding responsive electronic communication that is responsive to the corresponding electronic communication;

training an encoder model based on the positive training instances, wherein training the encoder model based on a given instance of the positive training instances comprises;

generating an input encoding based on processing the input using the encoder model;

generating a response encoding based on processing the response using the encoder model;

generating a final response encoding based on processing the response encoding using a reasoning model;

determining a value based on comparison of the input encoding and the final response encoding; and

updating both the reasoning model and the encoder model based on comparison of the value to a given value indicated by the given instance; and

training the encoder model based on a plurality of distinct additional training instances, wherein the plurality of distinct additional training instances are for a task that is distinct from the task of the plurality of positive training instances, and wherein training the encoder model based on a given distinct instance of the distinct additional training instances comprises;

generating a first encoding based on processing a first input of the given distinct instance using the encoder model;

generating a second encoding based on processing a second input of the given distinct instance using the encoder model;

generating a prediction based on processing of the first encoding and the second encoding using an additional model, wherein the additional model is not utilized in training the encoder model based on the positive training instances; and

updating both the additional model and the encoder model based on comparison of the prediction to a labeled output of the given distinct instance;

after training the encoder model;

using the trained encoder model, independent of the reasoning model, to determine a similarity value of two textual segments, wherein the similarity value indicates semantic similarity of the two textual segments.

13. The method of claim 12, wherein the labeled output indicates a particular category, of a plurality of potential categories, for a natural language inference task.

14. The method of claim 12, wherein training the encoder model based on the plurality of distinct additional training instances occurs simultaneously with training the encoder model based on the positive training instances.

15. The method of claim 14, wherein training the encoder model based on the plurality of distinct additional training instances is by one or more first worker threads and wherein training the encoder model based on the positive training instances is by one or more second worker threads.