MULTI-MODELS THAT UNDERSTAND NATURAL LANGUAGE PHRASES

• US 20190332668A1
• Filed: 04/22/2019
• Published: 10/31/2019
• Est. Priority Date: 04/25/2018
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Wang, Yu, Shen, Yilin, Jin, Hongxia
• Granted Patent: US 10,902,211 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G06F 40/284 Lexical analysis, e.g. tokenisation or collocates
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 19/00 Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis in musical instruments G10H

Abstract

A system determines intent values based on an object in a received phrase, and detail values based on the object in the received phrase. The system determines intent state values based on the intent values and the detail values, and detail state values and an intent detail value based on the intent values and the detail values. The system determines other intent values based on the intent values and another object in the received phrase, and other detail values based on the detail values and the other object in the received phrase. The system determines a general intent value based on the other intent values, the other detail values, and the intent state values, and another intent detail value based on the other intent values, the other detail values, and the detail state values.

Claims

1. A system comprising:

one or more processors; and

a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to;

determine, by an intent encoder, intent values based on an object in a received phrase;

determine, by a detail encoder, detail values based on the object in the received phrase;

determine, by an intent decoder, intent state values based on the intent values and the detail values;

determine, by a detail decoder, detail state values and an intent detail value based on the intent values and the detail values;

determine, by the intent encoder, other intent values based on the intent values and another object in the received phrase;

determine, by the detail encoder, other detail values based on the detail values and the other object in the received phrase;

determine, by the intent decoder, a general intent value based on the other intent values, the other detail values, and the intent state values; and

determine, by the detail decoder, another intent detail value based on the other intent values, the other detail values, and the detail state values.

2. The system of claim 1, wherein the plurality of instructions, when executed, will further cause the one or more processors to:

at least one of modify parameters in at least one of the intent encoder and the intent decoder based on a difference between the determined general intent value and a verified general intent; and

modify parameters in at least one of the detail encoder and the detail decoder based on a difference between at least one of the determined intent detail value, the determined other intent detail value and at least one verified intent detail.

3. The system of claim 1, wherein the plurality of instructions, when executed, will further cause the one or more processors to:

output a response to the received phrase, the response being based on at least one of the determined general intent value, the determined intent detail value, and the determined other intent detail value.

4. The system of claim 1, wherein the plurality of instructions, when executed, will further cause the one or more processors to:

determine, by a supplemental encoder, supplemental values based on a received supplemental object;

determine, by a supplemental decoder, supplemental state values and a supplemental value based on the intent values, the detail values, and the supplemental values;

determine, by the supplemental encoder, other supplemental values based on the intent values, the detail values, and the received supplemental object; and

determine, by the supplemental decoder, another supplemental value based on the other intent values, the other detail values, and the supplemental state values.

5. The system of claim 4, wherein the supplemental encoder comprises an image encoder, the supplemental values comprise image values, the received supplemental object comprises an image object, the supplemental decoder comprises an image decoder, the supplemental state values comprise image state values, the supplemental value comprises an image value the other supplemental values comprise other image values, and the other supplemental value comprises another image value.

6. The system of claim 4, wherein one of the supplemental encoder comprises an audio encoder, the supplemental values comprise audio values, the received supplemental object comprises an audio object, the supplemental decoder comprises an audio decoder, the supplemental state values comprise audio state values, the supplemental value comprises an audio value, the other supplemental values comprise other audio values, and the other supplemental value comprises another audio value, and the supplemental encoder comprises a suggestion encoder, the supplemental values comprise suggestion values, the received supplemental object comprises a suggestion object, the supplemental decoder comprises a suggestion decoder, the supplemental state values comprise suggestion state values, the supplemental value comprises a suggestion value, the other supplemental values comprise other suggestion values, and the other supplemental value comprises another suggestion value.

7. A method comprising:

determining, by an intent encoder, intent values based on an object in a received phrase;

determining, by a detail encoder, detail values based on the object in the received phrase;

determining, by an intent decoder, intent state values based on the intent values and the detail values;

determining, by a detail decoder, detail state values and an intent detail value based on the intent values and the detail values;

determining, by the intent encoder, other intent values based on the intent values and another object in the received phrase;

determining, by the detail encoder, other detail values based on the detail values and the other object in the received phrase;

determining, by the intent decoder, a general intent value based on the other intent values, the other detail values, and the intent state values; and

determining, by the detail decoder, another intent detail value based on the other intent values, the other detail values, and the detail state values.

8. The method of claim 7, wherein the method further comprises:

at least one of modifying parameters in at least one of the intent encoder and the intent decoder based on a difference between the determined general intent value and a verified general intent; and

modifying parameters in at least one of the detail encoder and the detail decoder based on a difference between at least one of the determined intent detail value, the determined other intent detail value and at least one verified intent detail.

9. The method of claim 7, wherein the method further comprises outputting a response to the received phrase, the response being based on at least one of the determined general intent value, the determined intent detail value, and the determined other intent detail value.

10. The method of claim 7, wherein the method further comprises:

determining, by a supplemental encoder, supplemental values based on a received supplemental object;

determining, by a supplemental decoder, supplemental state values and a supplemental value based on the intent values, the detail values, and the supplemental values;

determining, by the supplemental encoder, other supplemental values based on the intent values, the detail values, and the received supplemental object; and

determining, by the supplemental decoder, another supplemental value based on the other intent values, the other detail values, and the supplemental state values.

11. The method of claim 10, wherein the supplemental encoder comprises an image encoder, the supplemental values comprise image values, the received supplemental object comprises an image object, the supplemental decoder comprises an image decoder, the supplemental state values comprise image state values, the supplemental value comprises an image value, the other supplemental values comprise other image values, and the other supplemental value comprises another image value.

12. The method of claim 10, wherein one of the supplemental encoder comprises an audio encoder, the supplemental values comprise audio values, the received supplemental object comprises an audio object, the supplemental decoder comprises an audio decoder, the supplemental state values comprise audio state values, the supplemental value comprises an audio value, the other supplemental values comprise other audio values, and the other supplemental value comprises another audio value, and the supplemental encoder comprises a suggestion encoder, the supplemental values comprise suggestion values, the received supplemental object comprises a suggestion object, the supplemental decoder comprises a suggestion decoder, the supplemental state values comprise suggestion state values, the supplemental value comprises a suggestion value, the other supplemental values comprise other suggestion values, and the other supplemental value comprises another suggestion value.

13. A method comprising:

determining, by an intent encoder, intent values based on an object in a received phrase;

determining, by a detail encoder, detail values based on the object in the received phrase;

determining, by an intent decoder, intent state values based on the intent values and the detail values;

determining, by the intent encoder, other intent values based on the intent values and another object in the received phrase;

determining, by the detail encoder, other detail values based on the detail values and the other object in the received phrase;

determining, by the intent decoder, a general intent value based on the other intent values, the other detail values, and the intent state values; and

modifying parameters in at least one of the intent encoder and the intent decoder based on a difference between the determined general intent value and a verified general intent.

14. The method of claim 13, wherein the method further comprises:

determining, by a detail decoder, detail state values and an intent detail value based on the intent values and the detail values;

determining, by the detail decoder, another intent detail value based on the other intent values, the other detail values, and the detail state values; and

modifying parameters in at least one of the detail encoder and the detail decoder based on a difference between at least one of the determined intent detail value, the determined other intent detail value and at least one verified intent detail.

15. A computer program product comprising computer-readable program code to be executed by one or more processors when retrieved from a non-transitory computer-readable medium, the program code including instructions to:

determine, by an intent encoder, intent values based on an object in a received phrase;

determine, by a detail encoder, detail values based on the object in the received phrase;

determine, by an intent decoder, intent state values based on the intent values and the detail values;

determine, by a detail decoder, detail state values and an intent detail value based on the intent values and the detail values;

determine, by the intent encoder, other intent values based on the intent values and another object in the received phrase;

determine, by the detail encoder, other detail values based on the detail values and the other object in the received phrase;

determine, by the intent decoder, a general intent value based on the other intent values, the other detail values, and the intent state values; and

determine, by the detail decoder, another intent detail value based on the other intent values, the other detail values, and the detail state values.

16. The computer program product of claim 15, wherein the program code further includes instructions to:

at least one of modify parameters in at least one of the intent encoder and the intent decoder based on a difference between the determined general intent value and a verified general intent; and

modify parameters in at least one of the detail encoder and the detail decoder based on a difference between at least one of the determined intent detail value, the determined other intent detail value and at least one verified intent detail.

17. The computer program product of claim 15, wherein the program code further includes instructions to output a response to the received phrase, the response being based on at least one of the determined general intent value, the determined intent detail value, and the determined other intent detail value.

18. The computer program product of claim 15, wherein the program code further includes instructions to:

determine, by a supplemental encoder, supplemental values based on a received supplemental object;

determine, by a supplemental decoder, supplemental state values and a supplemental value based on the intent values, the detail values, and the supplemental values;

determine, by the supplemental encoder, other supplemental values based on the intent values, the detail values, and the received supplemental object; and

determine, by the supplemental decoder, another supplemental value based on the other intent values, the other detail values, and the supplemental state values.

19. The computer program product of claim 18, wherein the supplemental encoder comprises an image encoder, the supplemental values comprise image values, the received supplemental object comprises an image object, the supplemental decoder comprises an image decoder, the supplemental state values comprise image state values, the supplemental value comprises an image value, the other supplemental values comprise other image values, and the other supplemental value comprises another image value.

20. The computer program product of claim 18, wherein one of the supplemental encoder comprises an audio encoder, the supplemental values comprise audio values, the received supplemental object comprises an audio object, the supplemental decoder comprises an audio decoder, the supplemental state values comprise audio state values, the supplemental value comprises an audio value, the other supplemental values comprise other audio values, and the other supplemental value comprises another audio value, and the supplemental encoder comprises a suggestion encoder, the supplemental values comprise suggestion values, the received supplemental object comprises a suggestion object, the supplemental decoder comprises a suggestion decoder, the supplemental state values comprise suggestion state values, the supplemental value comprises a suggestion value, the other supplemental values comprise other suggestion values, and the other supplemental value comprises another suggestion value.