Conversational interface for matching text of spoken input based on context model

• US 10,274,911 B2
• Filed: 06/25/2015
• Issued: 04/30/2019
• Est. Priority Date: 06/25/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Uppala, Anantha Deepthi, Jackson, Kara E., Yarvis, Mark D., Cobbley, David J., Dickinson, Andrew S., Pandit, Milind
• Primary Examiner(s): Lerner, Martin
• Application Number:
  US14/750,482
• Publication Number:
  US 20160378080A1
• Time in Patent Office: 1,405 Days
• Field of Search: 704255, 704257, 704270, 7042701, 704275, 704243, 704244
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Technologies for conversational control of a home automation system include a computing device in communication with one or more home automation devices. The computing device receives text input indicative of a spoken user request including one or more parameters associated with a home automation command. The computing device matches the text input against a home automation language model including several predefined dialog patterns. The computing device resolves parameters of the command using a context model. The computing device may resolve the location, device, or requested device state of the command. The computing device generates a text response and a home automation command using the language model, and then presents the text response to the user and performs the home automation command using the home automation devices. In some embodiments, the computing device may control an application automation system or a data exploration system. Other embodiments are described and claimed.

Claims

1. A computing device for conversational system control, the computing device comprising:

a controller module to receive input text indicative of a spoken user request, wherein the spoken user request includes one or more parameters associated with a system control command for a controlled system;

a conversational interface module to match the input text against a language model to identify a matching pattern, wherein the language model includes a plurality of dialog patterns, wherein the dialog patterns are arranged hierarchically to define one or more dialogs, and wherein each dialog pattern, when evaluated against the input text, determines whether the input text matches the corresponding dialog pattern, and wherein the matching pattern is associated with the system control command; and

a context management module to determine one or more of the parameters of the system control command based on a context model in response to a match of the input text;

wherein the conversational interface module is further to generate the system control command based on the language model in response to a determination of the one or more parameters; and

wherein the controller module is further to perform the system control command to cause a change in system state of the controlled system, wherein the controlled system comprises a home automation system, and wherein the change in the system state comprises a change in device state of a device of the home automation system, wherein the device state comprises an activation state of the device or a set point of the device;

wherein the computing device further comprises a learning module to (i) observe the change in the system state of the controlled system in response to performance of the system control command, (ii) generate a new dialog pattern based on the input text and the change in the device state, wherein the new dialog pattern is associated with the system control command, and (iii) update the language model to include the new dialog pattern.

2. The computing device of claim 1, wherein:

the matching pattern is further associated with text response;

the conversational interface module is further to generate the text response based on the language model; and

the controller module is further to present the text response to a user of the computing device.

3. The computing device of claim 1, further comprising:

an automated speech recognition module to perform automated speech recognition to generate the input text; and

a text-to-speech module to generate speech output based on the text response with a text-to-speech engine.

4. The computing device of claim 1, wherein:

the system control command comprises a home automation command;

the one or more parameters of the home automation command include a location, a device, or a requested device state; and

the context model includes conversational context data, user context data, or home automation device context data.

5. The computing device of claim 4, wherein to determine the one or more of the parameters of the home automation command based on the context model comprises to determine a location parameter, a device parameter, or a device state parameter of the home automation command based on the conversational context data.

6. The computing device of claim 4, wherein to determine the one or more of the parameters of the home automation command based on the context model comprises to determine a location parameter of the home automation command based on the user context data.

7. The computing device of claim 4, wherein to determine the one or more of the parameters of the home automation command based on the context model comprises to determine a device parameter or a device state parameter of the home automation command based on the home automation device context data.

8. The computing device of claim 4, wherein the context management module is further to update the context model based on data received from the home automation system.

9. The computing device of claim 4, wherein:

the controller module is further to determine whether the home automation command is valid based on the home automation device context data of the context model in response to the determination of the one or more parameters of the home automation command; and

the conversational interface module is to generate an error message for the text response with the language model in response to a determination that the home automation command is not valid;

wherein to generate the system control command based on the language model comprises to generate the home automation command in response to a determination that the home automation command is valid.

10. A method for conversational system control, the method comprising:

receiving, by a computing device, input text indicative of a spoken user request, wherein the spoken user request includes one or more parameters associated with a system control command for a controlled system;

matching, by the computing device, the input text against a language model to identify a matching pattern, wherein the language model includes a plurality of dialog patterns, wherein the dialog patterns are arranged hierarchically to define one or more dialogs, and wherein each dialog pattern, when evaluated against the input text, determines whether the input text matches the corresponding dialog pattern, and wherein the matching pattern is associated with the system control command;

determining, by the computing device, one or more of the parameters of the system control command using a context model in response to matching the input text;

generating, by the computing device, the system control command using the language model in response to determining the one or more parameters;

performing, by the computing device, the system control command to cause a change in system state of the controlled system, wherein the controlled system comprises a home automation system, and wherein the change in the system state comprises a change in device state of a device of the home automation system, wherein the device state comprises an activation state of the device or a set point of the device;

observing, by the computing device, the change in the system state of the controlled system in response to performing the system control command;

generating, by the computing device, a new dialog pattern based on the input text and the change in the system state, wherein the new dialog pattern is associated with the system control command; and

updating, by the computing device, the language model to include the new dialog pattern.

11. The method of claim 10, further comprising:

generating, by the computing device, a text response using the language model in response to determining the one or more parameters, wherein the matching pattern is associated with the text response; and

presenting, by the computing device, the text response to a user of the computing device.

12. The method of claim 10, wherein:

the system control command comprises a home automation command;

the one or more parameters of the home automation command include a location, a device, or a requested device state; and

the context model includes conversational context data, user context data, or home automation device context data.

13. The method of claim 12, further comprising:

determining, by the computing device, whether the home automation command is valid based on the home automation device context data of the context model in response to determining the one or more parameters of the home automation command; and

generating, by the computing device, an error message for the text response using the language model in response to determining that the home automation command is not valid;

wherein generating the system control command based on the language model comprises generating the home automation command in response to determining that the home automation command is valid.

14. One or more non-transitory, computer-readable storage media comprising a plurality of instructions that in response to being executed cause a computing device to:

receive input text indicative of a spoken user request, wherein the spoken user request includes one or more parameters associated with a system control command for a controlled system;

match the input text against a language model to identify a matching pattern, wherein the language model includes a plurality of dialog patterns, wherein the dialog patterns are arranged hierarchically to define one or more dialogs, and wherein each dialog pattern, when evaluated against the input text, determines whether the input text matches the corresponding dialog pattern, and wherein the matching pattern is associated with the system control command;

determine one or more of the parameters of the system control command using a context model in response to matching the input text;

generate the system control command using the language model in response to determining the one or more parameters;

perform the system control command to cause a change in system state of the controlled system, wherein the controlled system comprises a home automation system, and wherein the change in the system state comprises a change in device state of a device of the home automation system, wherein the device state comprises an activation state of the device or a set point of the device;

observe the change in the system state of the controlled system in response to performing the system control command;

generate a new dialog pattern based on the input text and the change in the system state, wherein the new dialog pattern is associated with the system control command; and

update the language model to include the new dialog pattern.

15. The one or more non-transitory, computer-readable storage media of claim 14 further comprising a plurality of instructions that in response to being executed cause the computing device to:

generate a text response using the language model in response to determining the one or more parameters, wherein the matching pattern is associated with the text response; and

present the text response to a user of the computing device.

16. The one or more non-transitory, computer-readable storage media of claim 14, wherein:

the system control command comprises a home automation command;

the one or more parameters of the home automation command include a location, a device, or a requested device state; and

the context model includes conversational context data, user context data, or home automation device context data.

17. The one or more non-transitory, computer-readable storage media of claim 16, wherein to determine the one or more of the parameters of the home automation command using the context model comprises to determine a location parameter, a device parameter, or a device state parameter of the home automation command based on the conversational context data.

18. The one or more non-transitory, computer-readable storage media of claim 16, wherein to determine the one or more of the parameters of the home automation command using the context model comprises to determine a location parameter of the home automation command based on the user context data.

19. The one or more non-transitory, computer-readable storage media of claim 16, wherein to determine the one or more of the parameters of the home automation command using the context model comprises to determine a device parameter or a device state parameter of the home automation command based on the home automation device context data.

20. The one or more non-transitory, computer-readable storage media of claim 16, further comprising a plurality of instructions that in response to being executed cause the computing device to:

determine whether the home automation command is valid based on the home automation device context data of the context model in response to determining the one or more parameters of the home automation command; and

generate an error message for the text response using the language model in response to determining that the home automation command is not valid;

wherein to generate the system control command based on the language model comprises to generate the home automation command in response to determining that the home automation command is valid.