HISTORY-BASED KEY PHRASE SUGGESTIONS FOR VOICE CONTROL OF A HOME AUTOMATION SYSTEM

• US 20170097618A1
• Filed: 10/05/2015
• Published: 04/06/2017
• Est. Priority Date: 10/05/2015
• Status: Active Grant

• Current Assignee: Savant Systems, LLC
• Original Assignee: Savant Systems, LLC
• Inventors: Cipollo, Nicholas J., Kitson, Ryan E., Katsiris, George T.
• Granted Patent: US 10,018,977 B2
• Time in Patent Office: Days
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G08C 2201/31 Voice input
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04Q 2213/13103 Memory
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

In one embodiment, history-based key phrase suggestions are provided for voice control of a home automation system. A host controller builds a history database that maintains a history of events, each event indicating one or more services activated, one or more scenes activated or one or more types of media content presented by the home automation system. A predictions process on the host controller selects a subset of the events of the history database based on a present condition of the home automation system to produce a predication dataset. A grammar process on a remote control or mobile device selects one or more intentions from a list used by a voice recognition engine, based on the prediction dataset. Each intention includes a key phrase paired with an action. A voice user interface (UI) process suggests key phrases from the one or more selected intentions in a voice control UI on the remote control or mobile device.

Claims

1. A method for providing history-based key phrase suggestions for voice control of a home automation system, comprising

building a history database on a storage device that maintains events that have occurred in the home automation system, each event indicating one or more services activated, one or more scenes activated or one or more types of media content presented by the home automation system;

selecting a subset of the events of the history database based on a present condition of the home automation system to produce a predication dataset;

based on the prediction dataset, selecting one or more intentions from a list of intentions, wherein each intention includes a key phrase recognized by the voice recognition engine paired with an action performed in response to the key phrase; and

displaying, to a current user in a user interface on a remote control or mobile device of the home automation system, the key phrases from the one or more selected intentions as key phrase suggestions.

2. The method of claim 1, wherein each event in the history database is associated with a time, the present condition of the home automation system includes a present time, and the selecting is based on a comparison of the time of each event with the present time.

3. The method of claim 2, wherein the present time includes a present time of day and a present day of week and the comparison determines events in the history database that occurred on a same day of the week as the present day of week, in a time window that encompasses the present time of day on any day of the week, or in a time window that encompasses the present time of day on a same day of the week as the present day of the week.

4. The method of claim 1, wherein the prediction dataset is a scored prediction dataset, and the selecting further comprises determining a frequency of different types of events and scoring the different types of events based, at least in part, on the frequency.

5. The method of claim 4, wherein the selecting is based, at least in part, on a comparison of a time of each event with a time window, and the scoring is based on a length of the time window, with a shorter time window causing a higher score than a longer time window.

6. The method of claim 4, wherein the scoring is adjusted by a factor based on the type of event, with types of events indicating more specific user action receiving higher scores than events indicating less specific user action.

7. The method of claim 1, further comprising:

dynamically generating the list of intentions based upon at least one of configuration information for the home automation system, user content defined in the home automation system or a system state of the home automation system.

8. The method of claim 1, further comprising:

dynamically generating the list of intentions based upon the current user or a room being controlled.

9. The method of claim 1, further comprising:

storing the history database on a host controller that is in communication with the remote control or mobile device over a network.

10. The method of claim 1, wherein the list of intentions includes one or more service intentions whose action when performed activates one or more services within a room.

11. The method of claim 1, wherein the list of intentions includes one or more scenes intentions whose action when performed activates one or more scenes.

12. The method of claim 1, wherein the list of intentions includes one or more favorite intentions whose action when performed presents one or more types of media content associated with the current user.

13. The method of claim 1, further comprising:

receiving, at the voice recognition engine, speech input from a microphone of the remote control or mobile device;

comparing, by the voice recognition engine, the speech input against the key phrases of the list of intentions to find a matching intention; and

triggering the action associated with the matching intention.

14. The method of claim 13, wherein the action is a system action that activates one or more services, activates one or more scenes, or presents one or more types of media content in the home automation system.

15. The method of claim 13, wherein the action is a user interface (UI) action that controls one or more aspects of a home automation system control UI.

16. A home automation system that provides history-based key phrase suggestions for voice control, comprising:

a host controller that is coupled to and configured to control one or more audio/video (A/V) devices, lighting devices, heating ventilation and cooling (HVAC) devices or security devices within a structure to provide home automation, the host controller including

host activity recording software that builds a history database that maintains events that have occurred in the structure, each event indicating one or more services activated, one or more scenes activated or one or more types of media content presented, and

a predictions process configured to select a subset of the events of the history database based on a present condition of the home automation system to produce a predication dataset; and

a remote control or mobile device in communication with the host controller over a network, the remote control or mobile device including

a voice recognition engine, and

voice suggestion software configured to, based on the prediction dataset, select one or more intentions from a list of intentions, wherein each intention includes a key phrase recognized by the voice recognition engine paired with an action performed in response to the key phrase, and to display on a screen of the remote control or mobile device to a current user of the home automation system the key phrases from the one or more selected intentions as key phrase suggestions.

17. The home automation system of claim 16, each event in the history of events includes at least an indication of a time associated with the event, the present condition of the home automation system includes a present time, and the selection is based on a comparison of the time of each event with the present time.

18. The home automation system of claim 16, wherein the voice suggestion software includes a grammar process that is configured to dynamically generate the list of intentions based upon at least one of configuration information for the home automation system, user content defined in the home automation system or a system state of the home automation system.

19. The home automation system of claim 16, wherein the voice suggestion software includes a grammar process that is configured to dynamically generate the list of intentions based upon the current user or a room being controlled.

20. The home automation system of claim 16, wherein the voice recognition engine includes a recognizer process that is configured to receive speech input from a microphone of the remote control or mobile device, compare the speech input against the key phrases of the list of intentions to find a matching intention, and trigger the action associated with the matching intention.

22. A non-transitory electronic-device readable medium storing software that when executed on one or more processors is operable to:

receive a prediction dataset that includes events that occurred in a home automation system at a prior time that is associated with a present time, each event indicating one or more services activated, one or more scenes activated or one or more types of media content presented by the home automation system;

based on the prediction dataset, select one or more key phrases from a list of key phrases used by a voice recognition engine of the home automation system;

display the selected one or more key phases to a user as one or more key phrase suggestions;

receive speech input from a user corresponding to one of the one or more key phrase selections;

compare the speech input against the list of key phrases used by the voice recognition engine to find a match; and

trigger an action associated with the matching key phrase, wherein the action activates one or more services, activates one or more scenes, or presents one or more types of media content in the home automation system.

23. The non-transitory electronic-device readable medium of claim 22, wherein the list of key phrases is part of a list of intentions, with each intention pairing a key phrase and an action performed in response to the key phrase, and the selection of one or more key phrases includes the selection of one or more intentions.

24. The non-transitory electronic-device readable medium of claim 22, wherein the software when executed on the one or more processors is further operable to:

dynamically generate the list of intentions based upon at least one of configuration information for the home automation system, user content defined in the home automation system or a system state of the home automation system.

25. The non-transitory electronic-device readable medium of claim 22, wherein the software when executed on the one or more processors is further operable to:

dynamically generate the list of intentions based upon the current user or a room being controlled.