User interaction with building controller device using a remote server and a duplex connection

• US 10,782,043 B2
• Filed: 07/03/2018
• Issued: 09/22/2020
• Est. Priority Date: 08/21/2013
• Status: Active Grant

• Current Assignee: Ademco Inc. (Resideo Technologies, Inc.)
• Original Assignee: Ademco Inc. (Resideo Technologies, Inc.)
• Inventors: Nemcek, Adam, Hrebicek, Tomas, Becvar, Daniel
• Primary Examiner(s): Furdge, Larry L
• Application Number:
  US16/027,063
• Publication Number:
  US 20180313566A1
• Time in Patent Office: 812 Days
• CPC Class Codes: F24F 11/30 for purposes related to the operation of the system, e.g. for safety or monitoring
• CPC Class Codes: F24F 11/52 Indication arrangements, e.g. displays
• CPC Class Codes: F24F 11/526 giving audible indications
• CPC Class Codes: F24F 11/56 Remote control
• CPC Class Codes: F24F 11/58 using Internet communication
• CPC Class Codes: F24F 11/62 characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
• CPC Class Codes: F24F 11/66 Sleep mode
• CPC Class Codes: F24F 2110/00 Control inputs relating to air properties
• CPC Class Codes: F24F 2120/00 Control inputs relating to users or occupants
• CPC Class Codes: F24F 2130/40 Noise
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05D 23/1905 associated with tele control
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00

Abstract

An HVAC controller may be controlled in response to a natural language audio message that is not recognizable by the HVAC controller as a command, where the natural language audio message is translated into a command recognizable by the HVAC controller. Voice recognition software identifies a trigger phrase included in the natural language audio message and in response the HVAC controller performs an action. In response to identifying a trigger phrase at the HVAC controller, the HVAC controller may establish a single duplex connection with a remote server having a voice interaction module. An end user may then have a continuous dialog with the voice interaction module via the HVAC controller over the established single duplex connection. The voice interaction module may allow for an end user to interrupt the dialog at any time, as desired.

Claims

1. A thermostat configured to control one or more HVAC components of an HVAC system, the thermostat comprising:

a housing, the housing configured to house;

a temperature sensor;

a microphone;

a speaker;

a controller operatively coupled to the temperature sensor, the microphone and the speaker, wherein the controller is configured to;

establish a full duplex connection over a network between the thermostat and a remote server for transmitting and receiving at least audio streams to and from the remote server and for receiving one or more signals for controlling one or more HVAC components of the HVAC system simultaneously over the network;

receive an audio voice stream captured using the microphone, and transmit the received audio voice stream to the remote server via the full duplex connection;

receive a voice synthesized audio stream from the remote server via the full duplex connection, and play the received voice synthesized audio stream over the speaker;

wherein at least part of the audio voice stream is transmitted over the full duplex connection at the same time that at least part of the voice synthesized audio stream is being received; and

control the one or more HVAC components of the HVAC system in accordance with a control algorithm.

2. The thermostat of claim 1, wherein the voice synthesized audio stream is configured by the remove server and then played over the speaker of the thermostat in response to the remote server receiving the audio voice stream from the controller.

3. The thermostat of claim 1, wherein the controller is further configured to receive one or more controller understandable commands from the remote server via the full duplex connection.

4. The thermostat of claim 3, wherein the one or more controller understandable commands are received from the remote server in response to the transmitted audio voice stream.

5. The thermostat of claim 3, wherein the control algorithm controls the one or more HVAC components in accordance with one or more set points, and wherein one or more of the controller understandable commands change one or more of the set points.

6. The thermostat of claim 5, wherein the one or more set points comprises a temperature set point.

7. The thermostat of claim 3, wherein the full duplex connection includes a first channel for transmitting the audio voice stream to the remote server, a second channel for receiving the voice synthesized audio stream from the remote server, and a third channel for receiving the one or more controller understandable commands from the remote server.

8. The thermostat of claim 1, wherein the controller is configured to establish the full duplex connection in response to the controller identifying a natural language trigger phrase in a trigger phrase voice stream captured using the microphone.

9. The thermostat of claim 1, wherein the playing of the voice synthesized audio stream over the speaker is interrupted when the audio voice stream or a subsequent audio voice stream captured using the microphone indicates that the voice synthesized audio stream is no longer applicable.

10. The thermostat of claim 1, wherein the full duplex connection between the thermostat and the remote server includes a Transmission Control Protocol (TCP).

11. The thermostat of claim 1, wherein the full duplex connection between the controller and the remote includes a multi-channel protocol.

12. A method of operating a thermostat of an HVAC system, the method comprising:

controlling one or more HVAC components of the HVAC system in accordance with a control algorithm that references one or more set points;

establishing a full duplex connection over a network between the thermostat and a remote server for transmitting and receiving at least audio streams to and from the remote server and for receiving one or more signals for controlling the one or more HVAC components of the HVAC system simultaneously over the network;

receiving an audio voice stream via a microphone of the thermostat;

transmitting the received audio voice stream to the remote server via the full duplex connection;

receiving at the thermostat a voice synthesized audio stream from the remote server via the full duplex connection, wherein the voice synthesized audio stream is in response to the audio voice stream transmitted to the remote server;

playing the received voice synthesized audio stream on a speaker of the thermostat;

interrupting the playing of the voice synthesized audio stream when the audio voice stream or a subsequent audio voice stream received via the microphone of the thermostat indicates that the voice synthesized audio stream is no longer applicable; and

receiving one or more commands at the thermostat from the remote server via the full duplex connection, with the one or more commands is/are based on the audio voice stream transmitted to the remote server, and wherein at least one of the received commands changes one or more of the set points referenced by the control algorithm.

13. The method of claim 12, wherein the voice synthesized audio stream is configured by the remove server and then played on the speaker of the thermostat in response to the remote server receiving the audio voice stream from the thermostat.

14. The method of claim 12, wherein at least one of the commands changes a temperature set point that is referenced by the control algorithm.

15. The method of claim 12, wherein the received audio voice stream is transmitted to the remote server via a first channel of the full duplex connection, the voice synthesized audio stream is received by the thermostat from the remote server via a second channel of the full duplex connection, and the one or more commands are received by the thermostat from the remote server via a third channel of the full duplex connection.

16. The method of claim 12, wherein the full duplex connection between the thermostat and a remote server is established in response to receiving via the microphone of the thermostat a natural language trigger phrase.

17. A server configured to communicate with one or more thermostats of one or more HVAC systems, the server comprising:

processing circuitry configured to;

establish a full duplex connection over a network with a thermostat for transmitting and receiving at least audio streams to and from the thermostat and for transmitting one or more signals to the thermostat for controlling the thermostat simultaneously over the network;

provide an interactive audio dialog with a user of the thermostat by identifying one or more words in a voice stream received from the thermostat over the full duplex connection, and in response, send a return voice message to the thermostat over the full duplex connection;

interrupt the return voice message when the voice stream indicates that the return voice message is no longer applicable; and

determine and send one or more control commands to the thermostat over the full duplex connection to change at least one control parameter of the thermostat.

18. The server of claim 17, wherein the return voice message is sent to the thermostat over the full duplex connection while still receiving the voice stream from the thermostat.

19. The server of claim 17, wherein the processing circuitry is configured to interrupt the return voice message when the voice interaction module identifies one or more subsequent words in the voice stream that indicate that the return voice message is no longer applicable.

20. The server of claim 17, wherein the processing circuitry is configured to terminate the full duplex connection between the server and the thermostat in response to the voice interaction module determining that the interactive audio dialogue is over.