User interaction with building controller device using a remote server and a duplex connection

• US 10,030,878 B2
• Filed: 06/11/2015
• Issued: 07/24/2018
• Est. Priority Date: 08/21/2013
• Status: Active Grant

• Current Assignee: Ademco Inc. (Resideo Technologies, Inc.)
• Original Assignee: Honeywell International Inc.
• Inventors: Nemcek, Adam, Hrebicek, Tomas, Becvar, Daniel
• Primary Examiner(s): Furdge, Larry
• Application Number:
  US14/737,076
• Publication Number:
  US 20150276254A1
• Time in Patent Office: 1,139 Days
• CPC Class Codes: F24F 11/30 for purposes related to the operation of the system, e.g. for safety or monitoring
• CPC Class Codes: F24F 11/52 Indication arrangements, e.g. displays
• CPC Class Codes: F24F 11/526 giving audible indications
• CPC Class Codes: F24F 11/56 Remote control
• CPC Class Codes: F24F 11/58 using Internet communication
• CPC Class Codes: F24F 11/62 characterised by the type of control or by internal processing, e.g. using fuzzy logic, adaptive control or estimation of values
• CPC Class Codes: F24F 11/66 Sleep mode
• CPC Class Codes: F24F 2110/00 Control inputs relating to air properties
• CPC Class Codes: F24F 2120/00 Control inputs relating to users or occupants
• CPC Class Codes: F24F 2130/40 Noise
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05D 23/1905 associated with tele control
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00

Abstract

An HVAC controller may be controlled in response to a natural language audio message that is not recognizable by the HVAC controller as a command, where the natural language audio message is translated into a command recognizable by the HVAC controller. Voice recognition software identifies a trigger phrase included in the natural language audio message and in response the HVAC controller performs an action. In response to identifying a trigger phrase at the HVAC controller, the HVAC controller may establish a single duplex connection with a remote server having a voice interaction module. An end user may then have a continuous dialog with the voice interaction module via the HVAC controller over the established single duplex connection. The voice interaction module may allow for an end user to interrupt the dialog at any time, as desired.

Claims

1. A thermostat configured to control one or more HVAC components of an HVAC system, the thermostat comprising:

a housing, the housing configured to house;

a temperature sensor;

a microphone;

a controller operatively coupled to the temperature sensor and the microphone; and

wherein the controller is configured to identify an audio trigger received by the microphone, and in response to identifying the audio trigger, establish a full duplex connection between the thermostat and a remote server, wherein the audio trigger is a trigger phrase contained in an audio voice stream.

2. The thermostat of claim 1, wherein the audio trigger is identified by the controller of the thermostat.

3. The thermostat of claim 1, further comprising:

a speaker at least partially housed by the housing; and

wherein the controller is configured to play an audio voice message on the speaker, wherein the audio voice message is received from the remote server via the established full duplex connection.

4. The thermostat of claim 1, wherein the established full duplex connection between the thermostat and the remote server includes a plurality of sub-channels.

5. The thermostat of claim 4, wherein the plurality of sub-channels include a first sub-channel configured to send audio data to the remote server, a second sub-channel configured to receive audio data from the remote server, and a third sub-channel configured to receive one or more controller understandable commands from the remote server.

6. The thermostat of claim 5, wherein the controller of the thermostat is configured to receive audio data from the remote server over the second sub-channel while at the same time send audio data from the controller to the remote server over the first sub-channel.

7. The thermostat of claim 1, wherein the established full duplex connection between the thermostat and the remote server uses a single Transmission Control Protocol (TCP).

8. The thermostat of claim 1, wherein the full duplex connection between the controller and the remote server is established using a multi-channel protocol.

9. A method of operating a thermostat of an HVAC system, the method comprising:

the thermostat identifying an audible trigger phrase provided by a user;

in response to identifying the trigger phrase, the thermostat initiating a full duplex connection between the thermostat and a remote server;

in response to identifying the trigger phrase, the thermostat providing an audible request for a voice command from the user;

in response to the audible request, the thermostat configured to receive a voice stream from the user, the voice stream including a voice command;

the thermostat providing the received voice stream from the user to the remote server via the full duplex connection; and

the remote server processing the voice stream to identify the voice command.

10. The method of claim 9, wherein

the voice stream is received by the thermostat from the user via a microphone of the thermostat after providing the audible request via a speaker of the thermostat.

11. The method of claim 9, wherein identifying the audible trigger phrase includes identifying the trigger phrase in a voice stream that is received via a microphone of the thermostat.

12. The method of claim 9, further comprising:

receiving an audio message at the thermostat from the remote server after the remote server has identified the voice command.

13. The method of claim 12, further comprising:

sending a voice stream from the thermostat to the remote server over the full duplex connection while receiving the audio message from the remote server.

14. A server configured to communicate with one or more thermostats of one or more HVAC systems, the server comprising:

a communications module for establishing a full duplex connection with a thermostat upon a request by the thermostat, wherein the thermostat includes a microphone configured to receive an audio trigger that initiates the request by the thermostat;

a voice interaction module operatively coupled to the communications module, wherein the voice interaction module is configured to provide an interactive audio dialog with a user of the thermostat by identifying one or more words in a voice stream received from the thermostat over a first channel of the full duplex connection, and in response, sending a return voice message to the thermostat over a second channel of the full duplex connection; and

wherein in response to identifying one or more words in a voice stream, the voice interaction module is further configured to determine and then send a control command to the thermostat over a third channel of the full duplex connection to change at least one control parameter of the thermostat.

15. The server of claim 14, wherein the return voice message is sent to the thermostat over the full duplex connection while still receiving the voice stream from the thermostats.

16. The server of claim 15, wherein the voice interaction module is configured to interrupt the return voice message if the voice interaction module identifies one or more subsequent words in the voice stream that indicate that the return voice message is no longer applicable.

17. The server of claim 14, wherein the voice interaction module is configured to terminate the full duplex connection between the server and the thermostat in response to the voice interaction module determining that the interactive audio dialogue is over.