Devices and methods for interacting with a control system that is connected to a network

• US 10,145,579 B2
• Filed: 06/06/2013
• Issued: 12/04/2018
• Est. Priority Date: 05/01/2013
• Status: Active Grant

• Current Assignee: Ademco Inc. (Resideo Technologies, Inc.)
• Original Assignee: Honeywell International Inc.
• Inventors: Stoner, Marcus D., Kolavennu, Soumitri, Paine, Brad, Goh, Christopher
• Primary Examiner(s): SHECHTMAN, SEAN P
• Application Number:
  US13/911,638
• Publication Number:
  US 20140330435A1
• Time in Patent Office: 2,007 Days
• Field of Search: None
• CPC Class Codes: F24F 11/30 for purposes related to the operation of the system, e.g. for safety or monitoring
• CPC Class Codes: F24F 11/57 using telephone networks
• CPC Class Codes: F24F 11/58 using Internet communication
• CPC Class Codes: F24F 2110/00 Control inputs relating to air properties
• CPC Class Codes: F24F 2120/20 Feedback from users
• CPC Class Codes: F24F 2130/40 Noise
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: H04L 12/2816 Controlling appliance services of a home automation network by calling their functionalities arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station; in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00
• CPC Class Codes: H04L 51/56 Unified messaging, e.g. interactions between e-mail, instant messaging or converged IP messaging [CPM]

Abstract

A building automation system may be controlled in response to a natural language voice message. The natural language voice message may be recorded and then sent to a voice command manager via a network. The natural language voice message is then translated into a command recognizable by a building automation controller of the building automation system. Voice recognition software may be used to create a natural language text based message from the recorded natural voice message, and the natural language text based message may then be translated into the command recognizable by the building automation controller. In response to the command, the building automation controller may perform the desired action.

Claims

1. A system of controlling a network connected building automation system servicing a building, comprising:

a network;

a server located outside of the building and connected to the network;

a voice command manager implemented on the server;

a thermostat located inside of the building, the thermostat comprising a temperature sensor and a first voice sensor device, the thermostat is in communication with the voice command manager over the network and the thermostat is configured to receive natural language voice commands from a user via the first voice sensor device, and to communicate the received natural language voice commands in an audio file format to the voice command manager over the network without the thermostat attempting to recognize natural language voice commands in the audio file;

a second voice sensor device in communication with the voice command manager over the network, the second voice sensor device configured to receive natural language voice commands from a user via the second voice sensor device, and to communicate the natural language voice commands to the voice command manager over the network;

wherein the voice command manager is configured to translate the natural language voice commands received from the thermostat and the natural language voice commands received from the second voice sensor device into one or more commands understandable by the thermostat, and to transmit the one or more commands to the thermostat; and

wherein the voice command manager is configured to identify if there is a conflict between the natural language voice commands, and if so, send a query in a natural language format as to whether a second natural language voice command of the natural language voice commands should override a first natural language voice command of the natural language voice commands, and determine which one of the conflicting natural language voice commands is to be;

translated into one or more commands understandable by the thermostat; and

sent to the thermostat for execution.

2. The system of claim 1, wherein when the thermostat communicates a natural language voice command received by the thermostat from a user via the first voice sensor device to the voice command manager, and the second voice sensor device communicates a natural language voice command received by the second voice sensor device from a user to the voice command manager, the voice command manager is configured to translate both of the received natural language voice commands into corresponding commands understandable by the thermostat, and if there is an identified conflict, determine which of the corresponding commands, if any, is communicated to the thermostat.

3. The system of claim 2, wherein the voice command manager communicates back to the thermostat, the second voice sensor device, or both, which command of the corresponding commands was communicated to the thermostat.

4. The system of claim 1, wherein the voice command manager communicates back to the thermostat, the second voice sensor device, or both, with natural language acknowledgements.

5. The system of claim 1, wherein the second voice sensor device is a phone.

6. A system comprising:

a voice command manager for a building automation system of a building, comprising;

an input port for receiving two or more natural language voice commands over a network;

a memory for storing the received natural language voice commands;

a processor configured to translate each of the two or more natural language voice commands into one or more translated commands understandable by a building automation controller of the building automation system;

an output port for sending the one or more translated commands understandable by the building automation controller to the building automation controller for execution; and

wherein the processor is further configured to identify if there is a conflict between two or more of the natural language voice commands, and if so, send a query in a natural language format as to whether a second natural language voice command of the two or more natural language voice commands should override a first natural language voice command of the two or more natural language voice commands, and determine which one of the conflicting two or more natural language voice commands is to be;

translated into one or more commands understandable by the building automation controller;

provided to the output port; and

sent to the building automation controller for execution, so that only one of the conflicting two or more natural language voice commands is executed; and

a building automation controller configured to execute the sent command.

7. The system of claim 6, wherein the voice command manager is implemented in a server outside of the building.

8. The system of claim 6, wherein the input port of the voice command manager is implemented in a computer inside of the building, and wherein the input port of the voice command manager is in communication with a wireless network and receives at least one of the natural language voice commands via the wireless network.

9. The system of claim 8, wherein the input port receives at least one of the natural language voice commands from a voice sensor device located within the building via the wireless network.

10. The system of claim 6, wherein the input port receives at least one of the natural language voice commands from a voice sensor device located within the building.

11. The system of claim 10, wherein the voice sensor device is one or more of a thermostat, a security system panel, a sensor module, and a dedicated voice reception panel.

12. The system of claim 6, wherein the input port receives at least one of the natural language voice commands from a voice sensor device located outside of the building.

13. The system of claim 12, wherein the voice sensor device is one or more of a smart phone, a tablet computer, a desktop computer, and a lap top computer.

14. The system of claim 6, wherein the building automation system includes at least two sub-systems, wherein the processor is configured to determine which of the at least two sub-systems to send each of the one or more translated commands, and then sends each of the one or more translated commands to the appropriate sub-system via the output port for execution.

15. The system of claim 14, wherein the at least two sub-systems include one or more of an HVAC system, a security system, a lighting control system, a water heater system, a refrigerator, a clothes washer, a clothes dryer, and an oven.

16. The system of claim 6,

wherein the identified conflict is a schedule conflict between a first natural language voice command received at a first time and a second natural language voice command received at a second time.

17. A method of interfacing with a building automation system, the method comprising:

receiving natural language voice messages at a building automation controller, the natural language voice messages including natural language commands for the building automation controller of the building automation system;

communicating the natural language voice messages in an audio file format to a server that is located remotely from and in operative communication with the building automation controller without the building automation controller first attempting to recognize the natural language commands in the natural language voice message;

translating the natural language voice messages with the server to translated building controller commands understandable by the building automation controller of the building automation system;

identifying, with the server, if there is a conflict between the natural language commands, and if so, send a query in a natural language format as to whether a second natural language command of the natural language commands should override a first natural language command of the natural language commands, and determining, with the server, which one of the conflicting natural language commands is to be;

translated into one or more commands understandable by the building automation controller; and

sent to the building automation controller for execution;

sending the translated building controller command to the building automation controller of the building automation system for execution; and

executing the translated building controller command with the building automation controller.

18. The method of claim 17, further comprising:

through the building automation controller, providing a verbal communication of an action taken by the building automation controller in response to the translated command.

19. The method of claim 17, further comprising:

receiving a plurality of natural language voice messages from a plurality of voice reception units, including the building automation controller, where the plurality of natural language voices messages comprise two or more conflicting commands for the building automation controller of the building automation system; and

selecting with the server which of the two or more conflicting commands for the building automation controller to;

translate into one or more commands understandable by the building automation controller; and

send to the building automation controller for execution.

20. The method of claim 17, wherein the building automation controller comprises a thermostat.