Information access and device control using mobile phones and audio in the home environment

• US 8,825,020 B2
• Filed: 02/08/2012
• Issued: 09/02/2014
• Est. Priority Date: 01/12/2012
• Status: Active Grant

• Current Assignee: Sensory Incorporated
• Original Assignee: Sensory Incorporated
• Inventors: Mozer, Todd F., Mozer, Forrest S.
• Primary Examiner(s): TRAN, CONGVAN
• Application Number:
  US13/369,129
• Publication Number:
  US 20130183944A1
• Time in Patent Office: 937 Days
• Field of Search: 455/414.1, 455418-420, 455/550.1, 455/563, 704/246, 704/270.1, 704/273, 704/274, 704/275, 704/E21.019, 379/102.01, 379/102.02, 379/102.03, 379/102.05
• US Class Current: 455/414.1
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: H04L 12/2818 from a device located outside both the home and the home network access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

Embodiments of the present invention are directed toward systems, methods and devices for improving information access to and device control in a home automation environment. Functionality of multiple household device, such as lights, sound, entertainment, HVAC, and communication devices can be activated via voice commands. The voice commands are detected by a nearby control device and relayed via a network communication medium to another control device to which the desired device or system that the user wants to operate is connected. Each control device, disposed throughout the home, can detect a voice command intended for another control box and household device and relay the voice command to the intended control box. In such systems, a user can initiate a telephone call by saying a voice command to a local control box that will forward on the control signal to a mobile phone connected to another control box.

Claims

1. A home automation system comprising:

a plurality of control devices associated with a plurality of corresponding physical locations and a plurality of corresponding household devices, wherein the plurality of control devices each comprise a processor, a memory coupled to the processor, a microphone coupled to the processor, a household device connection coupled to the processor, and a network transceiver coupled to the processor, wherein the memory comprises instructions, that when executed by the processor, cause the processor to be configured to;

receive a voice command from a user through the microphone;

determine a physical location of the user relative to the plurality of corresponding physical locations;

perform voice recognition on the voice command;

determine a first control signal in response to the voice recognition and the physical location;

transmit the first control signal using the network transceiver to one or more other control devices in the plurality of control devices over a network communication medium;

receive a second control signal using the network transceiver from one or more other control devices in the plurality of control devices over the network communication medium; and

perform an action in response to the second control signal.

2. The system of claim 1 wherein the action in response to the second control signal comprises sending a signal to a household device using the household device connection or relaying the second control signal to one or more other control devices in the plurality of control devices using the network transceiver.

3. The system of claim 1 wherein the network transceiver comprises a sonic network transceiver, and wherein the network communication medium comprises a sonic network communication protocol comprising sonic tones.

4. The system of claim 1 wherein the instructions further cause the processor to be configured to determine a voice pattern from the received voice command, and determine user preferences associated with the user in response to the voice pattern, and wherein the control signal is further determined in response to the user preferences.

5. The system of claim 1 wherein at least one household device connection in the plurality of control devices comprises an electrical outlet connector and an electrical receptacle configured to connect to a household device.

6. The system of claim 1 wherein the instructions further cause the processor to be configured to determine a time of day, and wherein the control signal is further determined in response to the time of day.

7. The system of claim 1 wherein the instructions of at least one memory in the plurality of control devices further cause the corresponding processor to be configured to receive the voice commands from the user from a portable communication device through an external communication medium.

8. The system of claim 1 wherein the instructions further cause the processor to be configured to use the microphone to detect characteristic sounds associated with the user, and to track the physical location of the user based on the characteristics sounds.

9. The system of claim 1 wherein at least one household device connection in the plurality of control devices comprises a connection to connect to a communication device, and wherein the action comprises sending and receiving signals through household device connection to control the communication device to conduct a telephone call.

10. The home automation control system of claim 9 wherein the communication device is selected from the group consisting of a cellular telephone, a smartphone, a laptop computer, a tablet computer, and a personal digital assistant.

11. A method comprising:

receiving, in a first control device comprising a microphone, a household device connection, and a network transceiver, voice commands from a user through the microphone, wherein the first control device is associated with a plurality of other control devices;

determining, in the first control device, a physical location of the user;

performing, in the first control device, voice recognition on the voice command;

recognizing, in the first control device, a voice pattern associated with the user from the voice command;

determining, in the first control device, a first control signal in response to the voice recognition, the physical location and the voice pattern; and

transmitting the first control signal using the network transceiver from the first control device to one or more of the plurality of other control devices over a network communication medium;

receiving, in the first control device, other control signals using the network transceiver from one or more of the plurality of other control devices over the network communication medium; and

performing, in the first control device, an action using the household device connection in response to the received other control signals.

12. The method of claim 11 further comprising receiving, in the first control device, an acknowledgement signal in response to the first control signal from the one or more of the plurality of other control devices through the network communication medium using the network transceiver and providing a feedback message.

13. The method of claim 12 wherein providing the feedback message comprises playing a synthesized voice sound.

14. The method of claim 11 wherein the voice command comprises a command to make a telephone call and wherein the control signal comprises a signal to activate a mobile telephone connected to one of the plurality of other control devices to conduct the telephone call.

15. The method of claim 14 further comprising transmitting sounds of the user from the first control device to the mobile telephone through the network communication medium and the one of the plurality of other control devices.

16. The method of claim 15 further comprising receiving sounds from the mobile telephone at the first control device through the network communication medium and the one of the plurality of other control devices and playing the sounds from the mobile phone.

17. The method of claim 11 further comprising:

receiving, by the first control device, from at least one of the plurality of other control devices, one or more signals corresponding to characteristic sounds associated with the user;

and wherein determining the physical location of the user is in response to the one or more signals corresponding to the characteristic sounds.

18. The method of claim 11 wherein the network communication medium comprises a sonic network communication protocol comprising sonic tones, and wherein transmitting the control signal comprises transmitting the first control signal in accordance with the sonic network communication protocol.

19. The method of claim 11 wherein the control signal comprises instructions for multiple household devices.

20. A control device comprising:

a microprocessor;

a memory coupled to the processor;

a microphone coupled to the processor;

a household device connection coupled to the processor; and

a network transceiver coupled to the processor, wherein the memory comprises instructions, that when executed by the processor, cause the processor to be configured to;

receive a voice command from a user through the microphone,

determine a physical location of the user;

perform voice recognition on the voice command;

determine a first control signal in response to the voice recognition and the physical location;

transmit the first control signal using the network transceiver to one or more other control devices in a plurality of other control devices over a network communication medium;

receive a second control signal using the network transceiver from one or more other control devices in the plurality of other control devices over the network communication medium; and

perform an action in response to the second control signal.

21. The control device of claim 20 wherein the action in response to the second control signal comprises sending a signal to a household device using the household device connection or relaying the second control signal to one or more other control devices in the plurality of other control devices over the network communication medium using the network transceiver.

22. The control device of claim 20 wherein the network transceiver comprises a sonic network transceiver, and wherein the network communication medium comprises a sonic network communication protocol comprising sonic tones.

23. The control device of claim 20 wherein the instructions further cause the processor to be configured to determine a voice pattern from the received voice command, and determine user preferences associated with the user in response to the voice pattern, and wherein the control signal is further determined in response to the user preferences.

24. The control device of claim 20 wherein the instructions further cause the processor to be configured to determine a time of day, and wherein the control signal is determined further in response to the time of day.

25. The control device of claim 20 wherein the instructions further cause the processor to be configured to use the microphone to detect characteristic sounds associated with the user, and to track the physical location associated with the user based on the characteristics sounds.