Acoustic sensory network

• US 10,204,622 B2
• Filed: 09/09/2016
• Issued: 02/12/2019
• Est. Priority Date: 09/10/2015
• Status: Active Grant

• Current Assignee: Crestron Electronics Incorporated
• Original Assignee: Crestron Electronics Incorporated
• Inventors: Bargetzi, Fred, Seferian, Ara, Stene, Josh, LaBosco, Mark
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US15/261,296
• Publication Number:
  US 20170076721A1
• Time in Patent Office: 886 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

A system is provided for determining which controllable device an audible command is directed towards. The system comprises two or more controllable devices, two or more electronic devices, each of which is adapted to receive the audible command, add a respective electronic device identifier to the received audible command, time and date stamp the received audible command, and transmit the respective time and date stamped versions of the audible command, and wherein each of the two or more electronic device are further adapted to controller respective ones of the two or more controllable devices, and a central processor adapted to receive each of the transmitted time and date stamped versions of the audible command and perform processing based on the time and date stamp, wherein the electronic device that reports the earlier time and date stamp, as ascertained by the respective electronic device identifier and a comparison of each of the time and date stamps performed by the central processor, is the electronic device to which the audible command is directed towards.

Claims

1. A method for determining which controllable device an audible command is directed towards, the method comprising:

receiving at each of two or more controlling devices the audible command, the audible command being directed to control at least one of two or more controllable devices controlled by a respective one of the two or more controlling devices;

digitizing each of the received audible commands;

attaching a time-date stamp to each of the digitized audible commands using a time-date stamp generator that employs a time synch protocol, and further attaching a unique identifier to each of the time-date stamped digitized audible commands so as to uniquely correlate it to a respective controlling device;

determining a first received digitized audible command on the basis of an earliest time-date stamp, and further determining to which controlling device the audible command is directed to on the basis of the unique identifier of the first received digitized audible command;

performing speech recognition on the first received digitized audible command to determine a command for a controllable device; and

forwarding the command to the controlling device corresponding to the first received digitized audible command, the command corresponding to the spoken audible command that can be implemented on the controllable device controlled by the controlling device,

wherein the step of determining further comprises

determining a magnitude of each of the digitized audible commands from respective controlling devices, and

verifying that the magnitude of the digitized audible command with the earliest time-date stamp is equal to or larger than any other digitized audible commands; and

applying additional processing to determine which controlling device the received audible command is directed to if the magnitude of the received and digitized audible command with the earliest time-date stamp is less than any other received and digitized audible command,

wherein the step of applying additional processing comprises

checking one or more of an occupancy sensor reading, proximity sensor reading, and motion detector reading, each of which is associated with respective controlling devices to determine which controlling device the received audible command is directed to, and

determining that the occupancy sensor reading associated with a respective controlling device matches the received audible command in order to determine that the received audible command should be applied to the controllable device controlled by the respective controlling device.

2. The method according to claim 1, further comprising:

receiving the command by the controlling device; and

controlling the controllable device in accordance with the received command.

3. The method according to claim 1, further comprising:

determining that the proximity sensor reading associated with a respective controlling device matches the received audible command in order to determine that the received audible command should be applied to the controllable device controlled by the respective controlling device.

4. The method according to claim 1, further comprising:

determining that the motion detector reading associated with a respective controlling device matches the received audible command in order to determine that the received audible command should be applied to the controllable device controlled by the respective controlling device.

5. The method according to claim 1, further comprising:

applying noise reduction processing.

6. The method according to claim 5, wherein the step of applying noise reduction to the received audible command comprises:

filtering the received audible command in an analog domain to attenuate a first bandwidth of noise energy.

7. The method according to claim 5, wherein the step of applying noise reduction comprises:

using one or more of acoustic echo cancellation filtering, direction of arrival filtering, and directionally adaptive beam forming filtering, to filter the digitized audible command in a digital domain to attenuate noise energy and to amplify audible command energy.

8. The method according to claim 7, wherein the noise energy comprises:

noise energy generated by one or more of a fan motor, music, air conditioning noise, audio generated by multi-media presentations, and non-command words.

9. The method according to claim 1, further comprising:

receiving at least one additional audible command from at least one additional spatially separated microphone, the at least one additional spatially separated microphone associated with a respective controlling device;

digitizing the at least one additional audible command received from the at least one additional spatially separated microphone;

attaching a time-date stamp to the at least one digitized additional audible command, and further attaching a unique identifier to each time-date stamped digitized additional audible command so as to uniquely correlate it to its respective controlling device; and

using the at least one additional digitized audible command to assist in determining to which controlling device the audible command is directed to.

10. The method according to claim 1, wherein the step of determining, on the basis of the time-date stamp, to which controlling device the digitized audible command is directed to, comprises:

generating the time-date stamp using a time-date stamp generator, the time-date stamp generator located in at least one controlling device that includes a plurality of microphones located at the controlling device;

averaging all of the time-date stamps generated at the controlling device with a plurality of microphones; and

attaching the averaged time-date stamp to each of the plurality of digitized audible commando prior to the step of transmitting.

11. The method according to claim 1, wherein

the steps of determining, performing, and forwarding are performed by a central controller.

12. The method according to claim 1, wherein

the steps of digitizing, and attaching the time-date stamp are performed by the controlling device.

13. The method according to claim 1, wherein

the controllable devices include one or more of a sensor, lighting control device, shade device, audio/video device, environmental control device, security device, household appliance, control device, and industrial device.

14. The method according to claim 1, wherein the controlling device comprises:

a keypad.

15. An acoustic sensor network comprising:

two or more controllable devices;

two or more controlling devices, each of two or more controlling devices comprising at least one respective microphone, a time-date stamp generator, and an analog to digital converter, each of the two or more controlling devices being adapted to control a respective one of the two or more controllable devices, and wherein

each of the two or more controlling devices are further adapted to receive an audible command through its respective microphone, the received audible command being directed to control one of the two or more controllable devices controlled by a respective controlling device, and wherein

each of the analog to digital converters are adapted to digitize the received audible command, and wherein

each of the two or more controlling devices are further adapted to attach a time-date stamp to each of the digitized audible commands using the time-date stamp generator that employs a time synch protocol and attach a unique identifier to each of the time-date stamped digitized audible commands so as to uniquely correlate the time-date stamped digitized audible command to a respective controlling device; and

a central controller adapted to determine, on the basis of an earliest time-date stamp, a first received digitized audible command and the controlling device to which the audible command is directed to, and wherein

the central controller is further adapted to perform speech recognition on the first received digitized audible command to determine a command for a controllable device, and wherein the central controller is further adapted to

forward the command to the controlling device corresponding to the first received digitized audible command, the command corresponding to the audible command that can be implemented on the controllable device controlled by the controlling device**, and wherein

the central controller is further adapted to

determine a magnitude of each of the digitized audible commands from respective controlling devices,

verify that the magnitude of the digitized audible command with the earliest time-date stamp is equal to or larger than any other digitized audible command signal,

apply additional processing to determine which controlling device the received audible command is directed to if the magnitude of the digitized audible command signal with the earliest time-date stamp is less than any other received audible command,

check one or more of an occupancy sensor reading, proximity sensor reading, and motion detector reading, each of which is associated with respective controlling devices to determine which controlling device the received audible command is directed to, and

determine that the occupancy sensor reading associated with a respective controlling device matches the received audible command in order to determine that the received audible command should be applied to the controllable device controlled by the respective controlling device.

16. The acoustic sensor network according to claim 15, wherein

the controlling device that receives the command is adapted to control the controllable device in accordance with the received command.

17. The acoustic sensor network according to claim 15, wherein

the central controller is further adapted to

determine that the proximity sensor reading associated with a respective controlling device matches the received audible command in order to determine that the received audible command should be applied to the controllable device controlled by the respective controlling device.

18. The acoustic sensor network according to claim 15, wherein

the central controller is further adapted to

determine that the motion detector reading associated with a respective controlling device matches the received audible command in order to determine that the received audible command should be applied to the controllable device controlled by the respective controlling device.

19. The acoustic sensor network according to claim 15, wherein the controlling device further comprises:

a noise reduction processing circuit.

20. The acoustic sensor network according to claim 19, wherein

the noise reduction processing circuit is adapted to

filter the received analog audible command signal in an analog domain to attenuate a first bandwidth of noise energy.

21. The acoustic sensor network according to claim 19, wherein

the noise reduction processing circuit is adapted to

use one or more of acoustic echo cancellation filtering, direction of arrival filtering, and directionally adaptive beam forming filtering, to filter the digital audible command signal in a digital domain to attenuate noise energy and to amplify audible command energy.

22. The acoustic sensor network according to claim 21, wherein the noise energy comprises:

noise energy generated by one or more of a fan motor, music, air conditioning noise, audio generated by multi-media presentations, and non-command words.

23. The acoustic sensor network according to claim 15, further comprising:

at least one additional spatially separated microphone, adapted to receive the audible command and associated with one of the at least two controlling devices;

an analog to digital converter associated with the at least one additional spatially separated microphone, and adapted to digitize the received audible command;

a time-date stamp generator adapted to add a time-date stamp to the at least one additional digitized audible command, and further adapted to add a unique identifier to the at least one time-dated stamped additional digitized audible command, the unique identifier corresponding to the associated one of the controlling devices, and further wherein

the central controller uses the at least one additional digitized audible command to assist in determining to which controlling device the audible command is directed to.

24. The acoustic sensor network according to claim 15, wherein the controlling device further comprises:

a time-date stamp generator adapted to generate the time-date stamp; and

at least two microphones, each of which digitizes the received audible command.

25. The acoustic sensor network according to claim 24, wherein

the controlling device is further adapted to average all of the time-date stamps generated at the controlling device, and attach the averaged time-date stamps to each of the plurality of digitized audible commands prior to transmitting the same.

26. The acoustic sensor network according to claim 15, wherein

the controllable devices include one or more of a sensor, lighting control device, shade device, audio/video device, environmental control device, security device, household appliance, control device, and industrial device.

27. The acoustic sensor network according to claim 15, wherein the controlling device comprises:

a keypad.