Audio firewall

• US 10,607,610 B2
• Filed: 05/29/2018
• Issued: 03/31/2020
• Est. Priority Date: 05/29/2018
• Status: Active Grant

• Current Assignee: NICE North America LLC
• Original Assignee: Nortek Security & Control LLC (Melrose Industries Plc)
• Inventors: Bunker, Philip Alan, Saxena, Mayank
• Primary Examiner(s): Han, Qi
• Application Number:
  US15/991,809
• Publication Number:
  US 20190371337A1
• Time in Patent Office: 672 Days
• Field of Search: 704273, 704274, 704275, 7042701, 704231, 704258
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

An audio firewall system has a microphone that generates audio data. A speech-to-text engine converts the audio data to text data. The text data is parsed for a service wake word and corresponding content data. The service wake word identifies one of a local security system and a remote assistant server. A text-to-speech engine converts the service wake word and the corresponding content data to converted audio data. The converted audio data is provided to the remote assistant server. The content data is provided to the local security system. The audio firewall system receives a response from the remote assistant server or the local security system and outputs an audio signal corresponding to the response.

Claims

1. An audio firewall system comprising:

a microphone configured to generate audio data;

a speech-to-text engine configured to convert the audio data to text data prior to detecting a service wake word;

a service engine configured to detect the service wake word in the text data after parsing the text data for the service wake word and corresponding content data, the service wake word identifying one of a local security system and a remote assistant server;

a text-to-speech engine configured to convert the text data comprising the service wake word and the corresponding content data to converted audio data;

an audio anonymizer coupled between the microphone and the speech-to-text engine, the audio anonymizer configured to adjust at least one of a pitch and a speed of the audio data, and to provide the adjusted audio data to the speech-to-text engine;

a remote service interface configured to provide the converted audio data to the remote assistant server; and

a local security system interface configured to provide the content data to the local security system.

2. The audio firewall system of claim 1, wherein the remote service interface is configured to receive a response from the remote assistant server in response to providing the converted audio data to the remote assistant server, and further comprising:

a speaker configured to output an audio signal corresponding to the response.

3. The audio firewall system of claim 1, wherein the local security system interface is configured to receive a response from the local security system in response to providing the content data to the local security system, and further comprising:

a speaker configured to output an audio signal corresponding to the response.

4. The audio firewall system of claim 1, wherein the service engine is configured to identify the service wake word from a plurality of service wake words, and to identify the remote assistant server from a plurality of remote assistant servers, the remote assistant server corresponding to the service wake word, each remote assistant server identified with a corresponding service wake word.

5. The audio firewall system of claim 4, wherein the service engine is configured to receive a custom service wake word, to determine that the custom service wake word is different from the plurality of service wake words from the plurality of remote assistant servers, and to associate the custom service wake word with the local security system in response to determining that the custom service wake word is different from the plurality of service wake words.

6. The audio firewall system of claim 1, wherein the service engine is configured to identify the service wake word, and to identify the local security system corresponding to the service wake word.

7. The audio firewall system of claim 1, wherein the corresponding content data includes a request for the remote assistant server, wherein the remote service interface is configured to receive a response from the remote assistant server in response to the request.

8. The audio firewall system of claim 1, wherein the remote service interface is configured to communicate with a plurality of remote assistant servers, each remote assistant server having a corresponding service wake word.

9. The audio firewall system of claim 1, wherein the local security system is configured to receive the content data, to identify a device connected to the local security system based on the content data, to generate a command to the device based on the content data, and to receive a response from the device, and

wherein the audio firewall system further comprises a speaker configured to generate an audio signal corresponding to the response from the device.

10. A method comprising:

generating audio data with a microphone of an audio firewall system;

converting the audio data to text data with a speech-to-text engine prior to detecting a service wake word;

detecting the service wake word in the text data after parsing the text data for the service wake word and corresponding content data, the service wake word identifying one of a local security system and a remote assistant server;

converting the text data comprising the service wake word and the corresponding content data to converted audio data using a text-to-speech engine;

adjusting at least one of a pitch and a speed of the audio data;

providing the adjusted audio data to the speech-to-text engine;

providing the converted audio data to the remote assistant server; and

providing the content data to the local security system.

11. The method of claim 10, further comprising:

receiving a response from the remote assistant server in response to providing the converted audio data to the remote assistant server; and

outputting an audio signal corresponding to the response with a speaker at the audio firewall system.

12. The method of claim 10, further comprising:

receiving a response from the local security system in response to providing the content data to the local security system; and

outputting an audio signal corresponding to the response with a speaker at the audio firewall system.

13. The method of claim 10, further comprising:

identifying the service wake word from a plurality of service wake words, in the text data; and

identifying the remote assistant server from a plurality of remote assistant servers, the remote assistant server corresponding to the service wake word, each remote assistant server identified with a corresponding service wake word.

14. The method of claim 13, further comprising:

receiving a custom service wake word;

determining that the custom service wake word is different from the plurality of service wake words from the plurality of remote assistant servers; and

associating the custom service wake word with the local security in response to determining that the custom service wake word is different from the plurality of service wake words.

15. The method of claim 10, further comprising:

identifying the service wake word; and

identifying the local security system corresponding to the service wake word.

16. The method of claim 10, wherein the audio firewall system is configured to communicate with a plurality of remote assistant servers, each remote assistant server having a corresponding service wake word.

17. The method of claim 10, further comprising:

identifying a device connected to the local security system based on the content data;

generating a command to the device based on the content data;

receiving a response from the device; and

generating an audio signal corresponding to the response from device.

18. A machine-storage medium storing instructions that, when executed by one or more processors of a machine, cause the one or more processors to perform operations comprising:

generating audio data with a microphone of an audio firewall system;

converting the audio data to text data prior to detecting a service wake word;

detecting the service wake word in the text data after parsing the text data for the service wake word and corresponding content data, the service wake word identifying one of a local security system and a remote assistant server;

converting the text data comprising the service wake word and the corresponding content data to converted audio data using a text-to-speech engine;

adjusting at least one of a pitch and a speed of the audio data;

providing the adjusted audio data to the speech-to-text engine;

providing the converted audio data to the remote assistant server; and

providing the content data to the local security system.