Filtering audio-based interference from voice commands using natural language processing

• US 10,811,007 B2
• Filed: 06/08/2018
• Issued: 10/20/2020
• Est. Priority Date: 06/08/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Liu, Su, Rozner, Eric J., Hwang, Inseok, Yoo, Chungkuk
• Primary Examiner(s): Colucci, Michael
• Application Number:
  US16/004,229
• Publication Number:
  US 20190378503A1
• Time in Patent Office: 865 Days
• Field of Search: 704235, 704240, 704246, 704250, 704251, 704275, 379 8808, 455416, 701 2
• CPC Class Codes: G06F 40/20 Natural language analysis semantic analysis of natural language G06F40/30
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/51 for comparison or discrimination

Abstract

A computer-implemented method, according to one embodiment, includes: receiving a complex audio signal which includes an intended audio signal and at least one interfering audio signal. The complex audio signal is converted into text which represents a plurality of words included in the complex audio signal, and at least some of the text is identified as representing words which correspond to the at least one interfering audio signal. The identified text is discarded, and a remaining portion of the text is evaluated to determine whether the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in a predetermined range. Furthermore, the remaining portion of the text is output in response to determining that the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range.

Claims

1. A computer-implemented method, comprising:

receiving a complex audio signal, wherein the complex audio signal includes an intended audio signal and at least one interfering audio signal, wherein the intended audio signal is a voice-based command originating from a user, wherein the at least one interfering audio signal is background noise;

converting the intended audio signal and the at least one interfering audio signal into text which represents a plurality of words included in the complex audio signal;

identifying at least some of the text as representing words which correspond to the at least one interfering audio signal;

discarding the identified text;

evaluating a remaining portion of the text to determine whether the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in a predetermined range; and

outputting the remaining portion of the text in response to determining that the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range.

2. The computer-implemented method of claim 1, comprising:

identifying at least some of the text in the remaining portion of the text as representing words which correspond to the at least one interfering audio signal in response to determining that the remaining portion of the text represents words which convey the voice-based command at an accuracy that is not in the predetermined range;

discarding the identified text from the remaining portion of the text;

evaluating an updated remaining portion of the text to determine whether the updated remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range; and

outputting the updated remaining portion of the text in response to determining that the updated remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range.

3. The computer-implemented method of claim 1, wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

applying one or more natural language processing techniques to the text.

4. The computer-implemented method of claim 3, wherein applying one or more natural language processing techniques to the text includes:

comparing the text to known voice-based commands, wherein the known voice-based commands are previously logged commands;

detecting matches between portions of the text and the known voice-based commands; and

identifying the remaining text which does not match any of the known voice-based commands as representing words which correspond to the at least one interfering audio signal,

wherein comparing the text to known voice-based commands includes applying a clustering algorithm to the text.

5. The computer-implemented method of claim 1, comprising:

receiving information which corresponds to the at least one interfering audio signal, wherein the received information includes one or more audio samples collected by one or more other users at about the same time that the voice-based command originated from the user,

wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes;

comparing the one or more audio samples collected by the one or more other users against the complex audio signal, and

identifying any matches between the one or more audio samples and the complex audio signal as portions of the at least one interfering audio signal.

6. The computer-implemented method of claim 3, wherein applying one or more natural language processing techniques to the text includes:

comparing the text to a grammatical template;

detecting portions of the text which comply with the grammatical template; and

identifying the remaining text which does not comply with the grammatical template as representing words which correspond to the at least one interfering audio signal.

7. The computer-implemented method of claim 3, wherein applying one or more natural language processing techniques to the text includes:

using heuristic algorithms to compare the text to a word bank, wherein the word bank includes a plurality of common words that are detected frequently;

identifying portions of the text that match entries in the word bank as representing common words; and

identifying remaining portions of the text that do not match the entries in the word bank as representing words which correspond to the at least one interfering audio signal.

8. The computer-implemented method of claim 1, wherein outputting the remaining portion of the text includes:

selecting a known command which matches the remaining portion of the text most closely; and

outputting the known command,

wherein discarding the identified text includes erasing the identified text from memory.

9. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions readable and/or executable by a processor to cause the processor to perform a method comprising:

receiving, by the processor, a complex audio signal, wherein the complex audio signal includes an intended audio signal and at least one interfering audio signal, wherein the intended audio signal is a voice-based command originating from a user, wherein the at least one interfering audio signal is background noise;

converting, by the processor, the intended audio signal and the at least one interfering audio signal into text which represents a plurality of words included in the complex audio signal;

identifying, by the processor, at least some of the text as representing words which correspond to the at least one interfering audio signal;

discarding, by the processor, the identified text;

evaluating, by the processor, a remaining portion of the text to determine whether the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in a predetermined range; and

outputting, by the processor, the remaining portion of the text in response to determining that the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range.

10. The computer program product of claim 9, the program instructions readable and/or executable by the processor to cause the processor to perform the method comprising:

receiving, by the processor, information which corresponds to the at least one interfering audio signal,

wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes using the received information to identify the at least some of the text.

11. The computer program product of claim 10, wherein the received information includes:

a full copy of an audio file which produced the at least one interfering audio signal; and

a timing offset which identifies a portion of the audio file that matches the at least one interfering audio signal,

wherein using the received information to identify the at least some of the text as representing words which correspond to the at least one interfering audio signal includes comparing the audio file at the timing offset to the complex audio signal.

12. The computer program product of claim 9, wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

comparing the text to known voice-based commands;

detecting matches between portions of the text and the known voice-based commands; and

identifying the remaining text which does not match any of the known voice-based commands as representing words which correspond to the at least one interfering audio signal,

wherein comparing the text to known voice-based commands includes applying a clustering algorithm to the text.

13. The computer program product of claim 10, wherein the received information includes network activity experienced by a network router which corresponds to the user which issued the voice-based command included in the complex audio signal,

wherein using the received information to identify the at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

identifying audio signals included in the network activity;

comparing the audio signals identified in the network activity with the complex audio signal; and

identifying any matches between the audio signals identified in the network activity and the complex audio signal as portions of the at least one interfering audio signal.

14. The computer program product of claim 9, wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

comparing the text to a grammatical template;

detecting portions of the text which comply with the grammatical template; and

identifying the remaining text which does not comply with the grammatical template as representing words which correspond to the at least one interfering audio signal.

15. The computer program product of claim 9, wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

applying heuristic algorithms to detect portions of the text which represent uncommon words; and

identifying the detected portions of the text as representing words which correspond to the at least one interfering audio signal,

wherein applying heuristic algorithms to detect portions of the text which represent uncommon words includes;

using the heuristic algorithms to compare the text to a word bank of common words, and

detecting portions of the text which do not match the common words in the word bank.

16. The computer program product of claim 10, wherein the received information includes one or more audio samples collected by one or more other users at about the same time that the voice-based command originated from the user,

wherein using the received information to identify the at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

comparing the one or more audio samples collected by the one or more other users against the complex audio signal; and

identifying any matches between the one or more audio samples and the complex audio signal as portions of the at least one interfering audio signal,

wherein outputting the remaining portion of the text includes;

selecting a known command which matches the remaining portion of the text most closely; and

outputting the known command.

17. A system, comprising:

a processor; and

logic integrated with the processor, executable by the processor, or integrated with and executable by the processor, the logic being configured to;

receive, by the processor, a complex audio signal, wherein the complex audio signal includes an intended audio signal and at least one interfering audio signal, wherein the intended audio signal is a voice-based command originating from a user, wherein the at least one interfering audio signal is background noise;

convert, by the processor, the intended audio signal and the at least one interfering audio signal into text which represents a plurality of words included in the complex audio signal;

identify, by the processor, at least some of the text as representing words which correspond to the at least one interfering audio signal;

discard, by the processor, the identified text;

evaluate, by the processor, a remaining portion of the text to determine whether the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in a predetermined range; and

output, by the processor, the remaining portion of the text in response to determining that the remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range,

wherein outputting the remaining portion of the text includes;

selecting a known command which matches the remaining portion of the text most closely, and

outputting the known command.

18. The system of claim 17, the logic being configured to:

identify, by the processor, at least some of the text in the remaining portion of the text as representing words which correspond to the at least one interfering audio signal in response to determining that the remaining portion of the text represents words which convey the voice-based command at an accuracy that is not in the predetermined range;

discard, by the processor, the identified text from the remaining portion of the text;

evaluate, by the processor, an updated remaining portion of the text to determine whether the updated remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range;

output, by the processor, the updated remaining portion of the text in response to determining that the updated remaining portion of the text represents words which convey the voice-based command at an accuracy that is in the predetermined range; and

receive, by the processor, information which corresponds to the at least one interfering audio signal, wherein the received information includes;

a copy of a video file and/or an audio file which produced the at least one interfering audio signal, and

timing offsets which identify portions of the video file and/or audio file that match the at least one interfering audio signal,

wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes comparing the video file and/or the audio file at the timing offsets to the complex audio signal.

19. The system of claim 17, wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

comparing the text to known voice-based commands by applying a clustering algorithm to the text;

detecting matches between portions of the text and the known voice-based commands; and

identifying the remaining text which does not match any of the known voice-based commands as representing words which correspond to the at least one interfering audio signal.

20. The system of claim 17, wherein identifying at least some of the text as representing words which correspond to the at least one interfering audio signal includes:

comparing the text to a grammatical template;

detecting portions of the text which comply with the grammatical template;

comparing the detected portions of the text which comply with the grammatical template with a database of commands;

identifying matches between the detected portions of the text and the database of commands as representing words which correspond to the intended audio signal; and

identifying the remaining text which does not comply with the grammatical template as representing words which correspond to the at least one interfering audio signal,

wherein discarding the identified text includes marking data which corresponds to the identified text as invalid.