Automatic censorship of audio data for broadcast

• US 20060095262A1
• Filed: 10/28/2004
• Published: 05/04/2006
• Est. Priority Date: 10/28/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Danieli, Damon V.
• Granted Patent: US 7,437,290 B2
• Time in Patent Office: Days
• US Class Current: 704/251
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence

Abstract

An input audio data stream comprising speech is processed by an automatic censoring filter in either a real-time mode, or a batch mode, producing censored speech that has been altered so that undesired words or phrases are either unintelligible or inaudible. The automatic censoring filter employs a lattice comprising either phonemes and/or words derived from phonemes for comparison against corresponding phonemes or words included in undesired speech data. If the probability that a phoneme or word in the input audio data stream matches a corresponding phoneme or word in the undesired speech data is greater than a probability threshold, the input audio data stream is altered so that the undesired word or a phrase comprising a plurality of such words is unintelligible or inaudible. The censored speech can either be stored or made available to an audience in real-time.

Claims

1. A method for automatically censoring audio data, comprising the steps of:

(a) automatically processing the audio data to detect any undesired speech that may be included therein, by comparison to undesired speech data; and

(b) for each occurrence of undesired speech that is automatically detected, altering the undesired speech detected in the audio data, producing censored audio data in which the undesired speech is substantially no longer perceivable by a listening audience.

2. The method of claim 1, wherein the step of automatically processing comprises the steps of:

(a) generating a lattice of phonemes comprising word fragments that are likely included in the audio data;

(b) comparing the word fragments against corresponding word fragments that are included in the undesired speech data, to identify potential matches; and

(c) based upon a probability of a potential match and a probability threshold, determining whether any undesired speech is included in the audio data.

3. The method of claim 1, wherein the step of automatically processing comprises the steps of:

(a) comparing words in the audio data against words comprising the undesired speech, to identify potential matches; and

(b) based upon a probability of a potential match and a probability threshold, determining whether any undesired speech is included in the audio data.

4. The method of claim 2, further comprising the step of dynamically varying the probability threshold dependent upon at least one criterion.

5. The method of claim 4, wherein the at least one criterion includes at least one of:

(a) an expected audience for the audio data;

(b) an identity of a speaker that uttered the audio data;

(c) a time at which the audio data will be heard by an audience;

(d) a type of event resulting in the audio data;

(e) an application of the audio data; and

(f) a frequency with which undesired speech by a specific speaker has previously been detected in the audio data.

6. The method of claim 1, wherein the step of altering comprises one of the steps of:

(a) substantially reducing a volume of any portions of the audio data that match the undesired speech so that the portions are substantially inaudible;

(b) deleting any portions of the audio data that match the undesired speech;

(c) overwriting any portions of the audio data that match the undesired speech with an obscuring audio signal that prevents the undesired speech from being intelligible; and

(d) replacing any portions of the audio data that match the undesired speech with related speech that is acceptable and is not included in the undesired speech, the related speech being produced using phonemes that were previously uttered by a speaker whose speech is being replaced.

7. The method of claim 2, further comprising the step of dynamically adjusting the probability threshold based upon a frequency with which undesired speech by a specific speaker is detected in the audio data, so that as the occurrences of undesired speech that are detected increases, the probability threshold is reduced.

8. The method of claim 2, further comprising the step of setting different probability thresholds for specific words or phrases included in the undesired speech, so that more objectionable words or phrases have a lower probability threshold than less objectionable words or phrases.

9. A memory medium on which are stored machine executable instructions for carrying out the steps of claim 1.

10. A method for automatically censoring audio data to prevent undesired speech included therein from being understandable by an audience who may be listening to the audio data, comprising the steps of:

(a) accessing grammar data and undesired speech data that are in a desired format selected to be usable for comparison to the audio data;

(b) processing the audio data to produce processed audio data that is in the desired format;

(c) automatically comparing the processed audio data to the undesired speech data, to identify any portions of the processed audio data that match any undesired speech included in the undesired speech data; and

(d) if portions of the processed audio data are found to match any undesired speech, altering the audio data to produce censored audio data in which each occurrence of undesired speech is made incapable of being understood by the audience, else if none of the audio data is found to match any undesired speech, the audio data are not so altered.

11. The method of claim 10, wherein the step of processing the audio data comprises the step of producing a lattice of possible phonemes of speech that may be included in the audio data.

12. The method of claim 11, wherein the desired format comprises words of speech and wherein the step of processing the audio data further comprises the steps of:

(a) producing a lattice of one or more possible words of speech that may be included in the audio data, the lattice comprising one or more nodes, each node being associated with a time interval in the audio data and a probability; and

(b) using the probability associated with each node, determining an overall probability that the one or more words included in the lattice of one or more possible words comprises an utterance that is likely included in the audio data.

13. The method of claim 12, wherein the step of automatically comparing comprises the step of applying a probability threshold to determine if any utterance indicated in the lattice of one or more possible words of speech matches a corresponding utterance in the undesired speech data.

14. The method of claim 11, wherein the desired format comprises phonemes and wherein the lattice of phonemes comprises one or more nodes, each node having associated therewith a time interval for the node and a probability that the phoneme is included in the audio data, and wherein the step of processing comprises the step of using the probability associated with node and the time interval to determine an overall probability indicating the likelihood that the phonemes are actually included in the audio data.

15. The method of claim 14, wherein the step of comparing comprises the step of applying a probability threshold to determine if phonemes included in the lattice of phonemes likely match corresponding phonemes in the undesired speech data so as to indicate that the audio data includes undesired speech corresponding to the phonemes.

16. The method of claim 10, wherein the step of altering the audio data comprises one of the steps of:

(a) substantially reducing a volume of any portions of the audio data that match the undesired speech so that the portions are not audible;

(b) deleting any portions of the audio data that match the undesired speech;

(c) overwriting any portions of the audio data that match the undesired speech with an obscuring audio signal that prevents the undesired speech from being intelligible; and

(d) replacing any portions of the audio data that match the undesired speech with related speech that is acceptable and is not included in the undesired speech, using phonemes that were previously uttered by a speaker whose speech is being replaced.

17. The method of claim 10, wherein the step of automatically comparing applies a dynamically variable probabilistic determination, wherein a probability threshold for determining that a portion of the audio data matches undesired speech is determined based upon at least one criterion relating to the audio data.

18. The method of claim 17, wherein the at least one criterion includes at least one of:

(a) an expected audience for the audio data;

(b) an identity of a speaker that uttered the audio data;

(c) a time at which the audio data will be heard by an audience;

(d) a type of event resulting in the audio data;

(e) an application of the audio data; and

(f) a frequency with which undesired speech by a specific speaker has previously been detected in the audio data.

19. The method of claim 17, wherein the probability threshold is reduced for a speaker, as a frequency of undesired speech detected in the audio data for the speaker increases.

20. The method of claim 17, further comprising the step of setting different probability thresholds for specific words or phrases included in the undesired speech data, so that more objectionable words or phrases have a lower probability threshold than less objectionable words or phrases.

21. The method of claim 10, wherein the step of processing the audio data is carried out using a speech recognition engine.

22. The method of claim 10, wherein the audio data are processed in one of:

(a) a batch mode wherein the audio data are processed offline; and

(b) a real-time mode wherein the audio data are processed as produced and just before being heard by an audience.

23. A memory medium on which machine readable instructions are stored for carrying out the steps of claim 10.

24. A system for automatically censoring audio data to prevent undesired speech included therein from being understandable by an audience who may be listening to the audio data, comprising:

(a) a memory in which the undesired speech data and machine instructions are stored and which at least temporarily stores the audio data;

(b) a processor that is coupled to the memory and able to access the audio data at least temporarily stored therein, the processor executing the machine instructions, causing the processor to carry out a plurality of functions, including;

(i) automatically processing the audio data to detect any undesired speech that may be included therein; and

(ii) for each occurrence of undesired speech that is automatically detected, altering the occurrence in the audio data, producing censored audio data in which the undesired speech is substantially no longer perceivable by a listening audience.

25. The system of claim 24, wherein the machine instructions further cause the processor to:

(a) generate a lattice of phonemes comprising word fragments that are likely included in the audio data;

(b) compare the word fragments against corresponding word fragments that are included in the undesired speech data, to identify potential matches; and

(c) based upon a probability of a potential match and a probability threshold, determine whether any undesired speech is included in the audio data.

26. The system of claim 24, wherein the machine instructions further cause the processor to:

(a) compare words in the audio data against words comprising the undesired speech, to identify potential matches; and

(b) based upon a probability of a potential match and a probability threshold, determine whether any undesired speech is included in the audio data.

27. The system of claim 25, wherein the machine instructions further cause the processor to dynamically vary the probability threshold dependent upon at least one criterion.

28. The system of claim 27, wherein the at least one criterion includes at least one of:

(a) an expected audience for the audio data;

(b) an identity of a speaker that uttered the audio data;

(c) a time at which the audio data will be heard by an audience;

(d) a type of event resulting in the audio data;

(e) an application of the audio data; and

(f) a frequency with which undesired speech by a specific speaker has previously been detected in the audio data.

29. The system of claim 25, wherein different probability thresholds are assigned to different words and phrases comprising the undesired speech, depending upon a relative degree with which the words and phrases are objectionable.

30. The system of claim 24, the machine instructions cause the processor to alter the audio by doing at least one of:

(a) substantially reducing a volume of any portions of the audio data that match the undesired speech so that the portions are substantially inaudible;

(b) deleting any portions of the audio data that match the undesired speech;

(c) overwriting any portions of the audio data that match the undesired speech with an obscuring audio signal that prevents the undesired speech from being intelligible; and

(d) replacing any portions of the audio data that match the undesired speech with related speech that is acceptable and is not included in the undesired speech, the related speech being produced using phonemes that were previously uttered by a speaker whose speech is being replaced.

31. The system of claim 25, wherein the machine instructions further cause the probability threshold to be dynamically adjusted based upon a frequency with which undesired speech by a specific speaker is detected in the audio data, so that as the occurrences of undesired speech that are detected increases, the probability threshold is reduced.